note that unset FLY_REGIONS["a100-40gb"] is incorrectly quoted: it gets expanded as a glob (like ls files.[ch]pp). for instance it will match a file called FLY_REGIONSa if it exist, so you’ll end...
I make great use of associative arrays (hashmaps) to reduce calls to an sqlite database I store static information in. I make an API call to return 10 values I care about for a data point. I store...
I make great use of associative arrays (hashmaps) to reduce calls to an sqlite database I store static information in.
I make an API call to return 10 values I care about for a data point. I store these values in the database for future use. I also store them in the associate array val_one[${dataPoint}]="${value_one}". By doing this, if I need that data point later in the script (or on a future run, in which I'm getting the data from the sqlite db instead of an additional API call), I can call function getValOne "${dataPoint}"
function getValOne {
if [[ -z "${1}" ]]; then
echo "give me a data point u luddite"
return 1
fi
if [[ -n "${val_one[${1}]}" ]]; then
# Value already assigned
return 0
fi
# No value assigned, get one from sqlite
val_one["${1}"]="$(sqlite3 [...])"
}
It's sped up my scripts a decent amount that have a large number of potential sqlite lookups.
Unfortunately, I've started having to pad my array elements with an underscore, as some of my data points are mixed strings of integers and letters. Every now and then a string beginning with an integer will be interpreted (I forget, it's either interpreted lexicographically and it shouldn't be, or it shouldn't be and it is. Either way, padding with a leading underscore fixed that.)
I could be wrong, but I suspect that Redis wouldn't really help. The reason is that, unless the query is very complex or the key being searched isn't part of sqlite's index, the actual lookup once...
I could be wrong, but I suspect that Redis wouldn't really help. The reason is that, unless the query is very complex or the key being searched isn't part of sqlite's index, the actual lookup once sqlite is actually running is probably dirt cheap. I'd assume the expensive part is invoking a ton of short-lived processes to perform each lookup. Being that it's being invoked from bash I'd assume just as many processes would be invoked to do the lookups from Redis.
Oh I see, so part of this is that SQLite isn’t running at all times? Then yeah if the same constraint is applied to redis I think you’re right. In terms of processes you could of course just run...
Oh I see, so part of this is that SQLite isn’t running at all times? Then yeah if the same constraint is applied to redis I think you’re right. In terms of processes you could of course just run one redis process that is single threaded, but it sounds like that may not fit your solution
Oh it wasn't my code, its just that the example code is bash containing $(sqlite3 ...) which indicates that it is executing the program named sqlite3 to perform the query. Bash is just not...
Oh it wasn't my code, its just that the example code is bash containing $(sqlite3 ...) which indicates that it is executing the program named sqlite3 to perform the query. Bash is just not particularly well suited to doing things other than invoking processes. So even if you had Redis daemon running I cannot think of a sane way to query it from bash other than each query executing an application that makes a connection to the daemon and then invokes the query through said connection.
SQLite also does thing pretty differently than from most other databases too. It isn't something you continuously run as much as being an interface for a database file. With most types of databases you typically have them running somewhere and then open a connection to that instance. With SQLite though an open call typically takes a file path (or equivalent). In a typical program you would normally like to open the connection or database one time and then it becomes rather cheap to reuse, but when you cannot do that you can get stuck paying the performance penalty of having an application that does all the setup, a single query, and then the teardown. The sqlite3 process is surprisingly comparable to something like jq, but for sqlitedb files rather than json files.
Anything would be a better drop-in solution. My primary job is a patient care facing role in healthcare, I'm pretty far from anything tech, IT, or development. I started tinkering around on Ubuntu...
Anything would be a better drop-in solution. My primary job is a patient care facing role in healthcare, I'm pretty far from anything tech, IT, or development. I started tinkering around on Ubuntu as a teen back in 2007, and all of my *nix experience comes from using it for hobby/passion projects over the years. I should have learned Python, or something, years ago. But I just didn't have the time or mental fortitude when I could cop out and be like "Eh, bash script".
I've never really used Bash in ways that having collections, beyond maybe looping through a flat array or something similarly basic, would be particularly useful so hadn't considered this. Prior...
I've never really used Bash in ways that having collections, beyond maybe looping through a flat array or something similarly basic, would be particularly useful so hadn't considered this. Prior to reading the piece I was assuming that it would be something insane like old autohotkey (https://www.autohotkey.com/docs/v1/misc/Arrays.htm#pseudo). And yes, that came to mind because I've had to write stuff like that because I used autohotkey extensively before the feature that's now recommended was added.
For anyone that can't or doesn't feel like reading that link, it would be kind of like saying you should implement all associative containers in JS, including arrays, with a syntax like window["my_container" + key]. So while I was very wrong, I was expecting someone to have written that they were interpolating the variable names themselves and calling it close enough.
note that unset FLY_REGIONS["a100-40gb"] is incorrectly quoted: it gets expanded as a glob (like ls files.[ch]pp). for instance it will match a file called FLY_REGIONSa if it exist, so you’ll end up unsetting the variable FLY_REGIONSa instead of index a100-40gb of FLY_REGIONS
you want something like unset "FLY_REGIONS[a100-40gb]"
Thank you for all these bash posts by the way. I'm loving it.
I make great use of associative arrays (hashmaps) to reduce calls to an sqlite database I store static information in.
I make an API call to return 10 values I care about for a data point. I store these values in the database for future use. I also store them in the associate array
val_one[${dataPoint}]="${value_one}". By doing this, if I need that data point later in the script (or on a future run, in which I'm getting the data from the sqlite db instead of an additional API call), I can call functiongetValOne "${dataPoint}"It's sped up my scripts a decent amount that have a large number of potential sqlite lookups.
Unfortunately, I've started having to pad my array elements with an underscore, as some of my data points are mixed strings of integers and letters. Every now and then a string beginning with an integer will be interpreted (I forget, it's either interpreted lexicographically and it shouldn't be, or it shouldn't be and it is. Either way, padding with a leading underscore fixed that.)
Ha, I both love and hate this. Redis would probably be a pretty easy drop-in solution?
I could be wrong, but I suspect that Redis wouldn't really help. The reason is that, unless the query is very complex or the key being searched isn't part of sqlite's index, the actual lookup once sqlite is actually running is probably dirt cheap. I'd assume the expensive part is invoking a ton of short-lived processes to perform each lookup. Being that it's being invoked from bash I'd assume just as many processes would be invoked to do the lookups from Redis.
Oh I see, so part of this is that SQLite isn’t running at all times? Then yeah if the same constraint is applied to redis I think you’re right. In terms of processes you could of course just run one redis process that is single threaded, but it sounds like that may not fit your solution
Oh it wasn't my code, its just that the example code is bash containing
$(sqlite3 ...)which indicates that it is executing the program namedsqlite3to perform the query. Bash is just not particularly well suited to doing things other than invoking processes. So even if you had Redis daemon running I cannot think of a sane way to query it from bash other than each query executing an application that makes a connection to the daemon and then invokes the query through said connection.SQLite also does thing pretty differently than from most other databases too. It isn't something you continuously run as much as being an interface for a database file. With most types of databases you typically have them running somewhere and then open a connection to that instance. With SQLite though an open call typically takes a file path (or equivalent). In a typical program you would normally like to open the connection or database one time and then it becomes rather cheap to reuse, but when you cannot do that you can get stuck paying the performance penalty of having an application that does all the setup, a single query, and then the teardown. The sqlite3 process is surprisingly comparable to something like jq, but for sqlitedb files rather than json files.
Good point, I forgot that’s how SQLite works.
Anything would be a better drop-in solution. My primary job is a patient care facing role in healthcare, I'm pretty far from anything tech, IT, or development. I started tinkering around on Ubuntu as a teen back in 2007, and all of my *nix experience comes from using it for hobby/passion projects over the years. I should have learned Python, or something, years ago. But I just didn't have the time or mental fortitude when I could cop out and be like "Eh, bash script".
I've never really used Bash in ways that having collections, beyond maybe looping through a flat array or something similarly basic, would be particularly useful so hadn't considered this. Prior to reading the piece I was assuming that it would be something insane like old autohotkey (https://www.autohotkey.com/docs/v1/misc/Arrays.htm#pseudo). And yes, that came to mind because I've had to write stuff like that because I used autohotkey extensively before the feature that's now recommended was added.
For anyone that can't or doesn't feel like reading that link, it would be kind of like saying you should implement all associative containers in JS, including arrays, with a syntax like
window["my_container" + key]. So while I was very wrong, I was expecting someone to have written that they were interpolating the variable names themselves and calling it close enough.speaking of unexpected places where hashmaps are built-in, POSIX C has them