I'm currently building an MVP for a startup, solo. We've got Postgres pulling triple duty as the go-to database for all normal relational data, a vector database with pgvector, and a job queue (With the magic of SELECT ... FROM "Jobs" WHERE ... FOR UPDATE SKIP LOCKED LIMIT 1). Every time I go out looking for solutions to problems it feels like the world really wants me to get a dedicated vector store or to use Redis as a job queue.

Back when I was a Rails developer a good majority of the ActiveJob implementers used Redis. Now that I'm doing NodeJS the go-to is Bull which can only serialize jobs to Redis. They back this with claims that I can scale to thousands of jobs per second! I have to assume this theoretical throughput benefit from using Redis is utilized by 0.01% of apps running Bull.

So I ended up implementing a very simple system. Bull wouldn't have been a good fit anyway as we have both Python and Typescript async workers, so a simple system that I fully understand is more useful at the moment. I'm curious who else shares my philosophy.

Edit: I'll try to remember to update everyone in a year with the real world consequences of my design choices.

You could use any SQL database to create a few tables, insert some data, and do queries, while ignoring anything vendor-specific. But Postgres has a lot of other features and many extensions. What do you recommend checking out beyond the basics?

I've used a lot of databases, but it's been many years, so I assume things have changed quite a bit. I skimmed a few PostgreSQL release notes and learned that there's now a MERGE statement that looks pretty handy. (It's standard SQL.) And from Neon's list of supported extensions, the plv8 extension caught my eye. It would let me write stored procedures in JavaScript. Does anyone use that? Do you use stored procedures at all?

I wonder what Tildes uses?

(To keep discussion organized, please write about one feature per top-level comment.)

I recently discovered how much of a hassle it is to migrate off of Heroku Postgres. Both to keep the user from screwing themselves and as a vendor lock-in mechanism there's no possibility of running an external replica for your Heroku Postgres database. Unless I'm missing something (after a bunch of Googling) it seems like there is a market gap for a tool that allows for a seamless migration away from a Postgres DB that won't allow for replicas.

I imagine that, given some constraints on what queries you allow on your database, you could proxy connections to a managed Postgres database while feeding them simultaneously to a new database. Starting from a snapshot of the old database one could catch the new database up to a live state of the old database and then swap over all connections to the new one.

Does such a proxy already exist? I'd love to know as I could use it. If not it might be fun to build and a good side-project.