What's a simple, cheap way to run a database-backed website as a hobbyist?
I use Github and Netlify to run some simple websites for free. It works well. However, I've been thinking of experimenting with a database-backed website for fun and Netlify doesn't have any persistence.
What's a good way to do this that scales to zero when nobody's using it? I want to be able to forget about it entirely for months or years at a time. When someone visits, it should start up and run on demand without costing me $20 a month on standby.
Back in the day, I used Google App Engine for this. I learned a lot of datastore tricks to get around its poor latency, but I'm lazy and don't want to do that anymore. I'm pretty sure I want a SQL database and full text search. Either sqlite or Postgres would do, but I doubt there's a cheap enough way to run Postgres.
Litestream looks interesting and so does LiteFS, except that it's pre-1.0 and I don't know what changes fly.io will make that I have to keep up with. If I used Litestream, I'd have to figure out how to run it and where to store the replication logs.
Edit: one nice-to-have is being able to easily dump the database and run it locally or on another cloud provider. (I don't anticipate it getting so big that it's impractical.)
My recommendation is to learn Nix and deploy to a cheap Hetzner Cloud VPS ($4/month, I think) using NixOS.
This is how I deploy https://notado.app, and I run the DB on the same VM, taking daily backups.
I went from paying $100+ on DigitalOcean to paying nothing deploying everything to a single server (kinda; I already have a dedicated servers that I deploy services on, so no "dedicated" infra anymore).
I wrote an article about moving off of DO, but that too is outdated now as I'm no longer using fly.io.
If anyone wants a pointer on handling backups with Nix, this is how I handle the Postgres backups for Notado to GDrive:
And this is how I run the database (via Docker)
I honesty don't think the added complexity and mental space required to maintain on-demand scaling solutions for database-backed services is worth it. Just pay the $5/month and keep it running; your sanity is worth way more than those $5 potentially saved.
Thanks for the suggestion! It sounds like it works well for you.
However, I'm wondering what's the complexity and mental overhead of learning and using Nix? Is that fun? I've heard it's not fun and I have a fear of inscrutable build systems, having resolved to never use Gradle again if I can possibly avoid it.
But then again people have different ideas of fun. It sounds like you liked fly.io at the time. Why did you stop?
How does Nix compare to, say, Vagrant on the fun-o-meter? I haven't used that either. I can look at both and say "that is definitely a configuration file." It can't possibly be as bad as BCL, though.
Over-analyzing how to cloud host a database is my idea of fun, apparently. I may never deploy anything at this rate.
I found learning Nix fun because I knew what the end result would be (infra heaven). I actually started using Nix to manage my personal machine first because I was disappointed by various dotfile managers and fresh install init scripts, and then moved on to using it to manage my SaaS services.
I stopped with fly.io because at the time I was still hosting the database on DigitalOcean and using a Tailscale tunnel to expose it privately to the fly.io pod. After cutting out Kubernetes, DigitalOcean Postgres was my next biggest cost. However, fly.io's Postgres service is not really a managed service; you have to handle everything yourself as if you were running it on your own, and at this point it made sense to me to just manage everything with NixOS if I was going to be self-managing the database.
Nix's fun rating is off the scales for me. It's the first time since I was first exposed to the early versions of Terraform where I have the feeling again of the impossible being made possible.
If you want a gentle introduction without changing your build system, just build and push your Docker images / artifacts as you normally do and orchestrate them using NixOS (see the example above of how I define the Postgres container). There is a lot of talk of Nix as a build system, but I find that it's probably the best infrastructure-as-code tool for single-VM environments.
As a counter-point, I have NixOS running on my laptop, homelab, and am running Nix on my work laptop and have been for about half of a year. I, uh, hate Nix. I am probably going to be migrating off of it in the near future for everything.
For around a month, Nix was broken on my work macbook. I hadn't changed any actual configuration but home-manager had introduced a commit that required a nixpkgs-unstable update. However, something was broken about the mechanism to update my nix channels and so home-manager wasn't getting what it would need to run - which meant none of my packages would update.
It took me two hours to fix the problem. I ended up having to nuke my entire Nix installation. To note, there isn't really an official uninstall process for MacOS Nix and in doing so I encountered four different reported bugs. Two in Nix, two in home-manager. My configuration had not changed whatsoever and I never knew what had caused Nix to stop working. What really sucked was that while I was reinstalling Nix (which again was bugridden and awful), none of my development environment was available because I had installed everything through Nix.
But let's talk about my homelab next. As of right now, my homelab cannot update. This is because I need Linux kernel version 6.2+ to support the Intel Arc card I got for Jellyfin video transcoding but I also need ZFS for the storage array. ZFS is currently broken on kernel 6.3. NixOS aggressively removes support for EOL kernel versions so when I try to update, well, anything on the system then Nix will fail to resolve because it will error out on applying the kernel version. There are no partial applications so every aspect of my homelab is unable to be applied via code. I wrote systemd unit files, configuration files, directories, etc. I can't change ANY of it unless I build my own kernel from scratch - which AFAICT heavily discouraged.
One of Nix's selling points is supposed to be reproducibility; you write your configuration and it can then reproduce your system. My personal experience with this is that your configuration can remain the same but instability in nixpkgs will just leave you with a broken mess.
On a day-to-day experience with Nix, it feels awful and clunky to use. Whenever I try to translate something that I could easily accomplish with a regular package manager, Ansible, or even just shell scripts then I find myself wasting hours and hours trying to do something that really should be simple and straightforward. If I was required to use Nix professionally at work then I'd immediately quit. I have talked to a couple of coworkers and they honestly feel the same.
Nix-the-language is not very fun (and I say that as someone who's used NixOS as my daily driver for more than 5 years). I like it better than Gradle but it still gives me flashbacks to the Haskell hazing I went through in my programming languages class in college.
but, using NixOS doesn't require in-depth knowledge of Nix-the-language, as long as you stay on the beaten path of using existing options and packages (and that beaten path is very wide). as long as you're doing that it feels much more like editing a config file rather than writing a functional programming language.
I envy you that you find Gradle inscrutable. The horrors I've seen in chains of makes, cmakes, configures... Gradle is a pleasure heh
Well, those can be bad too, but at least you can usually run 'make -n' to find out what shell commands it would run.
The thing that broke me was digging into the Java implementation of Android build rules. Sometimes there are no shell commands, just more unexplained Java code.
You could run a DigitalOcean low-tier Droplet for $4-6 a month, depending on your resource needs. Not zero, but really not that bad.
And I just remembered, I think Vercel has a pretty generous free tier for their services.
This is what I would suggest as well, you can dockerize your project and even run the DB and the project in separate containers. If you do this, you’ll have no problems moving to virtually any other provider later on. For small personal projects you can use a simple cron job to take a db dump and move that file to another VPS, or to some other cloud data storing platform (or a local raspi…)
I see a lot of comments regarding choosing various cloud services here with their proprietary solutions, those almost always bite you in the butt sooner or later. Either when they close down or remove their free / cheap tier, or when you need mor capacity.
Cloud providers often use the free/cheap tier to get you in, but as soon as you need capacity costs go up 10-100x.
When you’re dockerized you can just move your project very easily to a different VPS, or set it up on one of the big cloud services docker products (or their vps services), or on your own hardware.
Never forget that your own time is valuable, even if it’s just a hobby project.
Docker can be useful but it only solves part of the problem. The actual data isn't in the Docker file.
For a database migration, I think being able to do a dump and restore is probably good enough as far as migration goes. So, that means a popular open source database like sqlite or Postgres (or MySQL, but I'd rather not) is probably a good idea, but I don't think the server-level configuration needs to persist across a migration?
Dockerizing your projects requries some up-front work, but it will guarantee save you time (if your project ends up living for 1+year)
Every project I've ever been involved with has migrated from one hosting to another. Sometiems voluntarily, sometimes not.
If you run your project directly on a VPS, you will need to set up the entire instance when you move, but when your dockerized - it's a lot more streamlined.
Often when you have to move from one host to another, it is at a very inconvenient time. Being able to speed that up significantly is very important.
Re: DB backups, you can also always set up two DB instances and just keep a slave/replication, but that will still need proper offline backups (e.g. dumps), as replicated copies will just replicate the crap data.
Yeah, makes sense. Docker means you don’t have to upgrade software if you don’t want to, but you still need to move.
My expectations were set by App Engine where I’ve had projects run for a decade without touching them. For one of them, Google has given me plenty of notice that I need to upgrade to a newer version of Go, but I’m lazy and the project isn’t important.
If you used the wrong API, though, you’d be out of luck when they deprecated it. (With plenty of notice.)
Github and Netlify have never made me upgrade anything, but that’s for static websites.
This is the option I've almost always gone with. It's one of those things that's cheap enough to have idle, flexible when it comes to scaling, and the product is just a linux box. There are some nice templates for preinstalled stuff, and some little ease of use features like an online console and backup/recovery features if you want to attach those, but ultimately it's just a blank linux box that works exactly like if you set it up locally. I find that really eases moving stuff online, since I usually build stuff out first in docker images or on Raspberry Pis on the local network. Using DigitalOcean you're never asking "How would I do this on DigitalOcean?" because there's nothing especially proprietary or in-house about what they're selling you--it's just the way you'd do it on dedicated hardware in general.
Databases seem a little different though, because if you do it yourself you need to worry about backups.
It depends on how aggressively you want to back up. For stuff like my blog, I just have the whole "droplet" take a snapshot of itself once a week, which is just a toggle in DigitalOcean's interface. That just grabs absolutely everything, database and all, for an extra 20% of the price of the droplet per month. On other projects where I'm more concerned with not losing a week's worth of data then ya, it is a roll-your-own solution.
Still, it may not be the solution for you since it will never scale to zero, and the snapshots do add to the cost on top of the hosting.
I currently use DBHub.io for sqlite since it's free, open source, and has an API. It's Patreon-supported at the moment
Edit: I tried using fly.io's free tier and ended up getting charged for spinning something up. Luckily their support was nice about it, but left a bitter taste. A proper free tier should prevent users from taking actions that cost money.
This seems to be "github, but a database." I see that executing a SQL statement is marked "experimental" in the API. There is an "Upload" API call to entirely replace the database with a new version, and a UI for making changes. Is that how you use it?
Dolthub is another product that's similar. I think it takes the concept further since it does merges, but they won't host a private database for free.
Ah, I forgot mine is a read-only database which is easy to implement with their query endpoint.
How much experience do you have with server administration? Oracle cloud's free tier might fit your bill if you're willing to invest a little bit of time in learning how to install whatever software you need and harden the server itself. I have a couple of things running on two of their ARM instances, and haven't had any real issues. Seems like the perfect solution if you can handle the self-managed aspect.
I ran StadiaDB on Firebase w/ Firestore.
It cost me ~$0.04/mo. in total, which went towards creating regular backups.
Without Firebase Functions, it would’ve cost me literally nothing (though $0.04 is nothing, really).
Firestore is the successor to Google AppEngine's datastore, so it would be pretty familiar. However, it doesn't support full text search and Google suggests third-party products.
I have some experience with Oracle. I even worked for them a very long time ago. I don't want to do server or database administration. Creating a few tables and indexes is what I'm interested in.
Oracle’s always free tier includes their Autonomous Database and NoSQL products. I haven’t used those, but I’ve been using their free compute instances and Kubernetes services for a few months now and so far they’re legit.
I think Oracle's is the most generous free tier out there. Very good for hosting hobby projects.
I have a Hetzner VPS on the cheapest tier. You have to setup your server and database yourself, but it costs less than 5€/month and is reliable.
I'm a big fan of Nearly Free Speech.Net, which supports MySQL databases. It doesn't scale to zero when nobody is using it, but it's extremely cheap in general.
My personal website hosting (which doesn't use a database) costs <$10 USD/yr.
Depending on the amount of data you need to store in your database, it might suit for your purposes.
Looks like they do daily backups and they recommend that you make your own backups.
I like using cloudlfare d1, along with all the code I deploy on workers etc.
There's a 100MB limit right now, but it's a simple sqllite I don't have to think about right now, and most of my apps won't reach that level anyways.
https://developers.cloudflare.com/d1/
I like Cloudflare a lot. Cloudflare Pages/Workers are an excellent free opinion if you don't need a db, or if you're happy to use their KV service (eventually consistent) instead of a db.
If you are up for paying $5/mo, you get D1 (like SQLite in the cloud) and Durable Objects, which are pretty neat. They can even support Websockets in a Serverless(tm) way.
In process of migrating worker to pages for a second project and again I'm just disappointed at the dx. No wrangler.toml, and in termination it doesn't close the vite process leaving the port in use.
Yeah definitely, DO and D1 pretty interesting. Do you know if it's possible to run a discord bot using the websocket in server less?
It says "Open Alpha" but I guess for playing around it would be okay? Backups are explained here.
If you're willing to try NoSQL, MongoDB Atlas is free, but they will halt your instance if it is not used for awhile. You could get around this by somehow sending heartbeats every few days to the db, like pulling some data from it without a purpose.
For hosting, Vercel is also free, and works very well with NextJS. While NextJS supports a backend interface, if you want to use something else for your backend like python, then like other commenters mentioned AWS Lambdas are probably your best bet. There are tools out there that can make FastAPI for python severless, and compatible with lambda.
If you have any special compute requirements, good luck you aren't scaling to zero like that since the start up time would be too egregious. Sounds like that won't be an issue for you though, so I recommend vercel with either Postgres or MongoDB atlas.
I'm going to skip NoSQL this time.
One of the things I'd like to have it easy migration to a different database. I see Vercel runs Postgres with a large number of extensions. I guess it's okay if I'm careful not to use them, but it might make it tricky to move?
Looks like Vercel's Postgres is actually provided by Neon. The also integrate with Suprabase, which also runs Postgres.
Vercel is a serverless hosting platform. You can have a normal database with them but you'll be constrained to only serverless backends.
That might be okay.
I probably should have said something about the architecture I had in mind. I'm thinking some kind of auto-managed way of responding to an API call from browser code by running a JavaScript function, which runs SQL transactions against a database. Deno deploy seems kind of appealing, but maybe sticking to Netlify makes more sense since I already have an account and it will do serverless functions?
If I went with Vercel, I guess I could do it there instead? Or are you saying I have to put the JavaScript backend tier there?
For the database itself, I want my website to be able to execute SQL statements in transactions with a reasonable API and to be able to do a database dump if I need to migrate the database. I don't want to manage the servers, just like I don't manage the servers for a Google spreadsheet.
If you use Vercel I think you might as well use its severless hosting. But you don’t need to. You could use their cdn hosting alone and call out to some other serverless host.
Also - the alternative to serverless isn’t managing a server. You could do that with a VPS. But the most popular way would be to manage a docker file. But in your case it sounds like serverless does what you need.
I see Vercel gives you one free Postgres database and additional ones are $1/month each.
I took a closer look at Neon. They are a very new company and are still in "Technical Preview.". (It's open source and written in Rust, for what that's worth.) They're running in a few different AWS regions. One neat thing about it is that you can branch a running database, for doing development with a production database. It seems like a fun project as databases go.
They will give you Postgres databases starting at $0 and charging for usage. They auto-suspend when there's no activity. If it were active 24/7 then it looks like it would be around $20/month. Looks like you need a Pro plan (enable billing) to have more than one project.
It seems Vercel has Neon integration, which would let you use Vercel and Neon with Neon's billing instead of Vercel's billing?
Netlify doesn't seem to talk about Neon or vice-versa, but Netlify does have a tutorial for using Suprabase, which is another Postgres database provider. Suprabase supports two projects in its free tier and it's $25/month after that.
Seems like a good deal. My personal preference is something a bit more oldschool than edge computing - like Render. They also have a free tier. And if you start paying you can at least manually shut down your services (without deleting anything) and they'll charge you $0 while they're off.
To look at this from another angle, maybe it would be better to see what some reasonably modern and actually portable software requires if you self-host?
Ghost’s requirements are here. You can run it on DigitalOcean for $5 a month. There is also a Docker image.
They require a full Linux instance (Ubuntu) and they recommend mysql_secure_installation which is a command you run to lock it down. To back it up, you run the ghost backup command, which creates a zip file. This seems quite reasonable for a blog.
I guess the idea here is to say that databases aren’t that special and you back it up by copying a file.
So, that’s a vote for “just use DigitalOcean.” I’ve done that before and I still have an account. Not wild about paying $5 a month when it’s not in use.
If you’re considering Digital Ocean, I’m not sure why you wouldn’t use Oracle’s free tier that’s been suggested upthread. You’ll get far more resources for free than with the $5 droplet. Maintaining a droplet would be almost exactly the same as maintaining a VPS on Oracle’s cloud infrastructure. You just won’t pay for it… and since it seems like you want it as cheap as possible, it seems like that’s probably the best way to go?
I can understand if you’re wanting something completely managed. But digital ocean is not that. And there are better choices than DO, price-wise (Hetzner, Linode, OCI).
My reluctance is mostly about company reputation. My knowledge is hazy and out of date, but I think of Oracle as a company with a lot of scary lawyers and bad license terms that concentrates on enterprise sales, so it's not really for a hobbyist, while I see Digital Ocean as a familiar, competent, user-friendly company with decent prices that are a little higher than I would prefer to pay, but I can well afford. Hetzner and Linode I know little about other than being slightly cheaper, but a lot of people use them, sure.
They would probably all do fine at running a Linux VM if that's how I choose to do it. It's great that running a Linux VM has become so standardized and Docker makes it even easier to switch.
The cost thing is somewhat of an aesthetic preference. I think that a website that gets no traffic shouldn't cost any money and needing a database isn't a good excuse for taking up compute resources unnecessarily. Also, in some sense it hasn't been necessary since App Engine launched 15 years ago. I was a big fan at the time.
But App Engine is also rather inflexible and I should probably go with what's standard, even if it's a little inefficient.
Ah, that makes a lot of sense! For what it's worth, I despise Oracle... but also don't mind taking free things from them. Unlike AWS where it's easy to get yourself in trouble, Oracle only uses your credit card for fraud prevention purposes unless you upgrade to outside of the free tier. I've been running a couple of instances with absolutely zero problems, billing or otherwise. But I do completely understand the hesitancy! My production VPS lives on Hetzner, which is a well-reputed vendor, but they're a German company so pricing is in Euros. I used Vultr before them, which also had zero issues. Hetzner was just cheaper, for more resources.
Hope you figure out something that works well for you!
This is great! Thanks for the tip, playing around with it now.
There's some odd things I couldn't quite figure out like moving the ssh port — I don't like it sitting on 22 and being hammered on by the entire skriptkiddie world. The ingress rules in the security group don't quite work as I would have thought (ie editing the default ssh rule to do 443 in and 22 to the host just plain doesn't work).
It was also a surprise, in a good way, to see iptables running and it was easy to solve the problem above using a redirect rule.
Still can't quite figure out how to get mosh setup, it complains "mosh: Nothing received from server on UDP port 60001".
I read a dated Oracle blog post about static IPs being free so I assume that is still true? I couldn't see anything about it in the pricing calculator.
Yeah, my instances have static IPs.
You may have to open up the port for Mosh in the control panel. I believe this article should give you the general idea of how to do that:
https://www.oracle.com/webfolder/technetwork/tutorials/obe/cloud/compute/permitting_public_tcp_traffic_to_compute_instances/permitting_public_tcp_traffic_to_compute_instances.html
Thank you!
options i've used but only works with static data: sql.js-httpvfs
options i've considered but haven't used: aws aurora serverless v2 and cloudflare d1
Do you mean a database hosted on AWS? Which one?
DynamoDB On-Demand is practically free under 25GB (or you may be eligible for free tier) if you're fine with NoSQL.
Off the wall option: OCI offers a free autonomous database that comes with APEX low-code front end. With the always free tier cloud service, you get a small one perpetually. I run a web-based EPG that helps me navigate my 3,000 channels of IPTV.
Here's a link, if you want to take a look at a simple site that costs me $0 and scales to intermediate demand without issues.
http://epg.gc04.xyz
You could also run whatever app you want on a VM for front end and point it at the free database that can be used either as a standard relational database or as a json store.
"Oracle Autonomous Database" seems to be the new name of Oracle's proprietary database offering. I think I'm going to skip that one because I want to use an open source database engine.
I am so hopelessly out of date on this. I vaguely remember Oracle 7 and 8 and apparently they're up to 23c now.
Version 21 added "blockchain tables!"
Completely fair and I understand the objection. Free is free, however.
I used OCI for a minecraft server and it ran like a dream, much better than $20 servers I have paid for in the past, those ARM processors are so great
How much data do you need to store? You could consider the free tiers at one of the DBaaS' cloud offerings:
These do seem interesting and plenty for my needs. I like that they are just databases, so I could probably use them with Netlify?
Looks like Netlify serverless functions run on AWS lambda with a default region of us-east-1, and you don't get to change it in the free tier. Planetscale and CockroachDB give you a choice of AWS and GCP regions. (I'm going to stop looking at Yugabyte unless someone tells me I should.)
One thing I'm concerned about is how many databases you get. Assuming I start using this and like it, I'd like to have one database for each project and they will probably be tiny. The limits on storage probably aren't going to apply.
Planetscale gives you one "production branch" and they have a "an additional replica to resist outages." Not sure how that works. I guess "branches" are like nodes and you can create multiple databases in them?
Looks like CockroachDB can use multiple regions for the same cluster. (In the free tier?) Looks like you can have multiple databases.
I'd like a "don't make me think, it will always be up*" solution, and this doesn't do it as well as App Engine, but it seems like this would be fairly close.
For PlanetScale, you only get one database for free. CockroachDB Serverless on the other hand gives you a cluster which you can create multiple databases under which sounds like a better fit for your use case. I was also mistaken, they give 10GB storage and not 5GB under their free tier.
I also like that you can self-host CockroachDB if you ever wanted to go that route later. PlanetScale is based on Vitess which is open source, but you can't self-host an instance of PlanetScale yourself.
The one thing I do like about Yugabyte is that if you do eventually want to go the self-hosted route later, their open source offering includes all features and doesn't lock out features behind an enterprise paywall like CockroachDB does.
I like the idea of CockroachDB but self-hosting doesn’t seem like a practical option for what’s supposed to be a fun project. I think I either want something with multiple hosting options available so I can dump and restore the database, or I’ll just shut it all down if there’s any need to migrate.
I’m using a tiny Linode (some new company now) server. I’m only using it to host a Wordpress site, but there are tons of other applications. I think I pay $5/month typically.
A lot of people are posting very technical solutions but... Well, WordPress kinda meets at of those criteria? Sql based database, easy enough to migrate from local to a live site, and super quick and easy to set up. So if it's for a hobby it hits the mark and won't take up too much time.
Yes, that is a database-backed website, but I'm thinking more along the lines of something where you create your own tables and make your own website from scratch. Unless WordPress has features I don't know about? (I've never used it.)
A long time ago, I liked FileMaker as a user-oriented database tool. I don't know what people use nowadays. Airtable maybe?
It sounds like a Linux VM that's hosted pretty much anywhere is the tried and true solution, but for fun I thought I'd try something a bit more bleeding-edge. Here's an example web site that I built using Deno Deploy, the Deno Fresh web framework, and Neon for the Postgres database.
Despite using some new stuff, it's a basic web page with some counters that could have been built in 1995, back when a "route" was called a "CGI script."
It will take a couple seconds for a cold start if the database has shut down.