5 votes

Getting frustrated with git and Heroku

So, I am new to git and Heroku. I have successfully deployed an app to Heroku using git (because Heroku requires git for the free tier). I'd like to start alpha testing soon. However, I don't think I'm quite ready since I don't understand how the interaction between git, Heroku, and my local machine works.

For example, I'll visit the live app at Heroku and make some changes to the sqlite3 database. Later, I'll make a change to one HTML file on my local machine. Then I'll use git add, git commit, and git push heroku master in order to update that HTML file on the live app.

What appears to be happening is all of the files are being rebuilt from my local machine and redeployed at Heroku which wipes any changes that I've made to the sqlite3 database.

How do I get around this?

13 comments

  1. Adys
    Link
    I'm not sure what you're doing here but you shouldn't be committing your sqlite database to git / sharing it between your local installation and your heroku installation. I haven't used Heroku all...

    I'll visit the live app at Heroku and make some changes to the sqlite3 database

    I'm not sure what you're doing here but you shouldn't be committing your sqlite database to git / sharing it between your local installation and your heroku installation.

    I haven't used Heroku all that much but you most likely want their hosted database product. https://www.heroku.com/postgres

    7 votes
  2. [4]
    teaearlgraycold
    Link
    Your database should be independent of your code. You should use a postgres server locally and in production. Any data that is necessary for server operation should be stored in a seed file...

    Your database should be independent of your code. You should use a postgres server locally and in production. Any data that is necessary for server operation should be stored in a seed file (basically a script that you can run to insert starting data). You’ll run that once when you set up the app locally and once when you set it up on Heroku.

    Heroku expects everything in your git repository to be stateless, i.e. you don’t use it to store any new data after deployment.

    4 votes
    1. [3]
      suspended
      Link Parent
      I just found this link. If it is that easy, then I'll definitely switch over.

      You should use a postgres server locally and in production.

      I just found this link. If it is that easy, then I'll definitely switch over.

      1. [2]
        teaearlgraycold
        Link Parent
        Well you also need to switch over your database driver in django and install postgres onto your computer. You’ll need to configure your app to play nice with how Heroku will expose the postgres...

        Well you also need to switch over your database driver in django and install postgres onto your computer. You’ll need to configure your app to play nice with how Heroku will expose the postgres database addon to you. I could see it taking a couple of hours to figure out the configuration changes.

        2 votes
        1. suspended
          Link Parent
          Thanks for the heads up.

          Thanks for the heads up.

  3. [8]
    stu2b50
    Link
    I'm a little confused about the details, so I'll try and fill in the gaps with assumptions. You likely have your sqlite database in your .gitignore (as you should), so it isn't tracked by your git...

    I'm a little confused about the details, so I'll try and fill in the gaps with assumptions. You likely have your sqlite database in your .gitignore (as you should), so it isn't tracked by your git repo. If you were to manually make changes to that file, then when you deploy it, Heroku is just going to do a pull and Django will recreate a blank sqlite database when you try to connect to a file that doesn't exist.

    One solution is that really that's not something you're "supposed" to do - locally modify your copy of the database for production use. If you were changing the schema, you should do it through something that tracks migrations (Django will do it for you so long as you use their ORM). Then python3 manage.py migrate will apply any schema changes that have yet to be done.

    If you were adding rows to the table, you should do that through Django's admin page.

    You could also re-add .db files from your .gitignore, but that's not a good idea, or, and I don't know what level of access Heroku gives you on the free tier, use one of scp/sftp/ftp/sshfs to move your local .db file to the server.

    But eventually you're going to have to move from sqlite3 to a "real" production database (mainly because sqlite only has single-threaded writes - it actually performs quite admirably for reads) where it's no longer tracked in a nice local file, so the last two options are just bandaids.

    Begin mildly offtopic rant: this is why I always tell beginners to start with just a linux server that they manage themselves. You definitely save some time in the beginning, but almost inevitably you deviate from the happy path to the sad path and then you have to spend more time than you saved trying to get back on the happy path understanding the black box that made it fast in the first place.

    3 votes
    1. [7]
      suspended
      Link Parent
      I may just have to get away from Heroku before I start alpha testing. This git stuff is frustrating. I just want to use FTP. Any advice?

      this is why I always tell beginners to start with just a linux server that they manage themselves.

      I may just have to get away from Heroku before I start alpha testing. This git stuff is frustrating. I just want to use FTP. Any advice?

      1. [2]
        teaearlgraycold
        Link Parent
        You don’t seem to be doing anything wrong with git. You are confused with the expectations of a container-like environment. Yes, you are deploying your code to a virtual machine. Yes, that virtual...

        You don’t seem to be doing anything wrong with git. You are confused with the expectations of a container-like environment. Yes, you are deploying your code to a virtual machine. Yes, that virtual machine has a disk you can write to. But that virtual machine is ephemeral. You should act as though it has no disk. You shouldn’t even think of it as a virtual machine. Heroku expects you to see it as your web application and nothing more.

        5 votes
        1. suspended
          Link Parent
          That makes sense. It's completely different from what I was used to many years ago.

          That makes sense. It's completely different from what I was used to many years ago.

          2 votes
      2. [4]
        stu2b50
        Link Parent
        If you're using Git basically as a glorified way to transfer files, then I can see how it seems like a bunch of pointless minutia, but it is important to keep using some kind of version control,...

        If you're using Git basically as a glorified way to transfer files, then I can see how it seems like a bunch of pointless minutia, but it is important to keep using some kind of version control, and git is the de facto version control of modern times. As a self plug I have a blog post on using git. I would keep at it.

        But indeed, if you switch to a VPS, then what i would recommend is to mount it as a virtual drive via SSHFS. Then, you can just pretend it's another harddrive connected to your local computer. You can use your local terminal to interact with it, you can open whatever local IDE/Text editor you use, etc. But I would still recommend using git. However, this does mean you can, say, move files easily in whatever file explorer you use in the midst of development.

        You can also use FTP/SFTP/etc. but it's going to be more cumbersome.

        1 vote
        1. [3]
          suspended
          Link Parent
          Jesus! I didn't know this existed. This is right up my alley. Thanks so much. I've seen quite a few people here recommend Digital Ocean as a VPS. Would you agree?

          But indeed, if you switch to a VPS, then what i would recommend is to mount it as a virtual drive via SSHFS.

          Jesus! I didn't know this existed. This is right up my alley. Thanks so much.

          I've seen quite a few people here recommend Digital Ocean as a VPS. Would you agree?

          1. [2]
            stu2b50
            Link Parent
            DigitalOcean or Linode are both good suggestions. If you want to dip your toes in without paying anything yet, both consistently offer $50-$100 promotional credits. Here's one for Linode from a...

            DigitalOcean or Linode are both good suggestions. If you want to dip your toes in without paying anything yet, both consistently offer $50-$100 promotional credits. Here's one for Linode from a podcast advertisement I listen to (note: not related in any way, it's just the one I remember) for $100 in credits, which can power the $5/month tier for 20 months.

            I will say that DIgitalOcean has some of the best tutorials around and googling "DigitalOcean <thing I want to do>" is almost certainly a good way to get a detailed guide on how to accomplish what you want. For example, here's one for your exact situation (a django app https://www.digitalocean.com/community/tutorials/how-to-set-up-django-with-postgres-nginx-and-gunicorn-on-ubuntu-18-04)

            The initial setup tutorial is also good to get a sudo user and ufw turned on: https://www.digitalocean.com/community/tutorials/initial-server-setup-with-ubuntu-18-04

            4 votes