21 votes

Rant: Docker is a labyrinth maze of brick walls and show-stopping issues that has done nothing but slow my development

Firstly, I apologise for the rant. I guess this is a meek follow-up to my submission earlier in ~comp, questioning how to deploy Docker into production. Since then, I haven't been able to dedicate much time to solving any of the issues I've outlined in that thread, but what I will say is that docker has caused me nothing but pain, and I have realised zero benefits from attempting to utilise it. Right from the start, the syntax for docker, docker-compose, and Dockerfiles is confusing and full of edge cases which no one explains to you in the hype of actually discussing it:

  • These 'images' you build grow to insane sizes unless you carefully construct and regiment your RUN, COPY, and other commands.
  • Docker complains to you about leaving empty lines in multi-line RUN commands (which is itself, as I see it, basically a hack to get around something called a "layer limit"), even if it contains a comment (which is not an empty line) and does not provide a friendly explanation on how to solve this issue.
  • There's basically no good distinction between bind mounts and volumes, and the syntax is even more confusing: declaring a volumes entry in a docker-compose.yml? You have no good idea if you're creating a volume or a bindmount.
  • Tutorials & documentation tends to either assume you're a power user who knows this sort of thing, or are so trivial they don't accurately represent a real-world solution, and are therefore basically useless.

I've suffered endless permissions issues trying to run portions of my application, such as being unable to write to log files, or do trivial things like clearing a cache—that I have tried a dozen different ways of fixing with zero success.

Then, when I run some things from within the docker container, such as tests, they can take an excruciatingly long time to run—only then did I discover that this is yet another docker issue. The whole point of docker is to abstract away the host OS and containerise things and it can't even do that.

So now I'm regenerating and rebuilding images and containers every 5 minutes trying to find a configuration that appears to work with the slow and complicated syntax of docker rm $(docker ps -aq) -f followed by docker rmi $(docker images -q) followed by docker-compose -f docker-compose.yml -f docker-compose.dev.yml up -d, followed by docker container exec -it php sh.

Docker-sync, kubernetes, docker-compose, images, containers. It's legitimately too much. I'm not a dev-ops or infrastructure guy. I just want to write code and have my app work. I don't have the money to employ anyone to solve this for me (I'm not even employing myself yet).

I guess you can say I've learnt my lesson. I'm sticking to git and a simple VPS for future endeavours. I don't know how you folks who manage to hype docker do it, and I don't know what I'm doing wrong, but Docker doesn't like me, and I don't like it.

15 comments

  1. [4]
    spit-evil-olive-tips
    Link
    Yep, Docker is a minefield of confusing and contradictory and halfway-deprecated technologies. If you want to know the root cause of why - Docker, at its heart, is fundamentally rather simple. The...
    • Exemplary

    Yep, Docker is a minefield of confusing and contradictory and halfway-deprecated technologies.

    If you want to know the root cause of why - Docker, at its heart, is fundamentally rather simple. The true "hard parts" are all in the Linux kernel, things like cgroups and network namespaces and the overlay filesystem, which are all available to other containerization products like LXC and Google's lmctfy (which predate Docker) as well as rkt and podman and others (which came after Docker).

    So Docker the technology is (relatively) simple. Meanwhile Docker the company has received a quarter of a billion dollars in VC funding and has 100+ people employed at Silicon Valley salaries. That means they're under heavy pressure to monetize and show profitability.

    As a company, they're stuck between a rock and a hard place. Their ubiquity and adoption over other containerization options relies on them being free & open-source, just an apt-get install away. And yet the company itself is doomed unless they find something that's not free & open-source, and yet is valuable enough that people are willing to shell out $$$ for it.

    This has led to Docker flailing around, trying something new seemingly every month to see if they can get traction with paying customers. There's Docker Enterprise - which of course, you should not confuse with Docker Engine - Enterprise:

    Starting with Docker Enterprise 2.1, Docker Enterprise - Basic is now Docker Engine - Enterprise, and both Docker Enterprise - Standard and Docker Enterprise - Advanced are now called Docker Enterprise.

    There's also Docker's Universal Control Plane which is "the enterprise-grade cluster management solution from Docker"...but apparently different from Docker Enterprise and Docker Engine - Enterprise. Or you can use Docker Swarm as a control plane, which has apparently been renamed to Docker Engine - Community...I think.

    You might also need Docker Desktop Enterprise. Do you need Docker Desktop Enterprise if you're also using Docker Enterprise or Docker Engine - Enterprise? Maybe!

    All this has led to the free parts of Docker getting neglected. They change things in the open-source core, but really only to support the enterprisey features, and updating the documentation for the free stuff gets forgotten about because no one is paying for it.

    To answer some of your specific issues:

    declaring a volumes entry in a docker-compose.yml? You have no good idea if you're creating a volume or a bindmount.

    I did warn you in your original post to steer clear of docker-compose until you were more comfortable using Docker itself, for exactly this sort of reason.

    I've suffered endless permissions issues trying to run portions of my application, such as being unable to write to log files, or do trivial things like clearing a cache—that I have tried a dozen different ways of fixing with zero success.

    A big part of this is an impedance mismatch, where there's an "idiomatic" way of building a containerized app that you're not following. If you're used to writing Java and try your hand at Python, you can write Python classes for AbstractFactoryBuilders...but it'll be far from idiomatic Python and you'll encounter a lot of issues that aren't strictly issues with Python, but instead issues with how you're using Python.

    Permissions issues trying to write to a log file? Don't do that. If you blow away the container, you lose the log file, unless you do wonky crap trying to volume-mount the log directory...and don't do that either. Send logs to stdout.

    You mention "trivial" things like clearing a cache...but having Docker-related problems there makes me think you're trying to do non-idiomatic things like store mutable state within the container. Don't do that either.

    If you need a cache, run something like Redis in its own dedicated container, and have your (stateless) app connect to it. Likewise for databases. Postgres or MySQL or whatever should also run in its own container, and its data volume should get volume-mounted. In general, your webapp should be stateless (no volume mounts at all) and connect to other stateful containers.

    Docker overlay networks are great for this. The documentation sucks as usual, but a simple example helps illustrate it:

    $ docker network create test
    $ docker run -d --net=test --name=redis redis:5
    $ docker run -it --net=test debian:stable bash
    root@638138a3acd2:/# ping -c 1 redis
    PING redis (172.18.0.3) 56(84) bytes of data.
    64 bytes from redis.test (172.18.0.3): icmp_seq=1 ttl=64 time=0.069 ms
    
    --- redis ping statistics ---
    1 packets transmitted, 1 received, 0% packet loss, time 0ms
    rtt min/avg/max/mdev = 0.069/0.069/0.069/0.000 ms
    

    The network name test is totally arbitrary. DNS magic happens behind the scenes, so by assigning them to the same network, they can talk to each other using the container name as the hostname. Redis listens on port 6379, so I can tell my app to connect to Redis on redis:6379 and it Just Works. Ditto with Postgres or whatever other supporting services you want to run.

    docker rm $(docker ps -aq) -f followed by docker rmi $(docker images -q) followed by ...

    Part of the problem here is that you're blowing away all those images, including intermediate layers that Docker will cache and try to re-use the next time you build the image. That'll hugely increase your image build time...especially if it's already slower than it should be since you're on a Mac with Docker running inside a VM.

    7 votes
    1. [3]
      unknown user
      (edited )
      Link Parent
      By "logs", I mean the framework I'm using would write to a log file for its own debug/info data—say for example if you set Mail to be outputted to a log file instead of actually being sent. It's...

      By "logs", I mean the framework I'm using would write to a log file for its own debug/info data—say for example if you set Mail to be outputted to a log file instead of actually being sent. It's not something I'm doing intentionally. I am aware that in terms of the principles of containerization, you'd have a Redis container, a DB container if you need it, etc.

      2 votes
      1. [2]
        spit-evil-olive-tips
        Link Parent
        For things like that you can run a Mailhog container and have your app send real SMTP to it. Here's a blog post about using it from PHP.

        For things like that you can run a Mailhog container and have your app send real SMTP to it. Here's a blog post about using it from PHP.

        1 vote
        1. unknown user
          Link Parent
          Well, that was just an example, but the truth is that the framework I'm using expects to be able to write to its own internal log file, irrespective of my actions. It's encountering permissions...

          Well, that was just an example, but the truth is that the framework I'm using expects to be able to write to its own internal log file, irrespective of my actions. It's encountering permissions issues not because I'm violating the informal "one container = one unit" rule, but rather because of how permissions are transferred in bind mounts/volumes from the host system in Docker.

          2 votes
  2. [4]
    Akir
    Link
    One problem is that you are using docker for Mac. Docker is hot trash outside of Linux, because on other platforms it has to run on a virtual machine instead of being a simple container. If you...

    One problem is that you are using docker for Mac. Docker is hot trash outside of Linux, because on other platforms it has to run on a virtual machine instead of being a simple container.

    If you are working on a project with just yourself, I don't really recommend using docker in general. It's just another layer of complexity.

    Docker is only really useful if you have a team wit

    13 votes
    1. [2]
      SourceContribute
      Link Parent
      This has struck me as messed up ever since I started using Linux; other devs use Macbooks and companies seem to force it and mandate it for all developers (business people use Thinkpads or...

      This has struck me as messed up ever since I started using Linux; other devs use Macbooks and companies seem to force it and mandate it for all developers (business people use Thinkpads or whatever Windows-equipped laptops are around) and yet we end up deploying our software onto Linux servers. All the user-facing stuff is in HTML/CSS and it would make more sense to run an emulator for Mac or Windows on top of Linux to make sure the frontend stuff looks good in different browsers.

      Instead it's a struggle between Linux and OS X which kinda sorta looks like a UNIX if you squint your eyes a bit. Docker is just one symptom of using the wrong environment for development.

      5 votes
      1. Akir
        Link Parent
        I pretty much agree with you, at least when it comes from a software perspective. It's really irritating when tar fails to work because you forget that you need to use the BSD syntax. That being...

        I pretty much agree with you, at least when it comes from a software perspective. It's really irritating when tar fails to work because you forget that you need to use the BSD syntax.

        That being said, the hardware is great. They are very reliable and pleasing to work with, and things 'just work'. It gives a very good experience overall, especially when you are using an iPhone and/or iPad.

        That being said, I am halfway convinced that a good amount of the power behind the mac-for-developers bandwagon is good old fashioned silicon valley techbro culture. The same kind of thought that brought us 'biohacking' with Soylent products, the gig economy, and - my personal favorite - the .com bubble.

        2 votes
    2. unknown user
      (edited )
      Link Parent
      I use docker to test my workstation configuration (i.e. run it if I make a rather big change, which is seldom these days; also I have a copy of my config repo on GitLab where I exploit the free...

      I use docker to test my workstation configuration (i.e. run it if I make a rather big change, which is seldom these days; also I have a copy of my config repo on GitLab where I exploit the free CI), and find it useful, albeit not really usable. Configuration and command line syntaxes are bad, and stuff that you'd think is easy are hard: have a shell run from inside a container, kill or destroy (i.e. stop and delete) one, find where the heck stuff is kept on your hard drive. But for testing stuff that depends on the environment, it is useful. Unless you're on something like NixOS or GuixSD, there is simply no guarantee something that works on your system now will work elsewhere, or even a few days later. Somethink like Docker is useful. Bare LXC isn't portable and is rather difficult to deal with. Maybe Vagrant is a better alternative?

      Edit: Vagrant even has an LXC provider for Debian, so it's not only limited to VMs. Just learned this a moment ago.

      1 vote
  3. vakieh
    Link
    Bottom line: with great power comes great responsibility. This is in the same vein as uni students who grew up on Python complaining about how hard C is because they have to manage their memory...
    • Exemplary

    Bottom line: with great power comes great responsibility. This is in the same vein as uni students who grew up on Python complaining about how hard C is because they have to manage their memory themselves and getting annoyed at null terminated strings.

    Docker is a powerful tool - it will do what you tell it to. In order to have it do what you want it to do, you need to learn it - and this is not a rapid or simple process, because Docker does not solve easy or simple problems. If your problems are easy or simple to solve (i.e. with nothing beyond git [and usually people only use 10% of git anyways]), then Docker is a 3 tonne excavator when what you need is one guy and a shovel.

    If you decide you DO need the excavator, then approach it like you would a complex new skill. Take a course on it, or work through a textbook. Don't immediately try to apply it to a current problem you have, learn it first with minimal examples and progressively more complex tasks, then once you KNOW how it works inside and out apply it to your use case.

    As an educator I will also warn you that you get what you pay for. Documentation is generally not there to educate, it exists as a reference. Free online tutorials (including Youtube) are like rolling a dice - odds are they're going to be garbage, and the 1/100 that aren't don't disprove the rule. The cheaper sorts of online courses like Udemy are almost universally worthless. The best non-human resource for an IT professional is a working relationship with a tech college or uni or similar where you take a course every year or 2 to upskill and keep up with new things.

    4 votes
  4. [3]
    Comment deleted by author
    Link
    1. [2]
      Omnicrola
      Link Parent
      Lot of good information in this post. However this bit: From personal experience with H2, MSSQL, PCF, and docker, be picky about H2. H2 is great for prototyping and initial development. However...

      Lot of good information in this post. However this bit:

      And as far as local testing with data goes, I strongly recommend using an in-memory database such as H2.

      From personal experience with H2, MSSQL, PCF, and docker, be picky about H2. H2 is great for prototyping and initial development. However I've inevitably run into times where syntax differences between H2 and the production MSSQL required writing a different query for each environment. In and of itself this is not a big deal, but over the lifetime of the app it grows and becomes more overhead. So I recommend ditching H2 as soon as you can, get a copy of whatever the prod DB is running locally.

      2 votes
      1. [2]
        Comment deleted by author
        Link Parent
        1. Omnicrola
          Link Parent
          I also enjoy ORMs, currently doing a lot with Hibernate and Spring Boot 2. The issue we keep encountering is migration scripts, as opposed to actual application queries. We use Flyway for...

          I also enjoy ORMs, currently doing a lot with Hibernate and Spring Boot 2. The issue we keep encountering is migration scripts, as opposed to actual application queries. We use Flyway for migrations, which I highly recommend. And it has options for managing different scripts for different environments which allows for H2 lovely and Azure SQL everywhere else. However having to write double migration scripts every time we do a seemingly innocuous change got real annoying. Plus there's a lot of benefit in having the local env mimic production as closely as possible.

          1 vote
  5. SourceContribute
    Link
    That's how I feel with any software development to be honest. The difference is how often the interruptions to development and the amount of yak-shaving that needs to be done. Docker is just yet...

    That's how I feel with any software development to be honest. The difference is how often the interruptions to development and the amount of yak-shaving that needs to be done. Docker is just yet another complicated bit of machinery that slows down dev once in a while for me (in a previous company it slowed down development a lot).

    3 votes
  6. [3]
    mike10010100
    (edited )
    Link
    Honestly, I've never seen what was so horribly hard about Docker. I almost exclusively work in Docker and K8S in an enterprise setting and other than a few hiccups, it's led to far less confusion...

    Honestly, I've never seen what was so horribly hard about Docker. I almost exclusively work in Docker and K8S in an enterprise setting and other than a few hiccups, it's led to far less confusion over what software is running where and solves the whole "works on my machine" issue entirely.

    If we go down your list of complaints:

    These 'images' you build grow to insane sizes unless you carefully construct and regiment your RUN, COPY, and other commands.

    Or just use multi-stage builds. Then you have intermediate containers where all your fat compile-time resources are run, and production-grade end containers that will just be your binary and supporting files (or packages, or folders, or whatever).

    Docker complains to you about leaving empty lines in multi-line RUN commands (which is itself, as I see it, basically a hack to get around something called a "layer limit"), even if it contains a comment (which is not an empty line) and does not provide a friendly explanation on how to solve this issue.

    I mean it's a little specific, and from the Stack Overflow article you linked to, it seems like it was fixed. So...?

    In addition, the layer limit isn't really an issue with multi-stage Docker builds. You can also think about docker layers like single filesystem operations, like this article discusses:

    https://agileek.github.io/docker/2015/04/11/docker-layers/

    But I highly, highly doubt you will ever run into the 42 layer limit (not even sure if this is still even a thing any more) if you just want to "write code and have your app work".

    There's basically no good distinction between bind mounts and volumes, and the syntax is even more confusing: declaring a volumes entry in a docker-compose.yml? You have no good idea if you're creating a volume or a bindmount.

    This is where I will almost always default to Kubernetes over Docker Swarm or Docker Compose. So you're right in this respect. Although, TBF, you're also using Docker for Mac, which is trash, since it runs in a VM. This is also why your tests are running so slowly.

    Tutorials & documentation tends to either assume you're a power user who knows this sort of thing, or are so trivial they don't accurately represent a real-world solution, and are therefore basically useless.

    I learned by looking at examples, and there's no end to the examples of successful Dockerfiles on GitHub.

    TBF, your critiques are valid coming from someone who uses Docker for Mac exclusively.

    But most of this seems like you're just not willing to learn the lingo/research solutions. That's not to say that Docker is fantastic, it definitely has stuff to improve on, but a lot of your issues seem like non-issues to me. Docker isn't meant to be a quick and effortless solution to every coders' problem, it's a toolset all on its own.

    To me, it feels like you want a panacea, not a tool. And it also feels like you're trying to shoehorn your past experiences into a new system in a non-idiomatic way.

    Pay for a few Docker courses, look at some successful and popular Docker containers, and I think you'll change your mind a bit.

    2 votes
    1. [2]
      unknown user
      Link Parent
      This is honestly a bit mean. That’s not a good faith interpretation of my post. I’m definitely trying and am going to continue trying. Doesn’t mean I’m not frustrated however. And dismissing my...

      But most of this seems like you're just not willing to learn the lingo/research solutions.

      This is honestly a bit mean. That’s not a good faith interpretation of my post. I’m definitely trying and am going to continue trying. Doesn’t mean I’m not frustrated however.

      And dismissing my criticisms into “oh that’s Docker for Mac” seems like a good use of scapegoating. It’s a valid issue affecting an end user, the specifics behind the scenes are irrelevant.

      9 votes
      1. mike10010100
        Link Parent
        It's not scapegoating, it's simply a fact. Docker for Mac sucks because it's running in a VM. I understand your frustration, but, again, it seems to stem from the fact that you don't really care...

        It's not scapegoating, it's simply a fact. Docker for Mac sucks because it's running in a VM.

        I understand your frustration, but, again, it seems to stem from the fact that you don't really care how your infrastructure will work, you just want to write code and make it run. But even if you were to run serverless, you'd still need to understand how the service hosts your code and some other intricacies about how to best handle a sudden request increase.

        You are never not going to need to know something about the infrastructure your code is run on, unless you hire a team of people whose job it is to worry about that.

        1 vote