I bet that most of the infrastructure designed for redundant operations like k8s, docker swarm, mongodb's multi-node setup, etc. is not necessary for most deployments. In 99% a LAMP stack on a...
I bet that most of the infrastructure designed for redundant operations like k8s, docker swarm, mongodb's multi-node setup, etc. is not necessary for most deployments.
In 99% a LAMP stack on a single machine will get you to 10 million visitors a month. With backups you can recover from failure relatively quickly.
If you need more, you can split the database and apache into separate servers, maybe deploy load balanced apache servers.
Swap out PHP in this stack for your favorite language's web framework.
That's the complexity 99% of people should need, 80% of them will however deploy on 20 AWS machines with autoscaling, swarm or k8s and all the other things they read about in medium blogposts.
I'm reminded of an old Dilbert strip that was taped to my office door. I'd simply change 'open source' to 'cloud' and 'free' to 'scalable' in this context. :P
I'm reminded of an old Dilbert strip that was taped to my office door. I'd simply change 'open source' to 'cloud' and 'free' to 'scalable' in this context. :P
I realized a bit ago that I would build projects for the joy of the infrastructure and tooling around them and not for the love of the project I was working on. Realizing that changed both how I...
I realized a bit ago that I would build projects for the joy of the infrastructure and tooling around them and not for the love of the project I was working on. Realizing that changed both how I approach problems as well as the types of problems I work on.
This reminds me of a well-known xtranormal video on MongoDB. Programmers (we're not engineers, please stop making real engineers look bad) like to use the latest, coolest new tech because it's the...
This reminds me of a well-known xtranormal video on MongoDB. Programmers (we're not engineers, please stop making real engineers look bad) like to use the latest, coolest new tech because it's the latest, coolest new tech. That's why many of us become programmers. I was there once*. It happens so much because so few programmers and sysadmins are able to step back and realize that the new fad technology rarely acutally provides any value to the end user. Sure, build your hobby projects with all the cool toys if you want, but for actually delivering value to your users, keep it simple.
I used the word "fad" deliberately, because the tech industry is exactly like the fashion industry in this way. That xtranormal video is at least eight years old, based on a timestamp of a reddit post linking to the original site. Eight years is an eternity in the tech world. Those cool fashionable Docker containers you use for everything? FreeBSD did it in 2000, and IBM did it 30 years before that. I'm only 35 and I've seen the fads in the tech industry cycle a few times already. If you've not seen it yourself, you're either very new (which is perfectly okay!) or not paying attention (which is less okay).
Now, this is not to say that you shouldn't design the system to be scalable should you need it to be, but that's an entirely different discussion.
* It was only once, because my boss at the time showed me I was being dumb and I learned from it.
I think there is going to have to be a split on this at some point where there are software engineers and software "technicians". A code boot camp graduate churning out web GUIs in the flavor of...
Programmers (we're not engineers, please stop making real engineers look bad)
I think there is going to have to be a split on this at some point where there are software engineers and software "technicians". A code boot camp graduate churning out web GUIs in the flavor of the month JS framework isn't an engineer. On the other hand, there are a small number of developers writing serious, robust, safety critical software that probably do deserve the title of engineer.
This deserves a separate thread, but I'd like to quickly justify and summarize my objection based on the definitions given by the IEEE. The issue I take with labelling programmers engineers is...
This deserves a separate thread, but I'd like to quickly justify and summarize my objection based on the definitions given by the IEEE.
The issue I take with labelling programmers engineers is that there doesn't seem to be any kind of actual engineering practice when developing software. If we look at the definition of engineering that the IEEE use in their software engineering glossary, they say it's "the application of a systematic, disciplined, quantifiable approach to structures, machines, products, systems, or processes" and that software engineering is "the application of a systematic, disciplined, quantifiable approach to the development, operation, and maintenance of software; that is, the application of engineering to software" (ibid).
So what exactly about programming is systematic or quantifiable? Sure, you can measure it, but is software development systematic? Hardly. It's very human and thus very messy. How do you quantify all that messiness? How do you quantify how long it'll take to write something? Sure, you can measure how many clients a web service can handle. But can you predict, to any meaningful precision or accuracy (I'm using these terms here in the technical sense), how many clients it can serve? That's the basis of my objection. I'd like to start a thread about this for extended discussion, but I want to get some more thoughts together first before I do.
You might enjoy reading NASA's coding guidelines. Code running inside rovers on Mars or in satellites is just a wee bit harder to update, so they don't fuck around when it comes to reliability....
You might enjoy reading NASA's coding guidelines. Code running inside rovers on Mars or in satellites is just a wee bit harder to update, so they don't fuck around when it comes to reliability. The consequences of one bug in that kind of code are astronomical.
Those are really good guidelines, and they're part of what I strive to adhere to for my own open source C code. That said, they don't approach the "systematic, quantifiable" definition I quoted in...
Those are really good guidelines, and they're part of what I strive to adhere to for my own open source C code. That said, they don't approach the "systematic, quantifiable" definition I quoted in my previous comment. Again, they're good guidelines, but by themselves don't seem to provide the kind of reliability I expect from an engineering discipline. Clearly, NASA does real engineering, but I wonder how much of it is in their software.
Again, this needs its own topic, and I want to gather some more thoughts before posting.
... and...
astronomical
I see what you did there. That was terrible. Are you happy with yourself? :)
The best software is engineered like that, though. It's something you will see more often in the enterprise realm. Software that is well engineered spends more time in planning stages than it does...
The best software is engineered like that, though. It's something you will see more often in the enterprise realm. Software that is well engineered spends more time in planning stages than it does in implementation stages.
That being said, the vast majority of software is not well engineered, and a good amount of it does not apply the principles of engineering in it's development whatsoever.
To be honest, I am not sure there is going to be anything that would convince you because the engineering, at least by your definition, is what happens before the software is coded. That type of...
To be honest, I am not sure there is going to be anything that would convince you because the engineering, at least by your definition, is what happens before the software is coded. That type of evidence is not typically released to the public.
That being said, there are a few examples that should prove themselves to be engineered. Microsoft has a couple I think work pretty well. The Ribbon interface, for example, came about after proper experimentation and continual adjustment until they were able to achieve a quantifiable goal. The same thing must be true for the software driving the Kinnect cameras, no?
Engineering is probably more self-evident in embedded systems where safety measures have to be proven to work reliably regardless of processor load, such as a car's ECU.
That's actually a really good point. I haven't made that post yet, but this is making me wonder if it would really make sense to. Hmm. I can't really comment on the Ribbon, or the Kinect cameras,...
engineering, at least by your definition, is what happens before the software is coded. That type of evidence is not typically released to the public.
That's actually a really good point. I haven't made that post yet, but this is making me wonder if it would really make sense to. Hmm.
The Ribbon interface, for example, came about after proper experimentation and continual adjustment until they were able to achieve a quantifiable goal.
I can't really comment on the Ribbon, or the Kinect cameras, besides expressing such intense dislike for the Ribbon that it makes me grateful I don't have to suffer through using Windows because of it. That said, it's absolutely possible to engineer something that people just hate using.
Engineering is probably more self-evident in embedded systems where safety measures have to be proven to work reliably regardless of processor load, such as a car's ECU.
That's an interesting point, really. There may be something close to, or meeting, the definition I gave, or at least the high standards I have in my head that I have trouble quantifying. Something to think about, thank you. :)
Totally agree with the premise of this post, but I think the example of CI is an unfortunate one. Some sort of automated testing is not that hard to set up and is useful. I have Gitlab's free CI...
Totally agree with the premise of this post, but I think the example of CI is an unfortunate one. Some sort of automated testing is not that hard to set up and is useful. I have Gitlab's free CI running for my config, which when I push to it runs a script which installs my config (including the basic set of packages and the system-wide config I use) on a Debian Testing image. Thus I can be sure I will be able to reproduce my setup from a bare-bones Debian image with 4 commands, almost bit identical to what I had before. CI has helped me straighten up a few problems in it which caused it to sometimes work and sometimes not. It took me a weekend to set up in total (spread over weeks). It was definitely harder than it should've been tho.
The most horrible part of web dev today is the a la mode front end. And I mean horrible in the literal sense, trying to make sense of this mess terrorises me. It is like a tale of building a neighbourhood of haphazard favelas on top of sand and sea turtles. Everything in that sort of front-end and also Node ecosystem is so badly put together that it is a miracle that it works at all. And that is one of the sources of this infrastructure fetish, IMHO, because Node.js is what made people believe that writing a crippled make script in JSON which loads a billion libraries and takes ages to just run a linter or what is basically a macro processor (the "minifier" or the "transpiler" [as if it was somehow different from a compiler] or the "uglifier" or all of them; it is basically a pipeline of filters on a regular set(s) of files, just what Unix and Make were made for) over a handful files. I witnessed Node.js come out (and I loved it back then), but then NPM happened, and software development had its own Eternal September (I'm not sure which of these events are first).
A couple years ago I made a website for my uncle's tile business. It was reportedly liked by everybody. I wrote it with m4 and make, and a couple tiny JS / CSS libs for a carousel on the front page and a gallery with lightboxes supported. It was a million times better than the shit people build with what is fashionable today, or the crime against humanity that is Shopify/Squarespace/Wix (fuck them for breaking the web and building a UI for spreading the disease efficiently).
I bet that most of the infrastructure designed for redundant operations like k8s, docker swarm, mongodb's multi-node setup, etc. is not necessary for most deployments.
In 99% a LAMP stack on a single machine will get you to 10 million visitors a month. With backups you can recover from failure relatively quickly.
If you need more, you can split the database and apache into separate servers, maybe deploy load balanced apache servers.
Swap out PHP in this stack for your favorite language's web framework.
That's the complexity 99% of people should need, 80% of them will however deploy on 20 AWS machines with autoscaling, swarm or k8s and all the other things they read about in medium blogposts.
I'm reminded of an old Dilbert strip that was taped to my office door. I'd simply change 'open source' to 'cloud' and 'free' to 'scalable' in this context. :P
I realized a bit ago that I would build projects for the joy of the infrastructure and tooling around them and not for the love of the project I was working on. Realizing that changed both how I approach problems as well as the types of problems I work on.
That sounds like you should be a sysadmin rather than a programmer. ;)
Haha :) I worked as a sysadmin a long time ago before switching into SAP finance implementations :)
This reminds me of a well-known xtranormal video on MongoDB. Programmers (we're not engineers, please stop making real engineers look bad) like to use the latest, coolest new tech because it's the latest, coolest new tech. That's why many of us become programmers. I was there once*. It happens so much because so few programmers and sysadmins are able to step back and realize that the new fad technology rarely acutally provides any value to the end user. Sure, build your hobby projects with all the cool toys if you want, but for actually delivering value to your users, keep it simple.
I used the word "fad" deliberately, because the tech industry is exactly like the fashion industry in this way. That xtranormal video is at least eight years old, based on a timestamp of a reddit post linking to the original site. Eight years is an eternity in the tech world. Those cool fashionable Docker containers you use for everything? FreeBSD did it in 2000, and IBM did it 30 years before that. I'm only 35 and I've seen the fads in the tech industry cycle a few times already. If you've not seen it yourself, you're either very new (which is perfectly okay!) or not paying attention (which is less okay).
Now, this is not to say that you shouldn't design the system to be scalable should you need it to be, but that's an entirely different discussion.
* It was only once, because my boss at the time showed me I was being dumb and I learned from it.
I think there is going to have to be a split on this at some point where there are software engineers and software "technicians". A code boot camp graduate churning out web GUIs in the flavor of the month JS framework isn't an engineer. On the other hand, there are a small number of developers writing serious, robust, safety critical software that probably do deserve the title of engineer.
This deserves a separate thread, but I'd like to quickly justify and summarize my objection based on the definitions given by the IEEE.
The issue I take with labelling programmers engineers is that there doesn't seem to be any kind of actual engineering practice when developing software. If we look at the definition of engineering that the IEEE use in their software engineering glossary, they say it's "the application of a systematic, disciplined, quantifiable approach to structures, machines, products, systems, or processes" and that software engineering is "the application of a systematic, disciplined, quantifiable approach to the development, operation, and maintenance of software; that is, the application of engineering to software" (ibid).
So what exactly about programming is systematic or quantifiable? Sure, you can measure it, but is software development systematic? Hardly. It's very human and thus very messy. How do you quantify all that messiness? How do you quantify how long it'll take to write something? Sure, you can measure how many clients a web service can handle. But can you predict, to any meaningful precision or accuracy (I'm using these terms here in the technical sense), how many clients it can serve? That's the basis of my objection. I'd like to start a thread about this for extended discussion, but I want to get some more thoughts together first before I do.
You might enjoy reading NASA's coding guidelines. Code running inside rovers on Mars or in satellites is just a wee bit harder to update, so they don't fuck around when it comes to reliability. The consequences of one bug in that kind of code are astronomical.
Those are really good guidelines, and they're part of what I strive to adhere to for my own open source C code. That said, they don't approach the "systematic, quantifiable" definition I quoted in my previous comment. Again, they're good guidelines, but by themselves don't seem to provide the kind of reliability I expect from an engineering discipline. Clearly, NASA does real engineering, but I wonder how much of it is in their software.
Again, this needs its own topic, and I want to gather some more thoughts before posting.
... and...
I see what you did there. That was terrible. Are you happy with yourself? :)
The best software is engineered like that, though. It's something you will see more often in the enterprise realm. Software that is well engineered spends more time in planning stages than it does in implementation stages.
That being said, the vast majority of software is not well engineered, and a good amount of it does not apply the principles of engineering in it's development whatsoever.
Any examples of things I can look at as a consumer that you would consider engineered? Does it meet the definition of engineering that I gave?
To be honest, I am not sure there is going to be anything that would convince you because the engineering, at least by your definition, is what happens before the software is coded. That type of evidence is not typically released to the public.
That being said, there are a few examples that should prove themselves to be engineered. Microsoft has a couple I think work pretty well. The Ribbon interface, for example, came about after proper experimentation and continual adjustment until they were able to achieve a quantifiable goal. The same thing must be true for the software driving the Kinnect cameras, no?
Engineering is probably more self-evident in embedded systems where safety measures have to be proven to work reliably regardless of processor load, such as a car's ECU.
That's actually a really good point. I haven't made that post yet, but this is making me wonder if it would really make sense to. Hmm.
I can't really comment on the Ribbon, or the Kinect cameras, besides expressing such intense dislike for the Ribbon that it makes me grateful I don't have to suffer through using Windows because of it. That said, it's absolutely possible to engineer something that people just hate using.
That's an interesting point, really. There may be something close to, or meeting, the definition I gave, or at least the high standards I have in my head that I have trouble quantifying. Something to think about, thank you. :)
Totally agree with the premise of this post, but I think the example of CI is an unfortunate one. Some sort of automated testing is not that hard to set up and is useful. I have Gitlab's free CI running for my config, which when I push to it runs a script which installs my config (including the basic set of packages and the system-wide config I use) on a Debian Testing image. Thus I can be sure I will be able to reproduce my setup from a bare-bones Debian image with 4 commands, almost bit identical to what I had before. CI has helped me straighten up a few problems in it which caused it to sometimes work and sometimes not. It took me a weekend to set up in total (spread over weeks). It was definitely harder than it should've been tho.
The most horrible part of web dev today is the a la mode front end. And I mean horrible in the literal sense, trying to make sense of this mess terrorises me. It is like a tale of building a neighbourhood of haphazard favelas on top of sand and sea turtles. Everything in that sort of front-end and also Node ecosystem is so badly put together that it is a miracle that it works at all. And that is one of the sources of this infrastructure fetish, IMHO, because Node.js is what made people believe that writing a crippled make script in JSON which loads a billion libraries and takes ages to just run a linter or what is basically a macro processor (the "minifier" or the "transpiler" [as if it was somehow different from a compiler] or the "uglifier" or all of them; it is basically a pipeline of filters on a regular set(s) of files, just what Unix and Make were made for) over a handful files. I witnessed Node.js come out (and I loved it back then), but then NPM happened, and software development had its own Eternal September (I'm not sure which of these events are first).
A couple years ago I made a website for my uncle's tile business. It was reportedly liked by everybody. I wrote it with
m4andmake, and a couple tiny JS / CSS libs for a carousel on the front page and a gallery with lightboxes supported. It was a million times better than the shit people build with what is fashionable today, or the crime against humanity that is Shopify/Squarespace/Wix (fuck them for breaking the web and building a UI for spreading the disease efficiently).