The distinction I’ve used to separate developers/programmers and software engineers is that the former two are broad classifications of people who can write code while software engineers...
The distinction I’ve used to separate developers/programmers and software engineers is that the former two are broad classifications of people who can write code while software engineers specifically are capable of handling major architectural/design sorts of decisions. A software engineer can architect and write a complex application from scratch, whereas a developer/programmer might only be capable of writing smaller components and working on existing projects.
It’s a bit thorny though since “engineer” is a protected title, and the majority of software engineers, even under my more restricted definition, don’t have the requisite qualifications for the title. Many are even self-taught. I have no idea what other name they’d be accurately described by, though.
I agree with your distinction. The way I see it in practical terms is developers/programmers can spool up an instance of their code & test it, totally screw it up, change the line of code, then...
I agree with your distinction. The way I see it in practical terms is developers/programmers can spool up an instance of their code & test it, totally screw it up, change the line of code, then test it again with no real stakes. You step in the realm of engineering when you haven't don't something yet and you're doing to planning/calculations/deep thoughts process for it to work with as few test/implementation cycles as possible.
I'm a test engineer where I have to build some fixtures to test components. Those fixtures when they deal with high pressure and I have to do safety reviews. Very tedious planning and design. That's engineering for me.
But I also build fixtures and test components that I do REALLY scrappy. And I just do it with assumptions because there is no stakes for cost or damange. I don't call that engineering, I call that hoodrat shit. I LOVE both aspects of my job, but half of it is engineering half of it is me just doing grunt mechanical work.
Is the distinction then, the stakes involved? Or more specifically the higher scrutiny, attention to detail, methodology, etc that go into a project that is higher stakes? I don't disagree with...
Is the distinction then, the stakes involved? Or more specifically the higher scrutiny, attention to detail, methodology, etc that go into a project that is higher stakes? I don't disagree with the distinction. Engineers in other fields have to be extremely meticulous in their planning, otherwise people literally die when something fails.
Software has an advantage in that you can build extremely complex things and test them before deploying them. You can't build a 1:1 test version of a bridge or rocket (or rather, you can but it would double the cost of the project). However you can test the code of a large application before it's deployed (unless you're Crowdstrike).
When this debate about "engineering" in software comes up I have wondered if the reason we see some of the colossal fuckups like Crowdstrike is because of this advantage. If culturally a software developer is not used to thinking slowly and carefully about the broad ramifications of every single code change, something is bound to slip through the cracks. In a world where problems can be fixed by deploying an emergency patch, are software developers unprepared to operate on the (relatively rare) system where the stakes are Crowdstrike-caused-global-disruption level high.
To add to what @ButteredToast and @vord have said, if an application has any complexity at all, the resources to fully test the state space of an application usually don't exist. The combination...
Software has an advantage in that you can build extremely complex things and test them before deploying them. You can't build a 1:1 test version of a bridge or rocket (or rather, you can but it would double the cost of the project). However you can test the code of a large application before it's deployed (unless you're Crowdstrike).
To add to what @ButteredToast and @vord have said, if an application has any complexity at all, the resources to fully test the state space of an application usually don't exist. The combination of even a handful of 32-bit inputs quickly becomes intractable. If you deal with messages (or anything real-time), then you can add all possible arrival timings on top of the value combinations.
This is why you end up with methodologies like code coverage, requirements coverage, etc. These are all ways of deciding "how much testing is enough" because "all of it" is not available.
But beyond that, you also have to deal with specification bugs, where the (incorrect) designed behavior is in implemented correctly. A unit test cannot even catch a specification bug.
A lot of bugs come from an incomplete idea of what can go wrong or how it can go wrong, misalignment of expectations at the system boundaries, or an incomplete model of the behavior of the external environment. Fuzz testing with a proper set of invariants can be a good complement to functional testing because it gets away from developer bias.
As you say, high reliability software should be held to a higher standard, and in that case, you have to go beyond testing to architectural approaches that assume failures will happen. However, trying to get a failure rate for a software component is super tricky to get right for myriad reasons, some of which are:
it's much easier to have common mode failures due to shared code, or even due to similarities in the training of developers and development processes
software fails in complex, nonlinear ways that aren't easily modeled by statistical failure methodologies that have their roots in electromechanical systems whose failure modes can be modeled and analyzed using the underlying physical properties of the materials.
One of the problems we face is that the gold standards people often point to for high reliability software (NASA, aviation back when the FAA had real oversight) are for software that was much, much simpler than what is being written today. It's not clear to me whether those methods can scale in practical cases of software being written at modern scale, even for organizations that have the will to engage with the extremely high cost.
Everything above is true for functional software (has requirements that could, in theory all be written down) before you even get into the ML stuff. There be dragons there because the whole point of ML is that it's too complex to write down the requirements. The technical term is "inscrutability", and I expect that eventually we will see inroads in ML reliability, assuming the rush to monetization doesn't do something to kill us all first.
If anyone wants some "light" reading, I could dig up some pointers.
I would argue that it's not actually possible to build full 1:1 test setups for some types of software, which complicates matters. There's some things that are nearly impossible to replicate in a...
I would argue that it's not actually possible to build full 1:1 test setups for some types of software, which complicates matters. There's some things that are nearly impossible to replicate in a test environment as well as some things that while possible to test for, might not be evident as needing to be tested prior to deployment or reaching certain scales. Sometimes your best option is to cover your bases as well as reasonably possible, prepare for the worst, and just see what happens when rubber hits the road.
That said I agree that software that mission-critical software specifically needs to be held to a higher standard. I've not worked on such projects myself, but intuitively the best option there would be to try to minimize size and complexity to the greatest degree possible, dividing large applications into fully discrete components and applying the *nix philosophy of "do one thing well". This limits the scope of testing necessary, makes remaining tester easier, and puts a limit on the size of the blast radius when things inevitably go wrong (and they will, no matter how thoughtful and guarded one might be in design and development). Massive monoliths placed to serve critical use cases are just asking for catastrophe.
I agree with this, however the point I was mainly driving at was that software can get a lot closer to parity with the final objective for a lot less time+money than a project that is building...
I would argue that it's not actually possible to build full 1:1 test setups for some types of software
I agree with this, however the point I was mainly driving at was that software can get a lot closer to parity with the final objective for a lot less time+money than a project that is building say, a bridge. This ability then yields to over-confidence more often in software than in physical projects.
I agree that software that mission-critical software specifically needs to be held to a higher standard.
Ahh ok, I see what you're saying now. I think some of that overconfidence might actually stem from illusions of how comparable testing environments are to their production counterparts, though; in...
I agree with this, however the point I was mainly driving at was that software can get a lot closer to parity with the final objective for a lot less time+money than a project that is building say, a bridge. This ability then yields to over-confidence more often in software than in physical projects.
Ahh ok, I see what you're saying now. I think some of that overconfidence might actually stem from illusions of how comparable testing environments are to their production counterparts, though; in my mind it seems very likely that it's common for screens full of green test checkmarks to inspire a high degree of overconfidence in softwares' readiness (and in some cases like Crowdstrike, in the tests themselves). In reality I believe the overwhelming majority of test environments are vastly different from production environments and would benefit greatly from being brought closer.
I can see that. I learned Test-Driven-Development at a place that applied it with near-religious fervor. It yielded a fairly high quality product, but even with 30,000 unit tests (not...
I can see that. I learned Test-Driven-Development at a place that applied it with near-religious fervor. It yielded a fairly high quality product, but even with 30,000 unit tests (not exaggerating) the software still had weird bugs that only appeared in real life scenarios. And that project had the benefit of running on a local PC to run a local piece of hardware, so we could test on the literal production hardware.
I consider that the highest bar I've ever encountered directly, and nobody else has come close. And yet I commonly encounter higher levels of confidence in projects that have 0 automated tests and minimal QA.
In Code Complete, Steve McConnell writes about what software practices have actually been proven to reduce production bugs in case studies. According to the research at the time, regular...
In Code Complete, Steve McConnell writes about what software practices have actually been proven to reduce production bugs in case studies. According to the research at the time, regular formalized code walkthroughs (not just reviewing PRs) were shown to pay the biggest dividends. Perhaps unsurprisingly, they also require more time investment and collaboration. This is part of why Extreme Programming emphasizes pair-programming so much.
Not coincidentally, that same project I mentioned above was also built by developers working in pairs for 100% of it's development. It's not for everyone, and it's not good for all tasks, but I'll...
Not coincidentally, that same project I mentioned above was also built by developers working in pairs for 100% of it's development. It's not for everyone, and it's not good for all tasks, but I'll be damned if I've ever seen a more effective tool for teaching/learning and real-time code review.
Show me a test that uses mock shims to simulate something and I'll show you a test not doing a 1:1 test. Especially anything nontrivial involving a database. Real world usage of a OLTP database...
I would argue that it's not actually possible to build full 1:1 test setups for some types of software,
Show me a test that uses mock shims to simulate something and I'll show you a test not doing a 1:1 test.
Especially anything nontrivial involving a database. Real world usage of a OLTP database can and will throw wrenches in any test assumptions about load, replication, and concurrency.
You can model them incredibly accurately applying the well understood science and math on paper or with the aid of a computer far far more completely, rigorously, and verifiably than testing most...
You can't build a 1:1 test version of a bridge or rocket
You can model them incredibly accurately applying the well understood science and math on paper or with the aid of a computer far far more completely, rigorously, and verifiably than testing most software is even possible (as others have described).
As far as I'm aware, engineer isn't a protected title in most countries (Canada is the biggest exception). I think it's fine to call people that do engineering engineers.
As far as I'm aware, engineer isn't a protected title in most countries (Canada is the biggest exception). I think it's fine to call people that do engineering engineers.
Engineer is a protected title in the US (see Professional Engineer). The disciplines covered are mainly things like mechanical, chemical, and civil engineering (list of exams). It can be domain...
Engineer is a protected title in the US (see Professional Engineer). The disciplines covered are mainly things like mechanical, chemical, and civil engineering (list of exams). It can be domain specific, but in many cases, engineering work has to be done or supervised by a PE.
Notably, there is no software engineering equivalent. When I was looking at that list, I do see there is a Computer Engineering exam, though I have no idea what it covers, and I have never met a computer engineering PE.
Engineer is not a protected title. Professional Engineer is a protected title. It's a subtle distinction, but it's important. The specifics of licensure vary by state. NSPE and NCEES are somewhat...
Engineer is not a protected title. Professional Engineer is a protected title. It's a subtle distinction, but it's important. The specifics of licensure vary by state. NSPE and NCEES are somewhat biased sources since they make money and increase their clout by licensing engineers.
Huh, thats really interesting. My mind image is quite different. In my current company we have software developers, senior developers, architects and software engineers too. I was never quite sure...
Huh, thats really interesting. My mind image is quite different. In my current company we have software developers, senior developers, architects and software engineers too. I was never quite sure what the difference between architects/developers and engineers. For me engineers is a people that more familiar with industry and can partly take a role of customer supports and partly role of arhitects. But I have no idea of actual meaning. And too lazy to ask. (and yes, my title includes "engineer").
It probably varies depending on the scale and scope of the software in question. For low to moderate complexity software, a software engineer can likely handle things end-to-end, but for past a...
It probably varies depending on the scale and scope of the software in question. For low to moderate complexity software, a software engineer can likely handle things end-to-end, but for past a certain size (for which the line is fuzzy), I would imagine dedicated architecture roles start becoming more necessary in which case software engineers play the role of figuring out and implementing the finer details of some large subsection.
Take these ideas with a grain of salt though, as I have zero experience working something FAANG-adjacent.
The way my undergrad program described it was software engineering was being able to understand, design, and implement the sort of strategic overhead view for projects; computer science was more...
The way my undergrad program described it was software engineering was being able to understand, design, and implement the sort of strategic overhead view for projects; computer science was more about...I dunno, making one's life's work the study of how to sort cat pictures faster or some other very specific single complex task.
Programming was just programming, more of a tool than a degree or role. Either major you obviously needed it.
In terms of motorcycles, I'd say it's like:
SE: Design the whole bike, but perhaps not every detail
Is software engineering “really” engineering? A lot of us call ourselves software engineers. Do we deserve that title? Are we mere pretenders to the idea of engineering? This is an important question, and like all important questions, it regularly sparks arguments online.
…
Nobody I read in these arguments, not one single person, ever worked as a “real” engineer.
…
So that’s what I set out to find: people who used to be professional engineers and then became professional software developers. I call these people crossovers, hybrids between the two worlds. I interviewed 17 crossovers on common software misconceptions, how the two worlds relate to each other, whether we can truthfully call what we do engineering, and what the different fields can teach and learn from each other.
…
Instead of asking how they felt about certain engineering topics, I just asked them point blank. “Do you consider software engineering actually engineering?”
Of the 17 crossovers I talked to, 15 said yes.
…
That said, many of the crossovers also added an additional qualification: software engineering is real engineering, but a lot of people who write software aren’t doing software engineering. This is not a problem with them, rather a problem with our field: we don’t have a rich enough vocabulary to talk about what these developers do. Not everybody who works with electricity is going to be an electrical engineer; many will be electricians. And this is okay. Electrical engineering is a very narrow skill set in the broader field of electric professions and plenty of people have other important skills in that space. But we use things like “programmer”, “software engineer”, and “software developer” interchangeably. What is the difference between a software engineer and a software developer?
I’m one of those people who has done at least a bit of both. Formally educated as a Chemical Engineer in the US, worked a bit in the industry (not long enough to get a Processional Engineer...
I’m one of those people who has done at least a bit of both. Formally educated as a Chemical Engineer in the US, worked a bit in the industry (not long enough to get a Processional Engineer licence though), and have been working in software for a long time. I don’t know what ChemE the author talked to, but the primary force behind designs in my education and experience is calculation and application of scientific principles/equations/etc to arrive at theoretically ideal solutions, and only then does “experimentation and iteration” come into play to get it across the real world finish line. But even then, knowing the ideal first lets you accurately document the changes, considerations, and compromises.
With that foundation, I don’t think any of us in software are engineers outside of places like NASA where everything is modeled, rigorously calculated and designed, and formally proven correct.
The rest of us, especially for web? Nowhere near the same rigor, detailed problem analysis, same application of science and mathematics to calculate concrete requirements and safety factors and design provably sufficient solutions, the formal verification of solutions, etc.
I’ve begrudgingly accepted “engineer” in my software title simply because it usually means people give it more respect and money.
I don’t think any of us in software are engineers outside of places like NASA where everything is modeled, rigorously calculated and designed, and formally proven correct.
In America, there are also other fields of engineering which don't have licensing exams. Perhaps surprisingly, biomedical engineering doesn't have a dedicated engineering license, and there's no...
In America, there are also other fields of engineering which don't have licensing exams. Perhaps surprisingly, biomedical engineering doesn't have a dedicated engineering license, and there's no real push for licensure in the field.
I'm doubtful whether licenses even make engineers safer. Passing an exam doesn't guarantee you won't make mistakes.
It doesn't but it does give liability traceability. Primarily with civil engineers and some electrical (power) engineers there is a real public cost to getting things wrong and if that happens...
I'm doubtful whether licenses even make engineers safer. Passing an exam doesn't guarantee you won't make mistakes.
It doesn't but it does give liability traceability. Primarily with civil engineers and some electrical (power) engineers there is a real public cost to getting things wrong and if that happens knowing who designed, approved, and possibly, inspected it are important. If nothing else it gives you a starting point to reassess other projects done by the same person.
In the case of software, if engineering were to become a protected title I think it would be important for the individuals managing engineers to be equally culpable given how common it is in the...
In the case of software, if engineering were to become a protected title I think it would be important for the individuals managing engineers to be equally culpable given how common it is in the software world for executives and management to push for cutting QA short and shipping faster.
If you exert decisionmaking authority over an engineer, you are responsible for any failures, especially if your decision is against said engineer's reccomendations. My GM Saturn trauma is leaking.
If you exert decisionmaking authority over an engineer, you are responsible for any failures, especially if your decision is against said engineer's reccomendations.
One way software engineering is different is that a senior engineer can't just hand off a lengthy design to a junior and expect them to implement it. In other fields, an engineer can draft some...
One way software engineering is different is that a senior engineer can't just hand off a lengthy design to a junior and expect them to implement it. In other fields, an engineer can draft some schematics then let a machinist do their work. The metaphor starts breaking down trying to compare factories cutting corners and software engineers skipping testing. There are similarities, but, as the article discusses, the whole project/product approach is very different in software.
Well put. It makes a lot of sense in traditional engineering project structures where there are clear-cut processes and check boxes for designs that could seriously harm people. At least with...
Well put. It makes a lot of sense in traditional engineering project structures where there are clear-cut processes and check boxes for designs that could seriously harm people.
At least with biomedical engineering, the whole process of getting to market with all the extensive testing (usually) shakes out any critical flaws in the design and product. It doesn't make much sense to have any one engineer approve something since there are already so many other safety checks in place.
I find it interesting that the article talks (I think quite deliberately) mainly in terms of engineering vs not engineering, but the discussion here is more about engineer vs not engineer. In...
I find it interesting that the article talks (I think quite deliberately) mainly in terms of engineering vs not engineering, but the discussion here is more about engineer vs not engineer. In general, I think the question "what is software engineering?" is a lot more useful than "am I a software engineer?", and I think Hillel Wayne's series here does a good job of answering the former.
I particularly like the discussion later on in the series about what software engineering does better than other engineering disciplines. I've worked occasionally in places that also did electrical engineering topics, and I can attest that things like version control, automatic testing, and continuous integration just aren't the norm there. I think as developers, we often compare our field to really lofty engineering pursuits like designing spacecraft or spanning rivers, but the vast majority of engineering is much more common and everyday items and challenges.
This is kind of interesting. I guess I consider engineering a kind of fluid term, and scientist a more protected term. Or perhaps science captures the academic knowledge, and engineering captures...
This is kind of interesting. I guess I consider engineering a kind of fluid term, and scientist a more protected term. Or perhaps science captures the academic knowledge, and engineering captures the application. I wouldn't mind "programmer" becoming the equivalent of "electrician" though, assuming all other factors stayed the same.
The bit about "logic" also made me think-- why not philosophers? Or computer linguist? Personally, I think I come into programming from an interest in logic (I think I would have been a lawyer, in another life) or in languages (which come easily to me).
Or even in systems designing-- computing is interesting because you design systems. This is different from other disciplines (like math or chemistry, as I understand them) because the system already exists, so you're only ever documenting what you observe. But to engineer feels more like the creation of an environment for the sake of it.
I think mathematical logic (as used in math and computer applications) isn't that closely related to how people use "logic" in fields like law. The difference is between the blind, systematic ,...
I think mathematical logic (as used in math and computer applications) isn't that closely related to how people use "logic" in fields like law. The difference is between the blind, systematic , machine-like use of logic, versus using it more informally, as a form of persuasion. When making a legal argument, the person using it is careful to make sure that the logic doesn't say anything that violates common sense or leads to a conclusion they don't like. A machine will just blindly carry on.
Machine logic can be dangerous if there isn't someone watching over it. More often, the computer does something silly.
Also, the distinction between "creation" and "discovery" is often blurry. In a mathematical sense, perhaps all creative works already exist in possibility space, but typically we say that it doesn't count until someone brings it into concrete reality.
I agree that logic as used in law is different from machine logic, but logic as used in philosophy encompasses both, and formal logic as used within linguistics is more or less the same as used in...
I agree that logic as used in law is different from machine logic, but logic as used in philosophy encompasses both, and formal logic as used within linguistics is more or less the same as used in mathematics or computer science, just with focus on different particular subsets.
There are a lot of different ways of using math and I think the same applies to logic? Sometimes we use logic like someone who is using math to do a story problem. If they’re careful, they are...
There are a lot of different ways of using math and I think the same applies to logic?
Sometimes we use logic like someone who is using math to do a story problem. If they’re careful, they are doing sanity checks along the way to make sure that the math applies to some real-world situation. The mapping between mathematical constructs and the real world is something people maintain, rather than accepting the result blindly, because math can be done wrong, and logic can also be misapplied.
With software, we do a lot of testing up front to make sure that the system does something sane when nobody is watching over the calculations. Often this involves detecting failures and carrying on, or halting with an error and waiting for human assistance when that’s the only safe thing to do.
When we rely on calculations, errors can be avoided via consistency checks, like doing the same calculations two different ways. Double-entry accounting is an example of that.
Philosophy is kind of odd since it’s often done with little in the way of consistency checks - there’s often little to keep it from going off the rails, other than seeing if others are convinced. I think philosophers also do pure math sometimes, but don’t know about the relationship between that and philosophical ideas.
In theoretical economics, my understanding is that coming up with a mathematical model is seen as a way to make sure your theory is self-consistent, but consistency seems like kind of a low bar as far as real-world applications are concerned?
I’m wondering if you could explain what theoretical linguists do? I suspect things have changed a fair bit with statistics-based machine translation? We have “large language models” now. Grammar doesn’t seem to be entirely distinct from other statistical regularities.
The study of theoretical linguists hasn't actually been all that directly impacted by statistics-based machine translation and large language models as far as I'm aware. It's largely the other way...
I’m wondering if you could explain what theoretical linguists do? I suspect things have changed a fair bit with statistics-based machine translation? We have “large language models” now. Grammar doesn’t seem to be entirely distinct from other statistical regularities.
The study of theoretical linguists hasn't actually been all that directly impacted by statistics-based machine translation and large language models as far as I'm aware. It's largely the other way around -- linguists who specialize in computational linguistics have been part of the fields of research that lead to such breakthroughs for a while. While the statistical methods involved in machine translation and LLMs are impressive, they don't actually much change the questions theoretical linguists are trying to answer (nor do they really contribute much to answering them).
Theoretical linguists are interested in how language in general functions. There are multiple subfields here. People who study phonetics and phonology care about the sounds of language and how they're produced, interpreted, and how they systematically correspond to meaningful units in the grammars of languages. People who study syntax and morphology care about how units of meaning are combined and arranged and the rules that govern those things. People who study semantics and pragmatics care about the meaning captured by utterances and their parts and how that meaning is created from the sum of their parts and the context in which an utterance is made (this is the area I specialized in when I studied theoretical linguistics, so it's the one I understand best by far).
Formal semantics uses the formal logic used in computer science and mathematics more generally, as formal semantics is specifically concerned with modeling the meanings of different linguistic elements formally (that is, with formal logic). There are other schools of semantics that approach meaning in different ways, but that's not relevant here -- this approach to semantics is very common and uses a lot of what is undoubtedly math (in my experience, mostly typed lambda calculus). I had to pay equal attention to parsing the lambda calculus expressions in formal semantics papers as I later had to pay attention to parsing the mathematical expressions in machine learning papers. If anything, actually understanding those parts of the paper was far more vital in formal semantics than it was in machine learning. There are also other applications of mathematical concepts like game theory to linguistics that are newer to the field (such as "rational speech act theory" which relies on game-theory to model how speakers and listeners interpret utterances, which was part of my master's thesis to the extent that I'm getting re-traumatized as I type this).
The fact that statistics on a shitton of human utterances can lead to something that models human language surprisingly well is cool, but it doesn't actually tell us much about how human language works on the level that a theoretical linguist is interested in. A lot of the discourse in theoretical linguistics is "which of our theoretical models best fits stuff we observe from natural language" and this is not something that's actually addressed by an LLM being able to produce utterances that fit the same patterns as the human utterances we've already observed -- especially since we know humans learn language very differently from LLMs.
The best tl;dr I can think of here is a metaphor -- theoretical linguistics is affected by the existence of statistics-based language models in much the same way that theoretical physics is affected by the existence of physics models. Occasionally there may be some usefulness for research, but models of the things we're studying are more useful for engineers trying to actually make things that do things as opposed to theoreticians who are studying the fundamental nature of the things being modeled.
Pivoting from theoretical linguistics to computational linguistics to work in tech is fairly common if only because it's by far the highest-paying option that has even a remote connection to what...
The bit about "logic" also made me think-- why not philosophers? Or computer linguist? Personally, I think I come into programming from an interest in logic (I think I would have been a lawyer, in another life) or in languages (which come easily to me).
Pivoting from theoretical linguistics to computational linguistics to work in tech is fairly common if only because it's by far the highest-paying option that has even a remote connection to what we learned in school. It's how I became a data scientist lol
The distinction I’ve used to separate developers/programmers and software engineers is that the former two are broad classifications of people who can write code while software engineers specifically are capable of handling major architectural/design sorts of decisions. A software engineer can architect and write a complex application from scratch, whereas a developer/programmer might only be capable of writing smaller components and working on existing projects.
It’s a bit thorny though since “engineer” is a protected title, and the majority of software engineers, even under my more restricted definition, don’t have the requisite qualifications for the title. Many are even self-taught. I have no idea what other name they’d be accurately described by, though.
I agree with your distinction. The way I see it in practical terms is developers/programmers can spool up an instance of their code & test it, totally screw it up, change the line of code, then test it again with no real stakes. You step in the realm of engineering when you haven't don't something yet and you're doing to planning/calculations/deep thoughts process for it to work with as few test/implementation cycles as possible.
I'm a test engineer where I have to build some fixtures to test components. Those fixtures when they deal with high pressure and I have to do safety reviews. Very tedious planning and design. That's engineering for me.
But I also build fixtures and test components that I do REALLY scrappy. And I just do it with assumptions because there is no stakes for cost or damange. I don't call that engineering, I call that hoodrat shit. I LOVE both aspects of my job, but half of it is engineering half of it is me just doing grunt mechanical work.
Is the distinction then, the stakes involved? Or more specifically the higher scrutiny, attention to detail, methodology, etc that go into a project that is higher stakes? I don't disagree with the distinction. Engineers in other fields have to be extremely meticulous in their planning, otherwise people literally die when something fails.
Software has an advantage in that you can build extremely complex things and test them before deploying them. You can't build a 1:1 test version of a bridge or rocket (or rather, you can but it would double the cost of the project). However you can test the code of a large application before it's deployed (unless you're Crowdstrike).
When this debate about "engineering" in software comes up I have wondered if the reason we see some of the colossal fuckups like Crowdstrike is because of this advantage. If culturally a software developer is not used to thinking slowly and carefully about the broad ramifications of every single code change, something is bound to slip through the cracks. In a world where problems can be fixed by deploying an emergency patch, are software developers unprepared to operate on the (relatively rare) system where the stakes are Crowdstrike-caused-global-disruption level high.
To add to what @ButteredToast and @vord have said, if an application has any complexity at all, the resources to fully test the state space of an application usually don't exist. The combination of even a handful of 32-bit inputs quickly becomes intractable. If you deal with messages (or anything real-time), then you can add all possible arrival timings on top of the value combinations.
This is why you end up with methodologies like code coverage, requirements coverage, etc. These are all ways of deciding "how much testing is enough" because "all of it" is not available.
But beyond that, you also have to deal with specification bugs, where the (incorrect) designed behavior is in implemented correctly. A unit test cannot even catch a specification bug.
A lot of bugs come from an incomplete idea of what can go wrong or how it can go wrong, misalignment of expectations at the system boundaries, or an incomplete model of the behavior of the external environment. Fuzz testing with a proper set of invariants can be a good complement to functional testing because it gets away from developer bias.
As you say, high reliability software should be held to a higher standard, and in that case, you have to go beyond testing to architectural approaches that assume failures will happen. However, trying to get a failure rate for a software component is super tricky to get right for myriad reasons, some of which are:
One of the problems we face is that the gold standards people often point to for high reliability software (NASA, aviation back when the FAA had real oversight) are for software that was much, much simpler than what is being written today. It's not clear to me whether those methods can scale in practical cases of software being written at modern scale, even for organizations that have the will to engage with the extremely high cost.
Everything above is true for functional software (has requirements that could, in theory all be written down) before you even get into the ML stuff. There be dragons there because the whole point of ML is that it's too complex to write down the requirements. The technical term is "inscrutability", and I expect that eventually we will see inroads in ML reliability, assuming the rush to monetization doesn't do something to kill us all first.
If anyone wants some "light" reading, I could dig up some pointers.
I would argue that it's not actually possible to build full 1:1 test setups for some types of software, which complicates matters. There's some things that are nearly impossible to replicate in a test environment as well as some things that while possible to test for, might not be evident as needing to be tested prior to deployment or reaching certain scales. Sometimes your best option is to cover your bases as well as reasonably possible, prepare for the worst, and just see what happens when rubber hits the road.
That said I agree that software that mission-critical software specifically needs to be held to a higher standard. I've not worked on such projects myself, but intuitively the best option there would be to try to minimize size and complexity to the greatest degree possible, dividing large applications into fully discrete components and applying the *nix philosophy of "do one thing well". This limits the scope of testing necessary, makes remaining tester easier, and puts a limit on the size of the blast radius when things inevitably go wrong (and they will, no matter how thoughtful and guarded one might be in design and development). Massive monoliths placed to serve critical use cases are just asking for catastrophe.
I agree with this, however the point I was mainly driving at was that software can get a lot closer to parity with the final objective for a lot less time+money than a project that is building say, a bridge. This ability then yields to over-confidence more often in software than in physical projects.
Fully agree. Incidentally if folks haven't ever read this article from 1996 on how NASA writes software, it's a really interesting look into the extreme end of "software that cannot under any circumstance fail because people will die" https://web.archive.org/web/20240118205531/https://www.fastcompany.com/28121/they-write-right-stuff
Ahh ok, I see what you're saying now. I think some of that overconfidence might actually stem from illusions of how comparable testing environments are to their production counterparts, though; in my mind it seems very likely that it's common for screens full of green test checkmarks to inspire a high degree of overconfidence in softwares' readiness (and in some cases like Crowdstrike, in the tests themselves). In reality I believe the overwhelming majority of test environments are vastly different from production environments and would benefit greatly from being brought closer.
I can see that. I learned Test-Driven-Development at a place that applied it with near-religious fervor. It yielded a fairly high quality product, but even with 30,000 unit tests (not exaggerating) the software still had weird bugs that only appeared in real life scenarios. And that project had the benefit of running on a local PC to run a local piece of hardware, so we could test on the literal production hardware.
I consider that the highest bar I've ever encountered directly, and nobody else has come close. And yet I commonly encounter higher levels of confidence in projects that have 0 automated tests and minimal QA.
In Code Complete, Steve McConnell writes about what software practices have actually been proven to reduce production bugs in case studies. According to the research at the time, regular formalized code walkthroughs (not just reviewing PRs) were shown to pay the biggest dividends. Perhaps unsurprisingly, they also require more time investment and collaboration. This is part of why Extreme Programming emphasizes pair-programming so much.
Not coincidentally, that same project I mentioned above was also built by developers working in pairs for 100% of it's development. It's not for everyone, and it's not good for all tasks, but I'll be damned if I've ever seen a more effective tool for teaching/learning and real-time code review.
Show me a test that uses mock shims to simulate something and I'll show you a test not doing a 1:1 test.
Especially anything nontrivial involving a database. Real world usage of a OLTP database can and will throw wrenches in any test assumptions about load, replication, and concurrency.
You can model them incredibly accurately applying the well understood science and math on paper or with the aid of a computer far far more completely, rigorously, and verifiably than testing most software is even possible (as others have described).
As far as I'm aware, engineer isn't a protected title in most countries (Canada is the biggest exception). I think it's fine to call people that do engineering engineers.
Engineer is a protected title in the US (see Professional Engineer). The disciplines covered are mainly things like mechanical, chemical, and civil engineering (list of exams). It can be domain specific, but in many cases, engineering work has to be done or supervised by a PE.
Notably, there is no software engineering equivalent. When I was looking at that list, I do see there is a Computer Engineering exam, though I have no idea what it covers, and I have never met a computer engineering PE.
Engineer is not a protected title. Professional Engineer is a protected title. It's a subtle distinction, but it's important. The specifics of licensure vary by state. NSPE and NCEES are somewhat biased sources since they make money and increase their clout by licensing engineers.
Huh, thats really interesting. My mind image is quite different. In my current company we have software developers, senior developers, architects and software engineers too. I was never quite sure what the difference between architects/developers and engineers. For me engineers is a people that more familiar with industry and can partly take a role of customer supports and partly role of arhitects. But I have no idea of actual meaning. And too lazy to ask. (and yes, my title includes "engineer").
It probably varies depending on the scale and scope of the software in question. For low to moderate complexity software, a software engineer can likely handle things end-to-end, but for past a certain size (for which the line is fuzzy), I would imagine dedicated architecture roles start becoming more necessary in which case software engineers play the role of figuring out and implementing the finer details of some large subsection.
Take these ideas with a grain of salt though, as I have zero experience working something FAANG-adjacent.
The way my undergrad program described it was software engineering was being able to understand, design, and implement the sort of strategic overhead view for projects; computer science was more about...I dunno, making one's life's work the study of how to sort cat pictures faster or some other very specific single complex task.
Programming was just programming, more of a tool than a degree or role. Either major you obviously needed it.
In terms of motorcycles, I'd say it's like:
SE: Design the whole bike, but perhaps not every detail
CS: Make the best carb ever
Programmer: The mechanics who put it all together
From the blog post:
…
…
…
…
I’m one of those people who has done at least a bit of both. Formally educated as a Chemical Engineer in the US, worked a bit in the industry (not long enough to get a Processional Engineer licence though), and have been working in software for a long time. I don’t know what ChemE the author talked to, but the primary force behind designs in my education and experience is calculation and application of scientific principles/equations/etc to arrive at theoretically ideal solutions, and only then does “experimentation and iteration” come into play to get it across the real world finish line. But even then, knowing the ideal first lets you accurately document the changes, considerations, and compromises.
With that foundation, I don’t think any of us in software are engineers outside of places like NASA where everything is modeled, rigorously calculated and designed, and formally proven correct.
The rest of us, especially for web? Nowhere near the same rigor, detailed problem analysis, same application of science and mathematics to calculate concrete requirements and safety factors and design provably sufficient solutions, the formal verification of solutions, etc.
I’ve begrudgingly accepted “engineer” in my software title simply because it usually means people give it more respect and money.
Biomedical tech.
In America, there are also other fields of engineering which don't have licensing exams. Perhaps surprisingly, biomedical engineering doesn't have a dedicated engineering license, and there's no real push for licensure in the field.
I'm doubtful whether licenses even make engineers safer. Passing an exam doesn't guarantee you won't make mistakes.
It doesn't but it does give liability traceability. Primarily with civil engineers and some electrical (power) engineers there is a real public cost to getting things wrong and if that happens knowing who designed, approved, and possibly, inspected it are important. If nothing else it gives you a starting point to reassess other projects done by the same person.
In the case of software, if engineering were to become a protected title I think it would be important for the individuals managing engineers to be equally culpable given how common it is in the software world for executives and management to push for cutting QA short and shipping faster.
If you exert decisionmaking authority over an engineer, you are responsible for any failures, especially if your decision is against said engineer's reccomendations.
My GM Saturn trauma is leaking.
One way software engineering is different is that a senior engineer can't just hand off a lengthy design to a junior and expect them to implement it. In other fields, an engineer can draft some schematics then let a machinist do their work. The metaphor starts breaking down trying to compare factories cutting corners and software engineers skipping testing. There are similarities, but, as the article discusses, the whole project/product approach is very different in software.
Well put. It makes a lot of sense in traditional engineering project structures where there are clear-cut processes and check boxes for designs that could seriously harm people.
At least with biomedical engineering, the whole process of getting to market with all the extensive testing (usually) shakes out any critical flaws in the design and product. It doesn't make much sense to have any one engineer approve something since there are already so many other safety checks in place.
I find it interesting that the article talks (I think quite deliberately) mainly in terms of engineering vs not engineering, but the discussion here is more about engineer vs not engineer. In general, I think the question "what is software engineering?" is a lot more useful than "am I a software engineer?", and I think Hillel Wayne's series here does a good job of answering the former.
I particularly like the discussion later on in the series about what software engineering does better than other engineering disciplines. I've worked occasionally in places that also did electrical engineering topics, and I can attest that things like version control, automatic testing, and continuous integration just aren't the norm there. I think as developers, we often compare our field to really lofty engineering pursuits like designing spacecraft or spanning rivers, but the vast majority of engineering is much more common and everyday items and challenges.
This is kind of interesting. I guess I consider engineering a kind of fluid term, and scientist a more protected term. Or perhaps science captures the academic knowledge, and engineering captures the application. I wouldn't mind "programmer" becoming the equivalent of "electrician" though, assuming all other factors stayed the same.
The bit about "logic" also made me think-- why not philosophers? Or computer linguist? Personally, I think I come into programming from an interest in logic (I think I would have been a lawyer, in another life) or in languages (which come easily to me).
Or even in systems designing-- computing is interesting because you design systems. This is different from other disciplines (like math or chemistry, as I understand them) because the system already exists, so you're only ever documenting what you observe. But to engineer feels more like the creation of an environment for the sake of it.
I think mathematical logic (as used in math and computer applications) isn't that closely related to how people use "logic" in fields like law. The difference is between the blind, systematic , machine-like use of logic, versus using it more informally, as a form of persuasion. When making a legal argument, the person using it is careful to make sure that the logic doesn't say anything that violates common sense or leads to a conclusion they don't like. A machine will just blindly carry on.
Machine logic can be dangerous if there isn't someone watching over it. More often, the computer does something silly.
Also, the distinction between "creation" and "discovery" is often blurry. In a mathematical sense, perhaps all creative works already exist in possibility space, but typically we say that it doesn't count until someone brings it into concrete reality.
I agree that logic as used in law is different from machine logic, but logic as used in philosophy encompasses both, and formal logic as used within linguistics is more or less the same as used in mathematics or computer science, just with focus on different particular subsets.
There are a lot of different ways of using math and I think the same applies to logic?
Sometimes we use logic like someone who is using math to do a story problem. If they’re careful, they are doing sanity checks along the way to make sure that the math applies to some real-world situation. The mapping between mathematical constructs and the real world is something people maintain, rather than accepting the result blindly, because math can be done wrong, and logic can also be misapplied.
With software, we do a lot of testing up front to make sure that the system does something sane when nobody is watching over the calculations. Often this involves detecting failures and carrying on, or halting with an error and waiting for human assistance when that’s the only safe thing to do.
When we rely on calculations, errors can be avoided via consistency checks, like doing the same calculations two different ways. Double-entry accounting is an example of that.
Philosophy is kind of odd since it’s often done with little in the way of consistency checks - there’s often little to keep it from going off the rails, other than seeing if others are convinced. I think philosophers also do pure math sometimes, but don’t know about the relationship between that and philosophical ideas.
In theoretical economics, my understanding is that coming up with a mathematical model is seen as a way to make sure your theory is self-consistent, but consistency seems like kind of a low bar as far as real-world applications are concerned?
I’m wondering if you could explain what theoretical linguists do? I suspect things have changed a fair bit with statistics-based machine translation? We have “large language models” now. Grammar doesn’t seem to be entirely distinct from other statistical regularities.
The study of theoretical linguists hasn't actually been all that directly impacted by statistics-based machine translation and large language models as far as I'm aware. It's largely the other way around -- linguists who specialize in computational linguistics have been part of the fields of research that lead to such breakthroughs for a while. While the statistical methods involved in machine translation and LLMs are impressive, they don't actually much change the questions theoretical linguists are trying to answer (nor do they really contribute much to answering them).
Theoretical linguists are interested in how language in general functions. There are multiple subfields here. People who study phonetics and phonology care about the sounds of language and how they're produced, interpreted, and how they systematically correspond to meaningful units in the grammars of languages. People who study syntax and morphology care about how units of meaning are combined and arranged and the rules that govern those things. People who study semantics and pragmatics care about the meaning captured by utterances and their parts and how that meaning is created from the sum of their parts and the context in which an utterance is made (this is the area I specialized in when I studied theoretical linguistics, so it's the one I understand best by far).
Formal semantics uses the formal logic used in computer science and mathematics more generally, as formal semantics is specifically concerned with modeling the meanings of different linguistic elements formally (that is, with formal logic). There are other schools of semantics that approach meaning in different ways, but that's not relevant here -- this approach to semantics is very common and uses a lot of what is undoubtedly math (in my experience, mostly typed lambda calculus). I had to pay equal attention to parsing the lambda calculus expressions in formal semantics papers as I later had to pay attention to parsing the mathematical expressions in machine learning papers. If anything, actually understanding those parts of the paper was far more vital in formal semantics than it was in machine learning. There are also other applications of mathematical concepts like game theory to linguistics that are newer to the field (such as "rational speech act theory" which relies on game-theory to model how speakers and listeners interpret utterances, which was part of my master's thesis to the extent that I'm getting re-traumatized as I type this).
The fact that statistics on a shitton of human utterances can lead to something that models human language surprisingly well is cool, but it doesn't actually tell us much about how human language works on the level that a theoretical linguist is interested in. A lot of the discourse in theoretical linguistics is "which of our theoretical models best fits stuff we observe from natural language" and this is not something that's actually addressed by an LLM being able to produce utterances that fit the same patterns as the human utterances we've already observed -- especially since we know humans learn language very differently from LLMs.
The best tl;dr I can think of here is a metaphor -- theoretical linguistics is affected by the existence of statistics-based language models in much the same way that theoretical physics is affected by the existence of physics models. Occasionally there may be some usefulness for research, but models of the things we're studying are more useful for engineers trying to actually make things that do things as opposed to theoreticians who are studying the fundamental nature of the things being modeled.
Pivoting from theoretical linguistics to computational linguistics to work in tech is fairly common if only because it's by far the highest-paying option that has even a remote connection to what we learned in school. It's how I became a data scientist lol
This essay and the following entries were great reads; thanks for posting.