Setting maybe aside your concerns about reactionary language, I can’t really discern from Linus’ comments how “slop” is coming to the Kernel project, as opposed to any LLM contribution undergoing...
Setting maybe aside your concerns about reactionary language, I can’t really discern from Linus’ comments how “slop” is coming to the Kernel project, as opposed to any LLM contribution undergoing the regular process of review, sign-off, and ultimately responsible (human) person via the sign-off/DCO.
I get the feeling this issue isn’t about code quality alone, as “slop” would imply to me at first. But if that’s really the case, maybe a better name is required. “Opposing LLM contribution on moral grounds” or similar, perhaps?
If that is not the case however, could someone enlighten me where you see cracks in the way Linux/-s wants to handle this topic going forward? Genuinely asking, as I felt the decisions so far to be coming from a mostly realist/pragmatic standpoint.
It is my opinion that proponents who take the stance that LLMs are "just a tool" in programming, and that you can embrace them while still maintaining enough institutional knowledge for humans to...
It is my opinion that proponents who take the stance that LLMs are "just a tool" in programming, and that you can embrace them while still maintaining enough institutional knowledge for humans to meaningfully review code and merge requests, are either deluding themselves or intentionally deluding others.
The act of reading and reviewing code is tedious, and anathema for most people who got into programming for the love of the craft. Everyone is excited about tools that make programming easier, but nobody is excited about tools that convert the job from "building things" into "making sure the LLM generated code works right." You might start out with good intentions (as I believe Linus is doing with the kernel), but that course only leads to one ultimate destination.
In any open source project that does not take a stance against LLMs, human developers are not going to review and validate every line of code that gets merged, and eventually human developers will not review and validate any line of code that gets merged. I mean removing human effort from the loop is the whole point of LLMs. As the tools get better and the developers get more and more burnt out, LLM code is going to be "just trusted" more and more. Skim the code, "looks right," click merge. Eventually you don't even skim the code anymore. Click "review code," an LLM says "looks right," click merge. Eventually you don't even click buttons anymore. LLM generated the code, LLM says "looks right," LLM merges. Talk to anyone with a job in software engineering, we're all watching this exact scenario play out in real-time.
I think there are two paths projects can take:
Take a hard stance against LLMs and allow human software engineers to maintain control and build institutional knowledge.
Allow LLM usage in some form and eventually end up as a fully LLM-generated and maintained product, because anyone who actually cares about understanding how the code works is going to get sick of it and fuck off and get replaced by more meat proxies.
Linus and others might think there's some middle-ground between the two; I think that's just wishful thinking. But I also think it's wishful thinking to believe that path 1 is even viable in the long-term. The obscenely wealthy companies developing and selling LLMs (and their army of flying monkies decrying the end of software engineering as a profession) are certainly doing everything within their power to ensure everyone believes LLMs are the future. It's very depressing because I do also think there's a possible world where LLMs really could be "just a tool" that actually empower human developers and creators with new ways to excel at their crafts, but sadly building those kinds of tools wouldn't extract as much wealth into the pockets of shareholders so they'll probably never come to be.
Anyway, all of that is to say, the thing that I and other Linux users are mourning is the fact that allowing LLMs into any project almost certainly spells the end of human involvement in the source code. Maybe not today, maybe not tomorrow, but eventually. There are people out there who think that's a good thing, because they believe LLMs will do a better job anyway, and maybe that's true (or will one day soon be true), but it still feels like we're losing something meaningful.
This seems pretty doomer, but I understand. I'm not sure what this means. LLMs don't prompt themselves, or have their own goals/ideas/direction for a project. They also aren't very good at finding...
This seems pretty doomer, but I understand.
allowing LLMs into any project almost certainly spells the end of human involvement in the source code.
I'm not sure what this means. LLMs don't prompt themselves, or have their own goals/ideas/direction for a project. They also aren't very good at finding UI/UX bugs or understanding what makes something feel nice to use, one of the most important aspects of any technology.
certainly doing everything within their power to ensure everyone believes LLMs are the future.
The open weight models are something like ~6 months behind the frontier models. If the frontier models ever plateau, it's unclear what their competitive advantage will be, especially if they continue going down the path of locking down their models to their own harnesses.
Perhaps somewhat inversely, price subsidization is inherently not sustainable. Businesses will have to decide between how to split costs between their own workforce vs an external vendor. Businesses that bet wrong, eliminating all humans, will either be left holding the bag OR up the creek without a paddle OR have no special sauce and be overtaken by someone who does.
LLMs cannot provide a direct replacement for domain knowledge gained first-hand within specific communities. That's not data you can scrape from a website.
Linus and others might think there's some middle-ground between the two; I think that's just wishful thinking.
Yeah, I'd certainly consider that a false dichotomy. From a really high level, there's way too much nuance in this domain for there to be only two options.
Skim the code, "looks right," click merge. Eventually you don't even skim the code anymore. Click "review code," an LLM says "looks right," click merge. Eventually you don't even click buttons anymore. LLM generated the code, LLM says "looks right," LLM merges.
I know for the codebase I maintain, it was already sprawling and inconsistent before LLMs. Getting agents to spit out anything useful required documenting (and starting to track) a lot of patterns (old vs new), assumptions, and use cases. So the end results for me have been:
general rubber duck benefits
way more linters to keep all parties on track
way more integration and unit tests
obscure bug fixes verified via TDD
a large amount of consolidation (a lot of replicated logic that has been DRY-ed up)
I care deeply about code readability and maintainability for the projects I work on. I'm reviewing a lot more code than I used to, not less. This is all anecdotal though.
I can add another anecdote. The development process I've settled on doesn't look anything like "prompt then review results". It's human guided throughout and I understand what comes out of it...
I can add another anecdote. The development process I've settled on doesn't look anything like "prompt then review results". It's human guided throughout and I understand what comes out of it deeply, roughly equivalent to the understanding I'd have if I'd written all of it by hand.
It's also slower than vibecoding, which is why there's a lot of slop being churned out right now, and there will be more to come. But agent assisted development doesn't automatically mean slop. It really is a tool that can be used well or used badly, and we're still figuring out what using it well looks like.
I am a bit unsure about this, but could we not liken LLMs to the introduction of compilers? Before those became common place people wrote assembly and even as far as the 90s people were better at...
I am a bit unsure about this, but could we not liken LLMs to the introduction of compilers? Before those became common place people wrote assembly and even as far as the 90s people were better at writing assembly code than compilers could reasonably produce. This might still be the case today, but I doubt it. It is definitely not true for most developers! Compilers now take a high level descriptions of the operations we want to perform and boils it down to some bare essentials. Now with LLMs we take a sequence of fairly vaguely worded instructions and generate a sequence of operations that then compilers will boil down further. Now, an argument can be made that LLMs are not the same because agent's are not deterministic... but that is only kind of true. If we discount hardware differences LLMs are still "just" an operation on input data and given the same seed you will get the same output. The same analogy could be made for compilers; if you give them slightly different input flags you can get wildly different results! So maybe we should take the last few decades of experience with working with software and see what we should keep around. Maybe an approach where we actually version control our prompts and other model parameters would be advisable? The point of building software has never been to create the software itself, it has always been to create something that solves a problem (or that entertains in the case of games, which can be viewed as the problem of boredom). So we never really care about the actual machine code anymore, maybe it is time we stopped caring about the source code too?
I hate it when people claim it's just like compilers. Compilers are deterministic. If you get different results because of a compiler flag, anyone looking over the assembly or anyone familiar...
I hate it when people claim it's just like compilers.
Compilers are deterministic. If you get different results because of a compiler flag, anyone looking over the assembly or anyone familiar enough with your programming language and compiler can point and say "that's not right", and fix it. This is possible because the source code and flags are right there for anyone to see, and someone still understands the process that creates your software. If the same input creates different output, that's a compiler bug.
LLMs don't have that. English is a terrible programming language, wayyy too much ambiguity. If we're genuinely going the "it's just like compilers" route, then we would be checking prompts and LLM versions into version control (no AGENTS.md is not that). Although that might help us understand why the clanker code looks like it does, still, it's a stochastic chatbot, it's not deterministic, it's not a compiler, and proceeding as if it were will eventually create an incoherent mess.
LLMs, as far as I understand the underlying implementations, are deterministic. Given the same input (text, seed, temperature etc) it will give you the same output. They behave non-deterministic...
LLMs, as far as I understand the underlying implementations, are deterministic. Given the same input (text, seed, temperature etc) it will give you the same output. They behave non-deterministic just because we usually currently run them that way. Committing the promts, models versions and other input into VCS would let you recreate the output deterministically. I think there is some nuances that might impact this in practice with how the service providers run the models (batching and hardware differences might impact the output, but I'm not sure as to what extent). Local models should mitigate that to a large extent of needed.
Way back were the compilers really deterministic? Are they that way even now? If I compile something on my laptop will I get the exact same binary as you would? I don't think so. Most likely there will be differences due to (in no particular order):
compilers versions
compilers flags
underlying hardware
operating system
There are probably other confounding issues there too. Some people can probably understand those differences in detail, most devs probably shrug and don't really care. If it compiles and pass the tests we consider them equivalent to another build.
I agree that English is a terrible programming language. So maybe we should look at ways of optimizing the input so we get better output. Maybe some semi structured format for defining tasks and expectations? Point is that today most devs don't care if the compiled assembly is "an incoherent mess", potentially we shouldn't care if the source code we compile is either.
I haven't really landed in what I think. I kind of mourn the loss of the craft. I like programming as a discipline and I don't enjoy poking an agent to do the right thing. It does however look like that's the direction the industry is barreling towards...
Committing all the LLM run parameters to get deterministic output later might be interesting, but without the ability to poke and prod at how the model is created, will all of its training data,...
Committing all the LLM run parameters to get deterministic output later might be interesting, but without the ability to poke and prod at how the model is created, will all of its training data, I'd argue it's still "opaque" output.
compilers versions
compilers flags
underlying hardware
operating system
Of course, but all of those are well understood, maybe not always by users, but definetly by the compiler people. If I can't get a build environment where I can recreate the exact binary you get, same inputs, then I don't like this compiler. Compiler engineering does focus on this, and most of the times when you can't reproduce an artifact byte for byte, external platform dependencies are to blame, not the compiler.
So maybe we should look at ways of optimizing the input so we get better output. Maybe some semi structured format for defining tasks and expectations?
Structured format for defining tasks is a programming language.
Structured format for defining expectations is programming and tests.
Point is that today most devs don't care if the compiled assembly is "an incoherent mess", potentially we shouldn't care if the source code we compile is either.
If the compiled assembly is an incoherent mess, the software will run like an incoherent mess, and we should definitely care, and we should poke at the compiler until we understand why it's doing what it is doing.
Yes, if it's a silly throwaway program where you don't really care, sure, who cares...
But as a basis for engineering practices, "(deep) understanding = wasted productivity" is a horrible set of practices for the industry to be barreling towards.
I still love computers and programming, but if the industry keeps going in this direction, I'll soon be mourning it as a profession, and only do it as a hobby.
Good points! While all structured formats aren't programming languages I get your point. This part I don't necessarily think is true. A lot of compiled languages ship bloated binaries which can be...
Structured format for defining tasks is a programming language.
Structured format for defining expectations is programming and tests.
Good points! While all structured formats aren't programming languages I get your point.
If the compiled assembly is an incoherent mess, the software will run like an incoherent mess, and we should definitely care, and we should poke at the compiler until we understand why it's doing what it is doing.
This part I don't necessarily think is true. A lot of compiled languages ship bloated binaries which can be really hard to unwind. Sometimes these run poorly too, but that's not a given. It's great if we can understand the underlying assembly, but that's not a requirement for functioning and working software. I'd even argue that as an industry pre llms few pieces of software had someone who fully understood it. If you go into legacy systems even the purpose of the system could be lost a decade ago... Now that doesn't mean we shouldn't care about software quality, just that the situation hasn't really changed much beside the pace at which (often por quality) software is produced.
Off topic; anecdotally, from what I've seen in software, every time someone sees a solution in "a structured format for defining tasks and expectations", it starts as a structured text config file...
structured format
Off topic; anecdotally, from what I've seen in software, every time someone sees a solution in "a structured format for defining tasks and expectations", it starts as a structured text config file and then tends to eventually evolve (or devolve, depending on how you look at it) into a programming language to express the tasks and expectations.
This part I don't necessarily think is true. A lot of compiled languages ship bloated binaries which can be really hard to unwind.
This is always seen as the result of an unavoidable trade-off between "development speed" and "quality". Looking at the state of software in the world, including the pre-LLMs era, I'd say the bloat is too much.
I suspect the truth is that 90% of the extra bloat we see in the software running today comes from bad software practices like "use the mega-kitchen-sink-framework, just in case, it's considered very productive and we must ship fast!". The extra development speed rarely justifies the boat.
Tooling that incorporates programming language research into their designs is amazing nowadays, and I think if we genuinely reconsidered how software engineers learn to deal with complexity over time a bit more deeply than "look at Java Object Oriented Programming, put the complexities in the classes and problems gone", by actually using tools to make life easier, we could have better development time & less bloat. LLMs can benefit a lot from static analysis too, although sometimes I see them do weird workarounds just to avoid a compiler warning.
Sometimes these run poorly too, but that's not a given.
Maybe modern CPUs and smart interpreters can do their magic to the point where users don't feel the difference, but still the software uses more resources than it should. If we keep building on top of it, eventually it will become a problem. If it was a bloated incoherent mess, either fix it or accept that it will get worse.
It's great if we can understand the underlying assembly, but that's not a requirement for functioning and working software.
Then the underlying assembly is doing what you expect it to do, and you trust that the compiler is working correctly, as the compiler is deterministic, and it follows a well-specified and debuggable set of rules that create the assembly. You've just traded the understanding requirement for trusting that the compiler engineers understand, so you can worry about your source code, instead of assembly. You now focus on making sure the source code follows the rules of the compiler. If it does, and you didn't introduce some logic but into your code, then you will have working software.
The understanding is still a requirement for working functioning and working software, because what is actually happening on the CPU is still raw instructions.
LLMs are not compilers, they're not the same process or type of abstraction: I can't trust an LLM, and, because of their design, they will not provide understanding, even if it was made deterministic with a set seed and temperature and all that.
If I can get the LLM to produce correct code, I will only be able to trust that my input was correct by looking at the produced source code. There simply is no set of rules it actually follows, at least not that a human can understand: it "simply" does matrix multiplications and spits out the highest scoring tokens.
For a one-shot program, fine, check the produced source code and go on.
But on a project that needs to evolve over time, which needs to be understood, if the "official description" or "high level definition" of what the software actually does on the CPU lives inside of an LLM, or if the specifications are the input to an LLM, as if really using it like a "higher level compiler", I still don't know what the software will actually do.
The best we can get is guesswork, which is why it's not the same kind of abstraction as compilers, where high level language (+ compilation flags) is enough.
BTW all of this ignores the ethical and technical limitations of LLMs.
I'd even argue that as an industry pre llms few pieces of software had someone who fully understood it.
When addressing "compilers abstraction <> LLM abstraction", the point is that someone you trust understands it, that it is well specified, that it always follows rules, enough to poke and prod at so that, if you really wanted to, eventually you can find out. All the passes between "high level specification" to "raw machine code" are understood by a group of people that we trust enough.
This doesn't work for an LLM: we don't know how to follow the LLMs "rules", because even if it's deterministic, some slight change might make it hallucinate completely wrong code, and I have no idea how to actually figure out why. If that can happen for every slight change, it's going to be a useless abstraction, especially compared to what compilers allowed us to do when moving from assembly to something like C.
If you go into legacy systems even the purpose of the system could be lost a decade ago...
I mean legacy software is actually a good example of why vibecoded projects turn into an incoherent mess: the understanding of why the code does things is missing.
Now that doesn't mean we shouldn't care about software quality, just that the situation hasn't really changed much beside the pace at which (often poor quality) software is produced.
I agree, I mostly wanted to address the "liken the introduction of LLMs to the introduction of compilers" part. I think the state of software engineering was already a disaster, noticably affecting the quality of software. Now it's accelerating in the wrong direction.
I can see that becoming true in the future. Maybe the near future. But right now I think the coding agents just have too limited context windows and too weak prompt adherence. They can’t keep a...
So we never really care about the actual machine code anymore, maybe it is time we stopped caring about the source code too?
I can see that becoming true in the future. Maybe the near future. But right now I think the coding agents just have too limited context windows and too weak prompt adherence. They can’t keep a whole project vision and architecture in “mind” without slowly drifting away from it over time. They'll start introducing “alternative” implementations that are odds with how the project was intended to function. They'll undo optimizations that were lovingly crafted by people who did it that way for a reason. They'll deliver what they were prompted for, but at the cost of subtle regressions in other features they weren't prompted about.
If they were able to consistently avoid those pitfalls, I'd agree that the source code doesn't matter anymore. Until then, software that’s worked fine for years is going to start getting… flaky. Buggy. Not just functionally, but visually too. All that nice pixel-perfect UI that frontend guys like me (until I left the field last year) pored over to get the colors and margins and hover states and transition timings juuuust right and slick and professionally branded… the AI doesn't care about that. It'll produce something good enough at first glance. Missing polish in subtle ways, but close, and that'll ship. Then the next PR that touches the same components will drift a bit further. Just gradually getting crappier over time, until the user base jumps ship because the system is just unpleasant to use now and feels like nobody at the company cares about quality anymore.
It’s early yet. I don’t think those limitations are going to be with us forever. But in the meantime I think they’re going to be a landline for a lot of software projects. I’m watching intently to see how many well-regarded companies’ products just start devolving like this. My guess is once things get to a certain point it’s too late to reverse course because the whole codebase is poisoned, and that’s just the death sentence for that project.
Edit, but posting standalone because I think it’s worth it: LWN: Development statistics for the 7.2 kernel About the article’s last plotted graph (take a look!): (sentence in bold highlighted by me)
Edit, but posting standalone because I think it’s worth it:
About the article’s last plotted graph (take a look!):
The noteworthy part of the plot, though, is at the right-hand side, where the number of commits carrying Fixes tags increases dramatically, after having been roughly level for quite a few releases. That, it seems, is what happens once large numbers of people start turning LLMs toward the task of finding bugs in existing software. This is going to be an interesting trend to watch. The number of bugs cannot be infinite — even in the kernel — so the rate of bug detection and fixing must eventually fall off. The existing bugs will have been fixed, and LLM review will have reduced the number of new bugs introduced. At least, that is the hope.
If that drop-off does not happen, there will be some interesting questions to answer. It seems reasonably clear that the spread of LLMs has enabled people to create kernel patches who would not have been capable of doing so before. If that is true, it is not much of a stretch to conclude that, in many cases, the submitters of these patches do not fully understand what those patches do. Perhaps the LLMs are good enough that this understanding is no longer necessary, but it could also be that we are creating a pile of code that nobody has ever really understood, but which will need to be maintained in the coming years.
The Linux kernel is aimed at production use, not research or experimentation. But there is currently a massive experiment being run on the kernel's development processes and community. This community has absorbed new processes and tools many times over its 35-year history, and will almost certainly find a way to do that this time as well. But that absorption has not always been without turbulence along the way.
Meanwhile, there are nearly 14,000 non-merge changesets sitting in the linux-next repository and poised to be pulled for the 7.3 release; over 1,100 of them contain Assisted-by tags. It would seem that the high level of activity that has characterized recent releases will not be ending anytime soon.
I had to stop this video multiple times because I could not handle the smug superiority of Mr. Gerard. Letting alone that the entire justification here is that "I say AI is slop, and therefore all...
I had to stop this video multiple times because I could not handle the smug superiority of Mr. Gerard. Letting alone that the entire justification here is that "I say AI is slop, and therefore all code ever made with AI is slop by default, and that means it is bad" is a level of disingenuous cherry picking you really only see in the open source community and, maybe some right wing discord servers, there's a serious level of... entitlement? Betrayal? An undertone of disappointment? It's weirdly emotional. Linux is an open source project, and unless you're personally involved with the maintainers, Linus Torvalds personally does not owe you anything. Open Source goes both ways: It's free of charge, and free of warranty.
And for the entirety of this video, he never stops being a fence sitting curmudgeon. AI-reported bugs drown issue trackers, so the Kernel team fights fire with fire. That's fine. I can personally tell you that I'd rather have the nonsense filtered out first before I actually start working on issues that matter to the open source project I'm voluntarily contributing to. What would Mr Gerard prefer? That maintainers waste their time trawling through mountains of garbage? Would he like to see the maintainer's time wasted? At least he can sleep soundly knowing that No AI Was Used, Promise!
I would put another paragraph here, but the man's just exhausting me with his insufferable presentation style. It's one thing to have an opinion, even a contrarian one, but only shitting on the project without any suggestions as to how to actually fix the problem he claims to care so much about makes me... eh, you get it.
I think for people who are quite emotionally invested in hating AI, the recent developments in AI programming are not going down well. All of the arguments about ethics, environment, financial...
I think for people who are quite emotionally invested in hating AI, the recent developments in AI programming are not going down well.
All of the arguments about ethics, environment, financial stability, etc are valid, but the reality is that AI programming tools really do work, and they work really well in the last few months. People like Linus are going to use them if it allows building the best Linux possible.
It's hard to get a man to believe that which his salary depends on him not believing. I've always tried to not live by that quote. Operate close to reality. Even if it doesn't pay well for the...
the recent developments in AI programming are not going down well
It's hard to get a man to believe that which his salary depends on him not believing.
I've always tried to not live by that quote. Operate close to reality. Even if it doesn't pay well for the moment, proximity to reality is one of your greatest assets. Seeing what others don't is a huge life advantage in the long run.
In my experience, AI-generated code is mediocre at-best, and usually just flat-out bad. Maybe it's because I'm in a more "niche" field of software engineering (embedded C++ and Python), but I'm...
... but the reality is that AI programming tools really do work, and they work really well in the last few months.
In my experience, AI-generated code is mediocre at-best, and usually just flat-out bad. Maybe it's because I'm in a more "niche" field of software engineering (embedded C++ and Python), but I'm not impressed with what codex is spitting out in the latest-and-greatest models (we've got a corporate OpenAI account and our CEO is pushing it hard). My biggest problem is that the code being generated is wildly over-engineered and highly complex for even a simple fix. Codex wants to add 8 classes (some of which already existed elsewhere in the code base) and 4 modules to my project for what would otherwise be maybe a 50-line change.
Or my coworker who just opened a +11000/-300 PR where the majority of the code was writing utility scripts in Python. One of them was building a custom SSH connection library from scratch. I had to tell him to just usefabric or paramiko rather than rolling his own. But everyone is so AI-pilled that this is the better way to do it...
I'm curious how people could be having a better experience with it? Is it that it is not great with the languages we use? The technologies we employ? Are my co-workers and I just really bad at prompting?
I think it depends on the degree of freedom you give it. I typically always watch what it is doing and review it really hard. I'm still winning and as a team we landed a PR deleting 10kloc this...
I think it depends on the degree of freedom you give it. I typically always watch what it is doing and review it really hard. I'm still winning and as a team we landed a PR deleting 10kloc this week. AI doesn't have to inexorably lead to bloat. We have landed many changes each savings worth thousands a month on AWS just by doing simple things like pointing an agent at metrics for a service and configuration and catching low hanging fruit like "your compression and batch settings have drifted from reality". Stuff that sure, anyone could do if they had a spare 30m, but Claude can make an artifact in 2m + short MR, that can be canaried in production in less time that it would have taken a human to do.
Do they write good code? Eh. I keep hearing that you can skill your way into making the bot behave like you, I don't buy it. Or I am in the same boat as you and I suck at writing skills. I really am bullish on ai for being in the details, but not on writing excellent code yet.
I really like it for triage and assisted code writing. I don't like it at all in "software factory" mode. Anthropic can't even get scroll back working properly Linux in Claude code.
That's kind of what I'm getting at - I don't think these tools are writing reliable, production-ready code, but I'm hearing from people/reading comments that say these tools are amazing and can do...
That's kind of what I'm getting at - I don't think these tools are writing reliable, production-ready code, but I'm hearing from people/reading comments that say these tools are amazing and can do everything and that software engineers will be obsolete soon. I don't think we're anywhere close to that (yet).
I'm not sure how companies are training their AI models for writing good code, but if they're just naively consuming every bad snippet of code posted to Stack Overflow or parsing every repository on GitHub, including the repos made by college kids hacking shit together to get through their next class, then it's never going to be that good... Hopefully they're not doing that, but you never know.
I'm going to be slightly mean and say that if you are happy across the board with AI output then it's not in factory mode (i.e. you are sweating the details in the loop) or you aren't reading the...
I'm going to be slightly mean and say that if you are happy across the board with AI output then it's not in factory mode (i.e. you are sweating the details in the loop) or you aren't reading the output or you aren't very good at software and this can't tell that the outputs are bad.
That is to say I think we agree? :) I was mostly trying to offer a corroborating view. I do a mix of embedded and dist sys (we sell hardware + associated software services).
But it has no semblance of sentinence to determine what that quality bar is. On more than one occasion a Opus 5+ prompt deployed an ancient outdated library instead the modern best practice....
But it has no semblance of sentinence to determine what that quality bar is. On more than one occasion a Opus 5+ prompt deployed an ancient outdated library instead the modern best practice. Because the old outdated library has a huge footprint on stackoverflow, and it was best practice until say circa 2018.
Reenforcement learning is how you get a bot deploying JQuery in 2026.
Tbf, I was once that junior engineer doing the same thing... I wish that Stack Overflow wasn't so terrible and had actually come up with a way to share knowledge with all the new AI tools and get...
Tbf, I was once that junior engineer doing the same thing...
I wish that Stack Overflow wasn't so terrible and had actually come up with a way to share knowledge with all the new AI tools and get updated in turn.
I'm a senior level and I still do it. Just our grey matter will usually help make those judgement calls better. Usually. That, and I'm not selling "automated StackOverflow as a service."
I'm a senior level and I still do it. Just our grey matter will usually help make those judgement calls better. Usually.
That, and I'm not selling "automated StackOverflow as a service."
Are you finding LLMs still do that? This was extremely common behavior back in Sonnet 3.5 days, but modern models are better at avoiding this in my experience.
Are you finding LLMs still do that? This was extremely common behavior back in Sonnet 3.5 days, but modern models are better at avoiding this in my experience.
I’m not even talking that advanced, or about code quality overall. The particular behaviour of copying code multiple times that vord is referring to, I’ve found has largely been a non-issue from...
I’m not even talking that advanced, or about code quality overall. The particular behaviour of copying code multiple times that vord is referring to, I’ve found has largely been a non-issue from around the Claude 4.5 series. Obviously, curious if others are seeing differently.
Had Fable 5.1 do it about a week ago, though it was not "just created." It did it to "prevent API breakage." It might have had to with there being a database schema. The change is one which would...
Had Fable 5.1 do it about a week ago, though it was not "just created."
It did it to "prevent API breakage."
It might have had to with there being a database schema. The change is one which would have added a table column, but to avoid a data migration just created a second table.
Yes, from that example, your coworker is clearly bad at prompting if they're rewriting SSH and asking you to review 11k lines. The tool might have output something of poor quality, but if your...
Yes, from that example, your coworker is clearly bad at prompting if they're rewriting SSH and asking you to review 11k lines. The tool might have output something of poor quality, but if your coworker couldn't detect that before asking for a code review, then what faith should one have in their ability to use the tool? For your own experience, if you find that Codex is doing more than you wanted it to do, then describe what you think it should have done instead and see if the next time, the prompt being more descriptive leads to the outcome you want. Or better yet, describe what the code change should have been, ask Codex to diff it, then ask Codex how the prompt could have been worded and store that difference in your expectations vs Codex's expectation in AGENTS.md.
If I have to spell out exactly what I want in excruciating detail, I might as well spend the 30 extra minutes to code it myself instead of spending an hour reviewing what spit out to make sure its...
If I have to spell out exactly what I want in excruciating detail, I might as well spend the 30 extra minutes to code it myself instead of spending an hour reviewing what spit out to make sure its correct.
The issue is that that is a big "if". Even if I trust Linus (and I don't know if I do), I don't trust 99.9% of other programmers to have his discipline and bar of quality.
People like Linus are going to use them if it allows building the best Linux possible.
The issue is that that is a big "if". Even if I trust Linus (and I don't know if I do), I don't trust 99.9% of other programmers to have his discipline and bar of quality.
The dichotomy is I think between people who are willing to put aside those other concerns and those that believe it unethical to put them aside. The quality of the code is primary to only the...
All of the arguments about ethics, environment, financial stability, etc are valid, but the reality is that AI programming tools really do work
The dichotomy is I think between people who are willing to put aside those other concerns and those that believe it unethical to put them aside. The quality of the code is primary to only the first group.
Here is the thing. The language these people use (most, not all of them) is about code quality and “slop”. If they want to be against it on ethical grounds, okay. But they don’t get to claim poor...
Here is the thing. The language these people use (most, not all of them) is about code quality and “slop”. If they want to be against it on ethical grounds, okay. But they don’t get to claim poor quality without mentioning ethics and have use steel man their arguments using reasoning that they themselves don’t use.
Let’s engage with the actual arguments people make, not hypothetical ones they could make.
There’s a tendency for people to have their cake and eat it when describing threatening things they don’t like. AI is both a grievous threat to human employment and is also useless slop...
There’s a tendency for people to have their cake and eat it when describing threatening things they don’t like. AI is both a grievous threat to human employment and is also useless slop generators. XYZ politician will both destroy everything if elected and also super stupid.
It’s causes some logical contradictions but I think people just prefer when someone or something bad is just bad in every way.
We're actively seeing this happen in real time. Our president's cabinet from his first term almost unilaterally said he is unfit for office. His vanity and willingness to side with whoever panders...
XYZ politician will both destroy everything if elected and also super stupid.
We're actively seeing this happen in real time. Our president's cabinet from his first term almost unilaterally said he is unfit for office. His vanity and willingness to side with whoever panders to him temporarily make him extremely manipulable. And yet he is also constantly signing EOs that are harmful to the environment and to institutions that have stood for decades.
AI is both a grievous threat to human employment and is also useless slop generators.
It can be both. If the output is bad but the people who make decisions don't have the experience to know it is bad, they won't care. They simply accept the faster output as a good sign and use it as a reason to reduce jobs.
In both cases you have people without relevant experience at the helm, and those who will see the actual consequences are those with the experience that the leaders need. This isn't a logical contradiction, it's the Peter Principle and Dunning-Krueger at the top.
Ding, ding, ding. A metric ton, if not the majority, of business decisions boil down to "how can I make this shittier to lower costs without losing customers." If it lets a feature give the...
If the output is bad but the people who make decisions don't have the experience to know it is bad, they won't care.
Ding, ding, ding. A metric ton, if not the majority, of business decisions boil down to "how can I make this shittier to lower costs without losing customers."
If it lets a feature give the appearance of shipping faster for less money, that boosts this quarters numbers, and tech debt is the next CIO's problem.
I think you describe it very well. It’s very frustrating to me since I feel it makes arguments weaker. If there are actually good, valid reasons to not like something, why do we bother shoveling...
I think you describe it very well. It’s very frustrating to me since I feel it makes arguments weaker. If there are actually good, valid reasons to not like something, why do we bother shoveling all the stupid, benign, and easily disproven reasons in too? It’s a common thing when people talk about apple, and it seems like AI is attracting the same argumentation style.
Why not both? Most people don't bring up the ethical arguments because they realize that corporations do not listen to ethical appeals. They should theoretically appeal to the idea of "I do not...
But they don’t get to claim poor quality without mentioning ethics and have use steel man their arguments using reasoning that they themselves don’t use.
Why not both? Most people don't bring up the ethical arguments because they realize that corporations do not listen to ethical appeals. They should theoretically appeal to the idea of "I do not like what this product outputs, I'm not going to give you money". So on the internet you'll hear a lot of the latter. because the former conversation does not fundamentally change.
I’ve only been briefly seeing Gerard, and yeah, I get the same reactionary vibe. Same as Zitron. He did a hard pivot from being anti-blockchain to this (I only really knew about Gerard because...
I’ve only been briefly seeing Gerard, and yeah, I get the same reactionary vibe. Same as Zitron. He did a hard pivot from being anti-blockchain to this (I only really knew about Gerard because Molly White had a recommendation for it).
It frustrates me because it drowns out actually good critique. For instance, this article on decoherence is, in my opinion, outstanding writing on some of the pitfalls we may run into.
I'd say we're already running into it hard, we just haven't hit the theoretical limit. That decoherence essay really put a bow on a phrase we've been using at work: Claude is the best tech-debt...
I'd say we're already running into it hard, we just haven't hit the theoretical limit. That decoherence essay really put a bow on a phrase we've been using at work:
Claude is the best tech-debt generator on the planet.
It's pretty good at spitting out smaller one-shots. But as you iterate on it, if you never go back and look over the code and refactor it yourself, it degenerates quickly. The larger the codebase gets, the more likely it breaks apart at the seams because it can't keep everything in context.
Especially since by default it feels like the models will spit out 80 tests for every 100 lines of code. Which then will become further tech debt that needs to be maintained.
Plus a lot of people on Tildes make similar arguments, which I also see in other left wing communities. We aren't special or immune. But yeah. Reactionaries gonna react.
a level of disingenuous cherry picking you really only see in the open source community and, maybe some right wing discord servers
Plus a lot of people on Tildes make similar arguments, which I also see in other left wing communities. We aren't special or immune.
Is it? How about we don't rely on AI reported bugs to begin with? We don't make software to fulfill the needs of LLM's, so they cannot triage for human needs. Why are their reports treated with...
AI-reported bugs drown issue trackers, so the Kernel team fights fire with fire. That's fine.
Is it? How about we don't rely on AI reported bugs to begin with?
We don't make software to fulfill the needs of LLM's, so they cannot triage for human needs. Why are their reports treated with the same weight as a human who ran into a bug and impacted them enough to report it?
I don't disagree, but look at it from a pragmatist perspective. How exactly will you, reliably, filter out LLM generated bug reports if they're not necessarily labelled as such? The advent of AI...
I don't disagree, but look at it from a pragmatist perspective. How exactly will you, reliably, filter out LLM generated bug reports if they're not necessarily labelled as such? The advent of AI made all bug reports more common, regardless of if they were marked as AI assisted or not. And, to play devil's advocate, it would also exclude real bug reports that had an AI trace the issue back to a specific place in code - not fully AI generated, but definitely assisted, and may help a maintainer isolate the issue.
Same way you filter out potentially vague human reports. Have a form, require minimal repro if appropriate, have the team able to triage priority. I don't think anything fundamentally changes in...
How exactly will you, reliably, filter out LLM generated bug reports if they're not necessarily labelled as such?
Same way you filter out potentially vague human reports. Have a form, require minimal repro if appropriate, have the team able to triage priority.
I don't think anything fundamentally changes in the dynamic. If all these LLM generated reports were high quality, it wouldn't be a problem to begin with. If this makes AI generated reports higher quality to hide its Ai-ness, then in the words of XKCD: "Mission. Fucking. Accomplished"
I really recommend watching GregKH's talk on security in the age of llm. Goes into the details about how they are handling it, I really enjoyed it. https://www.youtube.com/watch?v=NnV_cWeoo5Q
I really recommend watching GregKH's talk on security in the age of llm. Goes into the details about how they are handling it, I really enjoyed it.
I'd rather people spend more time thinking about unit tests scenarios, requirements, specifications, and architecture instead of the actual written code of any one particular function/method. I...
I'd rather people spend more time thinking about unit tests scenarios, requirements, specifications, and architecture instead of the actual written code of any one particular function/method. I think some of this anti-slop mindset was valid 1-2 years ago, but it barely holds up these days. The models have gotten substantially better. The only way I detect AI code vs human code these days is in the verbosity of the PR descriptions and overly commented code. If those are prompted tighter / cleaned up manually I can't really tell the difference anymore.
There are moral or ethical conversations to be had, but the quality argument is much harder now.
I have fully embraced Linux over the past couple years and really do not like Linus’ recent stance, right wing dog whistles, and sucking up to corporate. What can one do?
I have fully embraced Linux over the past couple years and really do not like Linus’ recent stance, right wing dog whistles, and sucking up to corporate. What can one do?
Didn't Linus just say that he doesn't care if a commit is made by AI as long as it's good code? It was a fairly milquetoast statement. The right wing dog whistles is a specific repo that's being...
Didn't Linus just say that he doesn't care if a commit is made by AI as long as it's good code? It was a fairly milquetoast statement.
The right wing dog whistles is a specific repo that's being funded/made by one specific dipshit. There's nothing that Linux can do against that, and frankly speaking it shouldn't try. It would be a huge blow against GPL.
That is not what I am referencing. I am referring to the video I linked where the speakers argued Linus is rewriting the history of Linus as not being a Social Warrior project is one not true and...
The right wing dog whistles is a specific repo that's being funded/made by one specific dipshit.
That is not what I am referencing. I am referring to the video I linked where the speakers argued Linus is rewriting the history of Linus as not being a Social Warrior project is one not true and is two specifically using conservative reactionary language.
Linus is pretty famously not right wing, for example: https://social.kernel.org/notice/AWSXomDbvdxKgOxVAm
Exemplary
Linus is pretty famously not right wing, for example:
I strongly suspect I am one of those “woke communists” you worry about. But you probably couldn’t actually explain what either of those words actually mean, could you?
I’m a card-carrying atheist, I think a woman’s right to choose is very important, I think that “well regulated militia” means that guns should be carefully licensed and not just randomly given to any moron with a pulse, and I couldn’t care less if you decided to dress up in the “wrong” clothes or decided you’d rather live your life without feeling tied to whatever plumbing you were born with.
And dammit, if that all makes me “woke”, then I think anybody who uses that word as a pejorative is a f*cking disgrace to the human race. So please just unfollow me right now.
From a Finnish perspective, he's like an unironically enlightened centrist with an european liberal slant. There's a reason why he moved out of the welfare heaven of Finland to USA and I suspect...
From a Finnish perspective, he's like an unironically enlightened centrist with an european liberal slant. There's a reason why he moved out of the welfare heaven of Finland to USA and I suspect it's not about the weather.
I haven't watched the video but read the blog post. I've been seeing a lot of Pivot to Ai's content recently and think they're mostly riding the wave of being a reactionary. The developing cult...
I haven't watched the video but read the blog post. I've been seeing a lot of Pivot to Ai's content recently and think they're mostly riding the wave of being a reactionary. The developing cult assuming AI is flawless and everyone wants it is weird and isn't productive. The developing cult declaring that AI is worthless and has no use is understandable and also isn't productive.
Yes every company involved should be prosecuted for at least copyright violations. No we don't need nuclear/coal powered data centers. Yes capitalism is breaking down under the strain of concentration of resources and lack of regulation. Good lord is the economy of promising basically all of computer component manufacturing for multiple years to things that might not get built on loaned money a clusterfuck.
I can use my gaming computer GPU to help modify a personal fork to add support for the OIDC provider I use for other local services. I have to monitor what it's doing and provide framework and stop it from going on tangents. But I don't have to write a few hundred lines of tests and boilerplate or hunt down and adapt similar examples from stack overflow. That's pretty useful.
The Linux group's stance is that they're a mostly volunteer non-profit organization and the overhead to sniff out if something "might be" AI coded or assisted, above and beyond testing if the code works and is bug free, is literally impossible for them.
The full quote (which he does link) is about exactly the religious fervor that some corners of the internet are developing about anything without an instant AI ban. Linus has been using similar language since the 90s, including in Revolution OS, about the kernel. He's a blunt Swedish Finnish ass who wants the tech to work better and without the artificial constraints that come from business models, the other aspects that open source enables are a pleasant side benefit, but the code is impartial.
Funny. You can do whatever you want, from switching to BSD or even more niche open-source software to forking the whole thing yourself. But you probably meant, what can one do without devoting a...
What can one do?
Funny. You can do whatever you want, from switching to BSD or even more niche open-source software to forking the whole thing yourself.
But you probably meant, what can one do without devoting a significant fraction of your life's time and energy to it. Because utility software is not truly a priority for most of us. In that case, I suggest coming to terms with your available capacity and focusing on your true priorities in life. Possibly by contributing to local political efforts, which is where most of us have the most leverage.
what if my "true priorities" lie precisely in maintaining quality and community, and not letting swaths of civilization come under seige of greedy corporate interests?
focusing on your true priorities in life.
what if my "true priorities" lie precisely in maintaining quality and community, and not letting swaths of civilization come under seige of greedy corporate interests?
Setting maybe aside your concerns about reactionary language, I can’t really discern from Linus’ comments how “slop” is coming to the Kernel project, as opposed to any LLM contribution undergoing the regular process of review, sign-off, and ultimately responsible (human) person via the sign-off/DCO.
I get the feeling this issue isn’t about code quality alone, as “slop” would imply to me at first. But if that’s really the case, maybe a better name is required. “Opposing LLM contribution on moral grounds” or similar, perhaps?
If that is not the case however, could someone enlighten me where you see cracks in the way Linux/-s wants to handle this topic going forward? Genuinely asking, as I felt the decisions so far to be coming from a mostly realist/pragmatic standpoint.
It is my opinion that proponents who take the stance that LLMs are "just a tool" in programming, and that you can embrace them while still maintaining enough institutional knowledge for humans to meaningfully review code and merge requests, are either deluding themselves or intentionally deluding others.
The act of reading and reviewing code is tedious, and anathema for most people who got into programming for the love of the craft. Everyone is excited about tools that make programming easier, but nobody is excited about tools that convert the job from "building things" into "making sure the LLM generated code works right." You might start out with good intentions (as I believe Linus is doing with the kernel), but that course only leads to one ultimate destination.
In any open source project that does not take a stance against LLMs, human developers are not going to review and validate every line of code that gets merged, and eventually human developers will not review and validate any line of code that gets merged. I mean removing human effort from the loop is the whole point of LLMs. As the tools get better and the developers get more and more burnt out, LLM code is going to be "just trusted" more and more. Skim the code, "looks right," click merge. Eventually you don't even skim the code anymore. Click "review code," an LLM says "looks right," click merge. Eventually you don't even click buttons anymore. LLM generated the code, LLM says "looks right," LLM merges. Talk to anyone with a job in software engineering, we're all watching this exact scenario play out in real-time.
I think there are two paths projects can take:
Linus and others might think there's some middle-ground between the two; I think that's just wishful thinking. But I also think it's wishful thinking to believe that path 1 is even viable in the long-term. The obscenely wealthy companies developing and selling LLMs (and their army of flying monkies decrying the end of software engineering as a profession) are certainly doing everything within their power to ensure everyone believes LLMs are the future. It's very depressing because I do also think there's a possible world where LLMs really could be "just a tool" that actually empower human developers and creators with new ways to excel at their crafts, but sadly building those kinds of tools wouldn't extract as much wealth into the pockets of shareholders so they'll probably never come to be.
Anyway, all of that is to say, the thing that I and other Linux users are mourning is the fact that allowing LLMs into any project almost certainly spells the end of human involvement in the source code. Maybe not today, maybe not tomorrow, but eventually. There are people out there who think that's a good thing, because they believe LLMs will do a better job anyway, and maybe that's true (or will one day soon be true), but it still feels like we're losing something meaningful.
This seems pretty doomer, but I understand.
I'm not sure what this means. LLMs don't prompt themselves, or have their own goals/ideas/direction for a project. They also aren't very good at finding UI/UX bugs or understanding what makes something feel nice to use, one of the most important aspects of any technology.
The open weight models are something like ~6 months behind the frontier models. If the frontier models ever plateau, it's unclear what their competitive advantage will be, especially if they continue going down the path of locking down their models to their own harnesses.
Perhaps somewhat inversely, price subsidization is inherently not sustainable. Businesses will have to decide between how to split costs between their own workforce vs an external vendor. Businesses that bet wrong, eliminating all humans, will either be left holding the bag OR up the creek without a paddle OR have no special sauce and be overtaken by someone who does.
LLMs cannot provide a direct replacement for domain knowledge gained first-hand within specific communities. That's not data you can scrape from a website.
Yeah, I'd certainly consider that a false dichotomy. From a really high level, there's way too much nuance in this domain for there to be only two options.
I know for the codebase I maintain, it was already sprawling and inconsistent before LLMs. Getting agents to spit out anything useful required documenting (and starting to track) a lot of patterns (old vs new), assumptions, and use cases. So the end results for me have been:
I care deeply about code readability and maintainability for the projects I work on. I'm reviewing a lot more code than I used to, not less. This is all anecdotal though.
I can add another anecdote. The development process I've settled on doesn't look anything like "prompt then review results". It's human guided throughout and I understand what comes out of it deeply, roughly equivalent to the understanding I'd have if I'd written all of it by hand.
It's also slower than vibecoding, which is why there's a lot of slop being churned out right now, and there will be more to come. But agent assisted development doesn't automatically mean slop. It really is a tool that can be used well or used badly, and we're still figuring out what using it well looks like.
I am a bit unsure about this, but could we not liken LLMs to the introduction of compilers? Before those became common place people wrote assembly and even as far as the 90s people were better at writing assembly code than compilers could reasonably produce. This might still be the case today, but I doubt it. It is definitely not true for most developers! Compilers now take a high level descriptions of the operations we want to perform and boils it down to some bare essentials. Now with LLMs we take a sequence of fairly vaguely worded instructions and generate a sequence of operations that then compilers will boil down further. Now, an argument can be made that LLMs are not the same because agent's are not deterministic... but that is only kind of true. If we discount hardware differences LLMs are still "just" an operation on input data and given the same seed you will get the same output. The same analogy could be made for compilers; if you give them slightly different input flags you can get wildly different results! So maybe we should take the last few decades of experience with working with software and see what we should keep around. Maybe an approach where we actually version control our prompts and other model parameters would be advisable? The point of building software has never been to create the software itself, it has always been to create something that solves a problem (or that entertains in the case of games, which can be viewed as the problem of boredom). So we never really care about the actual machine code anymore, maybe it is time we stopped caring about the source code too?
I hate it when people claim it's just like compilers.
Compilers are deterministic. If you get different results because of a compiler flag, anyone looking over the assembly or anyone familiar enough with your programming language and compiler can point and say "that's not right", and fix it. This is possible because the source code and flags are right there for anyone to see, and someone still understands the process that creates your software. If the same input creates different output, that's a compiler bug.
LLMs don't have that. English is a terrible programming language, wayyy too much ambiguity. If we're genuinely going the "it's just like compilers" route, then we would be checking prompts and LLM versions into version control (no AGENTS.md is not that). Although that might help us understand why the clanker code looks like it does, still, it's a stochastic chatbot, it's not deterministic, it's not a compiler, and proceeding as if it were will eventually create an incoherent mess.
LLMs, as far as I understand the underlying implementations, are deterministic. Given the same input (text, seed, temperature etc) it will give you the same output. They behave non-deterministic just because we usually currently run them that way. Committing the promts, models versions and other input into VCS would let you recreate the output deterministically. I think there is some nuances that might impact this in practice with how the service providers run the models (batching and hardware differences might impact the output, but I'm not sure as to what extent). Local models should mitigate that to a large extent of needed.
Way back were the compilers really deterministic? Are they that way even now? If I compile something on my laptop will I get the exact same binary as you would? I don't think so. Most likely there will be differences due to (in no particular order):
There are probably other confounding issues there too. Some people can probably understand those differences in detail, most devs probably shrug and don't really care. If it compiles and pass the tests we consider them equivalent to another build.
I agree that English is a terrible programming language. So maybe we should look at ways of optimizing the input so we get better output. Maybe some semi structured format for defining tasks and expectations? Point is that today most devs don't care if the compiled assembly is "an incoherent mess", potentially we shouldn't care if the source code we compile is either.
I haven't really landed in what I think. I kind of mourn the loss of the craft. I like programming as a discipline and I don't enjoy poking an agent to do the right thing. It does however look like that's the direction the industry is barreling towards...
Committing all the LLM run parameters to get deterministic output later might be interesting, but without the ability to poke and prod at how the model is created, will all of its training data, I'd argue it's still "opaque" output.
Of course, but all of those are well understood, maybe not always by users, but definetly by the compiler people. If I can't get a build environment where I can recreate the exact binary you get, same inputs, then I don't like this compiler. Compiler engineering does focus on this, and most of the times when you can't reproduce an artifact byte for byte, external platform dependencies are to blame, not the compiler.
Structured format for defining tasks is a programming language.
Structured format for defining expectations is programming and tests.
If the compiled assembly is an incoherent mess, the software will run like an incoherent mess, and we should definitely care, and we should poke at the compiler until we understand why it's doing what it is doing.
Yes, if it's a silly throwaway program where you don't really care, sure, who cares...
But as a basis for engineering practices, "(deep) understanding = wasted productivity" is a horrible set of practices for the industry to be barreling towards.
I still love computers and programming, but if the industry keeps going in this direction, I'll soon be mourning it as a profession, and only do it as a hobby.
Good points! While all structured formats aren't programming languages I get your point.
This part I don't necessarily think is true. A lot of compiled languages ship bloated binaries which can be really hard to unwind. Sometimes these run poorly too, but that's not a given. It's great if we can understand the underlying assembly, but that's not a requirement for functioning and working software. I'd even argue that as an industry pre llms few pieces of software had someone who fully understood it. If you go into legacy systems even the purpose of the system could be lost a decade ago... Now that doesn't mean we shouldn't care about software quality, just that the situation hasn't really changed much beside the pace at which (often por quality) software is produced.
Off topic; anecdotally, from what I've seen in software, every time someone sees a solution in "a structured format for defining tasks and expectations", it starts as a structured text config file and then tends to eventually evolve (or devolve, depending on how you look at it) into a programming language to express the tasks and expectations.
This is always seen as the result of an unavoidable trade-off between "development speed" and "quality". Looking at the state of software in the world, including the pre-LLMs era, I'd say the bloat is too much.
I suspect the truth is that 90% of the extra bloat we see in the software running today comes from bad software practices like "use the mega-kitchen-sink-framework, just in case, it's considered very productive and we must ship fast!". The extra development speed rarely justifies the boat.
Tooling that incorporates programming language research into their designs is amazing nowadays, and I think if we genuinely reconsidered how software engineers learn to deal with complexity over time a bit more deeply than "look at Java Object Oriented Programming, put the complexities in the classes and problems gone", by actually using tools to make life easier, we could have better development time & less bloat. LLMs can benefit a lot from static analysis too, although sometimes I see them do weird workarounds just to avoid a compiler warning.
Maybe modern CPUs and smart interpreters can do their magic to the point where users don't feel the difference, but still the software uses more resources than it should. If we keep building on top of it, eventually it will become a problem. If it was a bloated incoherent mess, either fix it or accept that it will get worse.
Then the underlying assembly is doing what you expect it to do, and you trust that the compiler is working correctly, as the compiler is deterministic, and it follows a well-specified and debuggable set of rules that create the assembly. You've just traded the understanding requirement for trusting that the compiler engineers understand, so you can worry about your source code, instead of assembly. You now focus on making sure the source code follows the rules of the compiler. If it does, and you didn't introduce some logic but into your code, then you will have working software.
The understanding is still a requirement for working functioning and working software, because what is actually happening on the CPU is still raw instructions.
LLMs are not compilers, they're not the same process or type of abstraction: I can't trust an LLM, and, because of their design, they will not provide understanding, even if it was made deterministic with a set seed and temperature and all that.
If I can get the LLM to produce correct code, I will only be able to trust that my input was correct by looking at the produced source code. There simply is no set of rules it actually follows, at least not that a human can understand: it "simply" does matrix multiplications and spits out the highest scoring tokens.
For a one-shot program, fine, check the produced source code and go on.
But on a project that needs to evolve over time, which needs to be understood, if the "official description" or "high level definition" of what the software actually does on the CPU lives inside of an LLM, or if the specifications are the input to an LLM, as if really using it like a "higher level compiler", I still don't know what the software will actually do.
The best we can get is guesswork, which is why it's not the same kind of abstraction as compilers, where high level language (+ compilation flags) is enough.
BTW all of this ignores the ethical and technical limitations of LLMs.
When addressing "compilers abstraction <> LLM abstraction", the point is that someone you trust understands it, that it is well specified, that it always follows rules, enough to poke and prod at so that, if you really wanted to, eventually you can find out. All the passes between "high level specification" to "raw machine code" are understood by a group of people that we trust enough.
This doesn't work for an LLM: we don't know how to follow the LLMs "rules", because even if it's deterministic, some slight change might make it hallucinate completely wrong code, and I have no idea how to actually figure out why. If that can happen for every slight change, it's going to be a useless abstraction, especially compared to what compilers allowed us to do when moving from assembly to something like C.
I mean legacy software is actually a good example of why vibecoded projects turn into an incoherent mess: the understanding of why the code does things is missing.
I agree, I mostly wanted to address the "liken the introduction of LLMs to the introduction of compilers" part. I think the state of software engineering was already a disaster, noticably affecting the quality of software. Now it's accelerating in the wrong direction.
I can see that becoming true in the future. Maybe the near future. But right now I think the coding agents just have too limited context windows and too weak prompt adherence. They can’t keep a whole project vision and architecture in “mind” without slowly drifting away from it over time. They'll start introducing “alternative” implementations that are odds with how the project was intended to function. They'll undo optimizations that were lovingly crafted by people who did it that way for a reason. They'll deliver what they were prompted for, but at the cost of subtle regressions in other features they weren't prompted about.
If they were able to consistently avoid those pitfalls, I'd agree that the source code doesn't matter anymore. Until then, software that’s worked fine for years is going to start getting… flaky. Buggy. Not just functionally, but visually too. All that nice pixel-perfect UI that frontend guys like me (until I left the field last year) pored over to get the colors and margins and hover states and transition timings juuuust right and slick and professionally branded… the AI doesn't care about that. It'll produce something good enough at first glance. Missing polish in subtle ways, but close, and that'll ship. Then the next PR that touches the same components will drift a bit further. Just gradually getting crappier over time, until the user base jumps ship because the system is just unpleasant to use now and feels like nobody at the company cares about quality anymore.
It’s early yet. I don’t think those limitations are going to be with us forever. But in the meantime I think they’re going to be a landline for a lot of software projects. I’m watching intently to see how many well-regarded companies’ products just start devolving like this. My guess is once things get to a certain point it’s too late to reverse course because the whole codebase is poisoned, and that’s just the death sentence for that project.
Edit, but posting standalone because I think it’s worth it:
LWN: Development statistics for the 7.2 kernel
About the article’s last plotted graph (take a look!):
(sentence in bold highlighted by me)
I had to stop this video multiple times because I could not handle the smug superiority of Mr. Gerard. Letting alone that the entire justification here is that "I say AI is slop, and therefore all code ever made with AI is slop by default, and that means it is bad" is a level of disingenuous cherry picking you really only see in the open source community and, maybe some right wing discord servers, there's a serious level of... entitlement? Betrayal? An undertone of disappointment? It's weirdly emotional. Linux is an open source project, and unless you're personally involved with the maintainers, Linus Torvalds personally does not owe you anything. Open Source goes both ways: It's free of charge, and free of warranty.
And for the entirety of this video, he never stops being a fence sitting curmudgeon. AI-reported bugs drown issue trackers, so the Kernel team fights fire with fire. That's fine. I can personally tell you that I'd rather have the nonsense filtered out first before I actually start working on issues that matter to the open source project I'm voluntarily contributing to. What would Mr Gerard prefer? That maintainers waste their time trawling through mountains of garbage? Would he like to see the maintainer's time wasted? At least he can sleep soundly knowing that No AI Was Used, Promise!
I would put another paragraph here, but the man's just exhausting me with his insufferable presentation style. It's one thing to have an opinion, even a contrarian one, but only shitting on the project without any suggestions as to how to actually fix the problem he claims to care so much about makes me... eh, you get it.
I think for people who are quite emotionally invested in hating AI, the recent developments in AI programming are not going down well.
All of the arguments about ethics, environment, financial stability, etc are valid, but the reality is that AI programming tools really do work, and they work really well in the last few months. People like Linus are going to use them if it allows building the best Linux possible.
It's hard to get a man to believe that which his salary depends on him not believing.
I've always tried to not live by that quote. Operate close to reality. Even if it doesn't pay well for the moment, proximity to reality is one of your greatest assets. Seeing what others don't is a huge life advantage in the long run.
In my experience, AI-generated code is mediocre at-best, and usually just flat-out bad. Maybe it's because I'm in a more "niche" field of software engineering (embedded C++ and Python), but I'm not impressed with what codex is spitting out in the latest-and-greatest models (we've got a corporate OpenAI account and our CEO is pushing it hard). My biggest problem is that the code being generated is wildly over-engineered and highly complex for even a simple fix. Codex wants to add 8 classes (some of which already existed elsewhere in the code base) and 4 modules to my project for what would otherwise be maybe a 50-line change.
Or my coworker who just opened a +11000/-300 PR where the majority of the code was writing utility scripts in Python. One of them was building a custom SSH connection library from scratch. I had to tell him to just use
fabricorparamikorather than rolling his own. But everyone is so AI-pilled that this is the better way to do it...I'm curious how people could be having a better experience with it? Is it that it is not great with the languages we use? The technologies we employ? Are my co-workers and I just really bad at prompting?
I think it depends on the degree of freedom you give it. I typically always watch what it is doing and review it really hard. I'm still winning and as a team we landed a PR deleting 10kloc this week. AI doesn't have to inexorably lead to bloat. We have landed many changes each savings worth thousands a month on AWS just by doing simple things like pointing an agent at metrics for a service and configuration and catching low hanging fruit like "your compression and batch settings have drifted from reality". Stuff that sure, anyone could do if they had a spare 30m, but Claude can make an artifact in 2m + short MR, that can be canaried in production in less time that it would have taken a human to do.
Do they write good code? Eh. I keep hearing that you can skill your way into making the bot behave like you, I don't buy it. Or I am in the same boat as you and I suck at writing skills. I really am bullish on ai for being in the details, but not on writing excellent code yet.
I really like it for triage and assisted code writing. I don't like it at all in "software factory" mode. Anthropic can't even get scroll back working properly Linux in Claude code.
That's kind of what I'm getting at - I don't think these tools are writing reliable, production-ready code, but I'm hearing from people/reading comments that say these tools are amazing and can do everything and that software engineers will be obsolete soon. I don't think we're anywhere close to that (yet).
I'm not sure how companies are training their AI models for writing good code, but if they're just naively consuming every bad snippet of code posted to Stack Overflow or parsing every repository on GitHub, including the repos made by college kids hacking shit together to get through their next class, then it's never going to be that good... Hopefully they're not doing that, but you never know.
I'm going to be slightly mean and say that if you are happy across the board with AI output then it's not in factory mode (i.e. you are sweating the details in the loop) or you aren't reading the output or you aren't very good at software and this can't tell that the outputs are bad.
That is to say I think we agree? :) I was mostly trying to offer a corroborating view. I do a mix of embedded and dist sys (we sell hardware + associated software services).
They're not. They use reinforcement learning and have a higher quality bar to avoid the lowest common denominator.
But it has no semblance of sentinence to determine what that quality bar is. On more than one occasion a Opus 5+ prompt deployed an ancient outdated library instead the modern best practice. Because the old outdated library has a huge footprint on stackoverflow, and it was best practice until say circa 2018.
Reenforcement learning is how you get a bot deploying JQuery in 2026.
Tbf, I was once that junior engineer doing the same thing...
I wish that Stack Overflow wasn't so terrible and had actually come up with a way to share knowledge with all the new AI tools and get updated in turn.
I'm a senior level and I still do it. Just our grey matter will usually help make those judgement calls better. Usually.
That, and I'm not selling "automated StackOverflow as a service."
Oh, you need a small feature added to a class I just built?
Let me duplicate that class and suffix it '_new_feature.'
Are you finding LLMs still do that? This was extremely common behavior back in Sonnet 3.5 days, but modern models are better at avoiding this in my experience.
Eh, Fable is better in matters of taste, but most people don't have access to that for work.
I’m not even talking that advanced, or about code quality overall. The particular behaviour of copying code multiple times that vord is referring to, I’ve found has largely been a non-issue from around the Claude 4.5 series. Obviously, curious if others are seeing differently.
Had Fable 5.1 do it about a week ago, though it was not "just created."
It did it to "prevent API breakage."
It might have had to with there being a database schema. The change is one which would have added a table column, but to avoid a data migration just created a second table.
I still encounter this with the latest models and harnesses. Not quite bleeding edge but still extremely new.
Yes, from that example, your coworker is clearly bad at prompting if they're rewriting SSH and asking you to review 11k lines. The tool might have output something of poor quality, but if your coworker couldn't detect that before asking for a code review, then what faith should one have in their ability to use the tool? For your own experience, if you find that Codex is doing more than you wanted it to do, then describe what you think it should have done instead and see if the next time, the prompt being more descriptive leads to the outcome you want. Or better yet, describe what the code change should have been, ask Codex to diff it, then ask Codex how the prompt could have been worded and store that difference in your expectations vs Codex's expectation in AGENTS.md.
If I have to spell out exactly what I want in excruciating detail, I might as well spend the 30 extra minutes to code it myself instead of spending an hour reviewing what spit out to make sure its correct.
The issue is that that is a big "if". Even if I trust Linus (and I don't know if I do), I don't trust 99.9% of other programmers to have his discipline and bar of quality.
The dichotomy is I think between people who are willing to put aside those other concerns and those that believe it unethical to put them aside. The quality of the code is primary to only the first group.
Here is the thing. The language these people use (most, not all of them) is about code quality and “slop”. If they want to be against it on ethical grounds, okay. But they don’t get to claim poor quality without mentioning ethics and have use steel man their arguments using reasoning that they themselves don’t use.
Let’s engage with the actual arguments people make, not hypothetical ones they could make.
There’s a tendency for people to have their cake and eat it when describing threatening things they don’t like. AI is both a grievous threat to human employment and is also useless slop generators. XYZ politician will both destroy everything if elected and also super stupid.
It’s causes some logical contradictions but I think people just prefer when someone or something bad is just bad in every way.
We're actively seeing this happen in real time. Our president's cabinet from his first term almost unilaterally said he is unfit for office. His vanity and willingness to side with whoever panders to him temporarily make him extremely manipulable. And yet he is also constantly signing EOs that are harmful to the environment and to institutions that have stood for decades.
It can be both. If the output is bad but the people who make decisions don't have the experience to know it is bad, they won't care. They simply accept the faster output as a good sign and use it as a reason to reduce jobs.
In both cases you have people without relevant experience at the helm, and those who will see the actual consequences are those with the experience that the leaders need. This isn't a logical contradiction, it's the Peter Principle and Dunning-Krueger at the top.
Ding, ding, ding. A metric ton, if not the majority, of business decisions boil down to "how can I make this shittier to lower costs without losing customers."
If it lets a feature give the appearance of shipping faster for less money, that boosts this quarters numbers, and tech debt is the next CIO's problem.
I think you describe it very well. It’s very frustrating to me since I feel it makes arguments weaker. If there are actually good, valid reasons to not like something, why do we bother shoveling all the stupid, benign, and easily disproven reasons in too? It’s a common thing when people talk about apple, and it seems like AI is attracting the same argumentation style.
Why not both? Most people don't bring up the ethical arguments because they realize that corporations do not listen to ethical appeals. They should theoretically appeal to the idea of "I do not like what this product outputs, I'm not going to give you money". So on the internet you'll hear a lot of the latter. because the former conversation does not fundamentally change.
I’ve only been briefly seeing Gerard, and yeah, I get the same reactionary vibe. Same as Zitron. He did a hard pivot from being anti-blockchain to this (I only really knew about Gerard because Molly White had a recommendation for it).
It frustrates me because it drowns out actually good critique. For instance, this article on decoherence is, in my opinion, outstanding writing on some of the pitfalls we may run into.
Wow, that decoherence essay was excellent. Thanks so much for sharing! Going to have to process that one for a while.
I'd say we're already running into it hard, we just haven't hit the theoretical limit. That decoherence essay really put a bow on a phrase we've been using at work:
Claude is the best tech-debt generator on the planet.
It's pretty good at spitting out smaller one-shots. But as you iterate on it, if you never go back and look over the code and refactor it yourself, it degenerates quickly. The larger the codebase gets, the more likely it breaks apart at the seams because it can't keep everything in context.
Especially since by default it feels like the models will spit out 80 tests for every 100 lines of code. Which then will become further tech debt that needs to be maintained.
For code degeneration itself, I've found SlopCodeBench to be an interesting benchmark at observing this.
Plus a lot of people on Tildes make similar arguments, which I also see in other left wing communities. We aren't special or immune.
But yeah. Reactionaries gonna react.
Is it? How about we don't rely on AI reported bugs to begin with?
We don't make software to fulfill the needs of LLM's, so they cannot triage for human needs. Why are their reports treated with the same weight as a human who ran into a bug and impacted them enough to report it?
I don't disagree, but look at it from a pragmatist perspective. How exactly will you, reliably, filter out LLM generated bug reports if they're not necessarily labelled as such? The advent of AI made all bug reports more common, regardless of if they were marked as AI assisted or not. And, to play devil's advocate, it would also exclude real bug reports that had an AI trace the issue back to a specific place in code - not fully AI generated, but definitely assisted, and may help a maintainer isolate the issue.
Same way you filter out potentially vague human reports. Have a form, require minimal repro if appropriate, have the team able to triage priority.
I don't think anything fundamentally changes in the dynamic. If all these LLM generated reports were high quality, it wouldn't be a problem to begin with. If this makes AI generated reports higher quality to hide its Ai-ness, then in the words of XKCD: "Mission. Fucking. Accomplished"
Yeah, that's the correct take. Mission fucking accomplished. Glad we agree
I really recommend watching GregKH's talk on security in the age of llm. Goes into the details about how they are handling it, I really enjoyed it.
https://www.youtube.com/watch?v=NnV_cWeoo5Q
Thanks for sharing this!
I'd rather people spend more time thinking about unit tests scenarios, requirements, specifications, and architecture instead of the actual written code of any one particular function/method. I think some of this anti-slop mindset was valid 1-2 years ago, but it barely holds up these days. The models have gotten substantially better. The only way I detect AI code vs human code these days is in the verbosity of the PR descriptions and overly commented code. If those are prompted tighter / cleaned up manually I can't really tell the difference anymore.
There are moral or ethical conversations to be had, but the quality argument is much harder now.
I have fully embraced Linux over the past couple years and really do not like Linus’ recent stance, right wing dog whistles, and sucking up to corporate. What can one do?
Didn't Linus just say that he doesn't care if a commit is made by AI as long as it's good code? It was a fairly milquetoast statement.
The right wing dog whistles is a specific repo that's being funded/made by one specific dipshit. There's nothing that Linux can do against that, and frankly speaking it shouldn't try. It would be a huge blow against GPL.
That is not what I am referencing. I am referring to the video I linked where the speakers argued Linus is rewriting the history of Linus as not being a Social Warrior project is one not true and is two specifically using conservative reactionary language.
Linus is pretty famously not right wing, for example:
https://social.kernel.org/notice/AWSXomDbvdxKgOxVAm
Thanks for linking that!
Linus is best described as a fairly well-principled person who is too damn busy to bother wasting time mincing words with people wasting his time.
From a Finnish perspective, he's like an unironically enlightened centrist with an european liberal slant. There's a reason why he moved out of the welfare heaven of Finland to USA and I suspect it's not about the weather.
I haven't watched the video but read the blog post. I've been seeing a lot of Pivot to Ai's content recently and think they're mostly riding the wave of being a reactionary. The developing cult assuming AI is flawless and everyone wants it is weird and isn't productive. The developing cult declaring that AI is worthless and has no use is understandable and also isn't productive.
Yes every company involved should be prosecuted for at least copyright violations. No we don't need nuclear/coal powered data centers. Yes capitalism is breaking down under the strain of concentration of resources and lack of regulation. Good lord is the economy of promising basically all of computer component manufacturing for multiple years to things that might not get built on loaned money a clusterfuck.
I can use my gaming computer GPU to help modify a personal fork to add support for the OIDC provider I use for other local services. I have to monitor what it's doing and provide framework and stop it from going on tangents. But I don't have to write a few hundred lines of tests and boilerplate or hunt down and adapt similar examples from stack overflow. That's pretty useful.
The Linux group's stance is that they're a mostly volunteer non-profit organization and the overhead to sniff out if something "might be" AI coded or assisted, above and beyond testing if the code works and is bug free, is literally impossible for them.
The full quote (which he does link) is about exactly the religious fervor that some corners of the internet are developing about anything without an instant AI ban. Linus has been using similar language since the 90s, including in Revolution OS, about the kernel. He's a blunt
SwedishFinnish ass who wants the tech to work better and without the artificial constraints that come from business models, the other aspects that open source enables are a pleasant side benefit, but the code is impartial.Funny. You can do whatever you want, from switching to BSD or even more niche open-source software to forking the whole thing yourself.
But you probably meant, what can one do without devoting a significant fraction of your life's time and energy to it. Because utility software is not truly a priority for most of us. In that case, I suggest coming to terms with your available capacity and focusing on your true priorities in life. Possibly by contributing to local political efforts, which is where most of us have the most leverage.
what if my "true priorities" lie precisely in maintaining quality and community, and not letting swaths of civilization come under seige of greedy corporate interests?
Then you should focus on cultivating the capacity to make effective contributions to those causes.
Nothing?