All of my SWE friends and coworkers agree this is a deeply weird time to be in the industry. LLM coding has dominated many (but not all) tech companies. Some of it is cool and exciting, but a lot of it is terrible and horrifying. I thought it could be a good time to do a vibe check.

Some possible questions to answer if you’d like:

• How has your work changed in the past year? How much of your daily work now involves LLM tools?
• Are you enjoying your work more, less, or the same? Has your relationship with work changed in general?
• Do you feel like you’re working more or less now? The same?
• Are you hopeful or fearful of the future (let’s say next 5 years) of SWE? Mixed?
• Have you found any useful tips or advice for thriving in the current landscape?
• Has your career trajectory changed? Do you plan on seeking out different types of roles?

Also it would be useful to give whatever background info you’re comfortable sharing to help ground things like: size of company, years of experience, type of role, location, etc.

Any and all discussion around the industry is welcome!

In all the hype and debate and excitement and frustration around AI, I think sometimes the fact that we've achieved something really cool gets lost. I want to talk about that a bit, which I'll try to do in a simple way, without too much math or computer science.

There's a surprising amount of intelligence encoded in applied language. To me that's the most remarkable insight in LLM technology. Except it's not really surprising at all. The whole point of language is to encode intelligence. Communication, at its core, is about sharing intelligence. LLMs agents are a way of turning the intelligence encoded in language, particularly written language, into useful work.

You might imagine that language is just the starting point when building LLMs. That it gets converted into computer stuff and then it's not language anymore. But in a way it's language all the way down. Distill language into math and then run gradient descent on it and the intelligence (for lack of a better term) bubbles out.

Forgive that hyperbole, but there is something a little bit magical about it, and the technology is entirely unmagical. The magic is in the language.

At the core the tech is pretty simple. First you go and get all of the language. Just completely idiotic amounts of applied language. Because there's a threshold: if you don't use enough language you don't get a useful result. The more language you use, assuming it's good quality, the more useful the result.

After you have all of the language, you convert it into tokens, which are essentially pieces of words represented as numbers. You also compute a compressed version of the relationships between the tokens. The frontier labs don't say how much this costs publicly anymore but it's safe to say hundreds of millions in compute are required for this process. Each time you do it.

Once you've determined the token relationships, you've pre trained an LLM. You have the initial weights. At this point your LLM can output credible prose pretty reliably when you run inference on the weights. Meaning that you can take a given set of language tokens (a prompt) and determine which token should come next, over and over again, until you have a response. The fluency of the prose can be uncanny, and there are latent tendencies towards useful output, but you can't really do much with it yet.

The next step, fine tuning, modifies the weights in order to nudge the inference towards useful things like instruction following, tool use and reasoning. Or a simulacrum of reasoning. Fine tuning is a massive, multi step, iterative process that turns your LLM into a useful tool that can do more than output believable prose.

Something that's really fascinating to me about fine tuning is that what you're training is still the language relationship weights, and what comes out during inference is still language. But now it's language that is more likely to result in useful output and behaviors. Language is the substrate that LLMs reason through. A model that's more likely to infer the words "but what if I look at it another way" is more likely to consider multiple possibilities. A model that reliably infers language around tool use in appropriate situations is an agent. The behavior is encoded in the language.

After tens of millions in fine tuning costs, assuming you got it right, you have the core of a LLM that can function as an AI agent. You can put it in a harness and ask it to do things in the language of your choice and it will actually be able to do them a shockingly high percentage of the time. It can even do things that weren't explicitly part of the training. That's something which has never happened before outside of science fiction. Until recently it was one of those computer sciencey things that was probably going to happen someday, after we were all dead.

It sucks that this tech revolution is being driven by the capital class, and that it's happening as a mad dash for market domination and golden IPOs. In a more perfect world technology derived from collective human intelligence would be a public good rather than a profit driver.

But nevertheless, the ability to imprint some part of human intelligence into an autonomous agent is a remarkable achievement. One that we can't yet see the full size of. And it's possible not because we typed instructions into an interface, but because we've spent centuries encoding our intelligence into language.

Hopefully the Tilderen aren't completely sick of the topic of AI writing.

To summarize my take: I think LLM agents are profoundly useful. And problematic. And I really really dislike AI writing when I encounter it outside of documentation. It's not only that it usually has a poor signal to noise ratio and a lot of annoying tropes, it also breaks a contract that has existed since writing was invented. And I dislike that it displaces human writing. Human writing is beautiful, we need more of it, not less.

By breaking a contract I mean that human writing implies an investment. The author invested time in creating it and has (sometimes) invested their reputation by publishing it. Whereas LLM prose requires minimal effort and thought. One great summary is:

"If you didn't take the time to write it, why should I take the time to read it?"

But it's not just about time or preference. We rely on signals of investment and competence to judge the value of information both personally and professionally. LLMs short circuit that completely. They incorporate signals of effort despite there being no effort involved. Signals that we could once rely on that someone has a deep understanding of a subject, or did extensive research, or is reasonably intelligent, or interesting, don't have any meaning when it's LLM prose.

Recently, while investigating Fable failure modes (while working in Claude Code), it outputted a summary that I think captures the situation really well: "My polish costs nothing".

Which is to say that LLM prose folds in signals of authority, competence, rigor and insight without any of those things needing to be present in the content. A fluent human is often worth listening to while a fluent LLM is just the default, the fluency carries no information. I think that's part of the reason for AI psychosis, I think it effectively uses, now outdated, expectations to hijack perception.

The reason that I sometimes take the time to investigate why an agent went wrong is partly to refine the scaffolding and partly to remind myself how dumb they are. Each generation is a little bit better, confidently right a little more than it's confidently wrong, so I make a point not to let myself fall into the trap of trusting their output too much.

I suppose I'm just posting to pass on the reminder.

I haven't been closely following the releases of new models and the research papers that sometimes go along with them. So I was wondering: is power consumption ever seriously talked about by OpenAI, Google, Anthropic, and others? Do we get some specific numbers of how much power their models actually consume to produce 100 tokens? The cost of training some of these models? Or we're not yet at that stage yet and nobody cares and at best we can just get really rough estimates based on "trust me bro" tweets by their respective CEO's?

Some time ago, I came across this post. GPT-OSS (2OB, 120B) are meager, yet energy efficient models and I was surprised that the power consumption is still larger than what I had estimated before.

For 120B to generate a 1000 tokens (which is really not a lot) it would take up around 83 Wh of electricity. For context my PC consumes ~100Wh when idling. So it's almost equivalent as leaving my PC on for an hour. Considering that proprietary models are TRILLIONS of parameters and probably not as energy efficient the true power consumption of these models is concerning. Of course, these big data centers do a lot of things to maximize the efficiency that this "test" fails to do. But even if you half the energy consumption it's still significant given their size and their pervasiveness in handling everyday, trivial tasks.

I haven't come across similar posts or studies for newer open source models, so if somebody has, please share.

Also, I don't seem to fully understand how does context size fit into all of this. The larger the context size the more power it would take to produce those 100 tokens?

I was musing about this recently with the recent models becoming more capable. The core of gen AI is the model, which is trained on a massive dataset. To date, gen AI has improved because the models have become larger, more efficient, the data they are trained on has become better and the software/harnesses around them has improved to help query them.

As I see it, surely the bottleneck will soon become the data they are trained on? If we imagine a scenario where a models could consume an infinite amount of training data, and there is no limit to the training time or quality. The sum of human skill/knowledge is the limiting factor. Gen AI should (in theory) never be able to out preform or push the boundary of the sum of humanity at time of training.

Or, counterpoint, is there enough randomness and speed to iterate that gen AI can actually step change and improve if training times/cost were less prohibitive? Most companies/models today will save good output and feed it back into the next iteration, but right now that's taking months. What if that took minutes?

What do you think?

Is gen AI going to take us to general intelligence?
Will gen AI get to a place where it's "intelligence" and reasoning is actually better than the sum of Humanity?

I have a friend that is lucky enough to have retired at 40. A year ago he was adamant he'd never work again, having been burnt out from his time at big tech. Back then he was also an absolute AI hater and wouldn't listen to anyone who claimed LLMs were useful for programming.

He finally tried LLMs when Claude Opus 4.6 released and immediately changed his mind in the face of the overwhelming evidence that LLMs can in fact program pretty well. And now with the release of Fable 5 he's giddily creating all sorts of things that would have taken far too long to make prior to AI-accelerated software development. He actually plans to try and found his own business now. He's a very smart guy, so I hope he can make something interesting that people want.

There are a lot of AI doomers and haters. In person I mostly see people doing the same thing they've always done, but now saving time on various tasks. But this is the first time I've seen someone go from grumpy and checked out to giddy and optimistic thanks to LLMs.

My daughter (11) is doing a week long Python class, which is not using LLMs.

It got me thinking about how I learned to program in the pre-internet days (laboriously, from books), and then what a marvel it was when you could just search for information, especially for troubleshooting. But for her, the first answer in the Google search is going to be the AI summary, and most of her search tools are going to be AI tools.

I wonder if it would be possible to make an LLM that has a didactic/socratic mode. So if you said, "help me write a program to do madlibs" maybe it would give you a skeleton of a function, then prompt you to come to with a plan, then critique that plan. Or if you said, "I'm getting this error", it wouldn't just fix it, it would explain what the error means and nudge you towards the answer.

Thinking in a larger sense, it could have a rubric of important concepts, even tiers of understanding. It could be using the interactions to track the user's understanding, which could let it then tune how it answers future questions, or even be used to customize assignments.

I recognize that this is potentially replacing a teacher with a machine, which wouldn't be my goal. Good teachers are more holistic in their teaching than a machine is ever likely to be. But for people who don't have access to good teachers, or need more directed support than is available from a teacher, or just want to self study, it seems like it could be a valuable addition.

Until they solve the obsequiousness problem, it would be vulnerable to prompt hacking, so really more of a tool for someone who recognizes the value of learning over just being given the answer.

What do folks think about using such a tool? What would you want it to do, or not do?

Aside: I forgot until I reached the end of this post, but this is also (somewhat) the plot of The Diamond Age, or A Young Lady's Illustrates Primer by Neal Stephenson.

I'm a web developer and find the playwright MCP to be genuinely useful. My LLM is able to navigate my site, measure the size of elements, see console errors, network requests, etc. This is the only MCP I've ever installed and haven't yet had any cause to use others. But I'm interested in hearing what other professionals are using.

I love writing, it's one of the most human things about humanity. It's communication, art and sharing all at once. It's been fundamental to culture and progress for 1000's of years.

LLMs are, in a way, really good at writing. They have the larger part of human creative output distilled into their weights. So it was inevitable that more and more people would start publishing articles and blog posts written (all or in part) by AI agents.

I don't like it but I accept it, there really isn't anything I can do about it. What I was hoping, though, is that high signal to noise ratio places on the internet (Tildes among them) would reject it and we could go on consuming 100% organic prose, at least for a while.

And for while that's exactly what happened. In techy places like Hacker News, AI posts were quickly flagged and downvoted into oblivion. At Tildes they mostly didn't show up at all, or if they did I missed them.

That seems to be ending though. Now I see agent written pieces on the front page of HN with 100's of comments. There's always a highly upvoted comment pointing out that the piece is slop, but you have to scroll to find it.

The reason I use HN as an example is that it's full of people with extensive experience using AI agents who are in a position to tell if something is slop. And it looks like the larger part of readers (or at least commenters) can't tell the difference anymore. If that's true at HN, it's going to be true everywhere.

It is getting harder to tell when something is slop, people are post editing, handwriting intros and getting better at prompting to remove obvious LLM tells. But if you have any practical experience with these tools, it's still pretty easy to tell. Somewhere during post training certain patterns end up getting heavily favored. Interestingly, many of them happen across all of the frontier models. Em-dashes are the most famous but there are so many more. Most are rhetorical tricks or formatting patterns rather than punctuation.

Reading LLM prose, many of the tropes don't stand out at first, instead they land as strong writing. But after you see them repeat enough times they start to become obvious. Even putting the tropes aside, the hallmark of a lot of LLM writing is that it's more rhetoric than substance. Low signal, lots of noise.

I don't have a solution, it's starting to look like many (maybe most) people aren't going to be able to tell when they're consuming something that required minimal thought by the "author" who prompted the AI. Which is sad because, up until now, we could assume that, when we read something, someone cared enough to put time and mental bandwidth into creating it. That's become increasingly less true.

I suppose this post is me feeling wistful for the internet we used to have, written exclusively by humans. I continue to hope that people will reject slop at places like Tildes, but in order for them to do that they have to be able to identify it. Maybe people will get better at that, there is definitely a point where you've consumed enough slop that you can smell it from a mile away. But of course the slop is going to keep getting harder to detect.

I don't want to go as far as to say that slop will take over the internet, I think (hope) that people will keep wanting to read organic, human, writing. And that as a result we'll come up with strategies and solutions to support that.

It's a weird time. Right now every LLM blog post and article that goes viral is signalling to the prompter, and anyone watching who can tell what's happening, that there is demand for slop. And of course with demand comes profit. I think we're at the beginning of a steep curve.

I probably triggered some PTSD right there.

Was just in a meeting at work, where we listed off everything that makes software development hard and slow. An excersize for the thread would be to replicate that list. It turned out that Claude helps with like 1/5th or less of it....especially in a collaborative environment.

So, the situation we're now encountering is that random business areas can vibe code out something, tell nobody, throw it in AWS, have it become a critical part of a business process that fails when they quit, and nobody even has access to look at what was made.

It gives me comfort that in about 5 years there will be a new surge in demand for programmers to reign in all the rogue applications that need shutdown because of the immense risk to continual operation of a company, from data leaks to broken payroll.

It'll be Y2K all over again.

For a long time programmers have had two types of program verification tools, static analysis (like a compiler's checks) and dynamic analysis (running a test suite). I find myself using LLMs to analyze newly written code more and more. Even when they spit out a lot of false positives, I still find them to be a massive help. My workflow is something like this:

1. Commit my changes
2. Ask Claude Opus "Find problems with my latest commit"
3. Look though its list and skip over false positives.
4. Fix the true positives.
5. git add -A && git commit --amend --no-edit
6. Clear Claude's context
7. Back to step 2.

I repeat this loop until all of the issues Claude raises are dismissable. I know there are a lot of startups building a SaaS for things like this (CodeRabbit is one I've seen before, I didn't like it too much) but I feel just doing the above procedure is plenty good enough and catches a lot of issues that could take more time to uncover if raised by manual testing.

It's also been productive to ask for any problems in an entire repo. It will of course never be able to perform a completely thorough review of even a modestly sized application, but highlighting any problem at all is still useful.

Someone recently mentioned to me that they use vision-capable LLMs to perform "aesthetic tests" in their CI. The model takes screenshots of each page before and after a code change and throws an error if it thinks something is wrong.

I've found myself replying to different people about the early 2025 METR study kind of often. So I thought I'd try posting a top level thread, consider it an unsolicitied public service announcement.

You might be familiar with the study because it has been showing up alongside discussions about AI and coding for about a year. It found that LLMs actually decreased developer productivity and so people love to use it to suggest that the whole AI coding thing is really a big lie and the people who think it makes them more productive are hallucinating.

Here's the thing about that study... No one seems to have even glanced at it!

First, it's from early 2025, they used Claude Sonnet 3.5 or 3.7. Those models are no way comparable to current gen coding agents. The commonly cited inflection point didn't happen until later in 2025 with, depending on who you ask, Sonnet 4.5 or Opus 4.5

The study was comprised of 16 people! If those 16 were even vaguely representative of the developer population at the time most of them wouldn't have had significant experience with LLMs for coding.

These are not tools that just work out of the box, especially back then. It takes time and experimentation, or instruction, to use them well.

It was cool that they did the study, trying to understand LLMs was a good idea. But it's not what anyone would consider a representative, or even well thought out, study. 16 people!

But wait! They did a follow up study later in 2025.

This time with about 60 people and newer models and tools. In that study they found the opposite effect, AI tools sped developers up (which is a shock to no one who has used these tools long enough to get a feel for them). They also mentioned:

However the true speedup could be much higher among the developers and tasks which are selected out of the experiment.

In addition they had some, kind of entertaining, issues:

Due to the severity of these selection effects, we are working on changes to the design of our study.

Back to the drawing board, because:

Recruitment and retention of developers has become more difficult. An increased share of developers say they would not want to do 50% of their work without AI, even though our study pays them $50/hour to work on tasks of their own choosing. Our study is thus systematically missing developers who have the most optimistic expectations about AI’s value.

And...

Developers have become more selective in which tasks they submit. When surveyed, 30% to 50% of developers told us that they were choosing not to submit some tasks because they did not want to do them without AI. This implies we are systematically missing tasks which have high expected uplift from AI.

And so...

Together, these effects make it likely that our estimate reported above is a lower-bound on the true productivity effects of AI on these developers.

[...]

Some developers were less likely to complete tasks that they submitted if they were assigned to the AI-disallowed condition. One developer did not complete any of the tasks that were assigned to the AI-disallowed condition.

[...]

Altogether, these issues make it challenging to interpret our central estimate, and we believe it is likely a bad proxy for the real productivity impact of AI tools on these developers.

So to summarize, the new study showed a productivity increase and they estimate it's larger than the ~20% increase the study found. Cheers to them for being honest about the issues they encountered. For my part I know for sure that the increase is significantly more than 20%. The caveat, though, is that is only true after you've had some experience with the tools.

The truth is that we don't need a study for this, any experienced engineer can readily see it for themselves and you can find them talking about it pretty much everywhere. It would be interesting, though, to see a well designed study that attempted to quantify how big the average productivity increase actually is.

For that the participants using AI would need to be experienced with it and allowed to use their existing setups.

I want to add that this is not an attempt to evangelize for AI. I find the tools useful but I'm not selling anything. I'm interested in them and I stay up to date on the conversations surrounding them and the underlying technology. I use them frequently both for my own projects and to help less technical people improve their business productivity.

Whether AI agents are a good thing or not, from a larger perspective, is a very different, and complicated, conversation. The important thing is that utility and impact are two different conversations. There isn't a debate anymore about utility.

I know this probably won't stop people from continuing to derail conversations with the claim that developers are wrong about utility, but I had to try. It's just hard to let it pass by when someone claims the sky is green.

I understand that AI makes people angry and I think they have good reason to be angry. There are a lot of aspects of the AI revolution that I'm not thrilled about. The hype foremost, the FOMO as part of the hype, the potential for increased wealth consolidation really sucks, though I lay that at the feet of systems that existed before LLMs came along.

It's messy, but let's consider giving the benefit of the doubt to professionals who say a tool works instead of claiming they're wrong. Let them enjoy it. We can still be angry at AI at the same time.

I've been thinking a lot about LLM assisted development, and in particular why I keep dropping the available tools after a few attempts at using them.

I realized recently that it's taking away the part of software development I enjoy: the creative problem solving that comes with writing code. What's left is code review tasks, testing, security checks, etc. Important tasks, but they all primarily involve heavy concentration, and much less creativity.

Why aren't agents focused on handling the mundane tasks instead? Tell me if I've just introduced a security vulnerability or a runtime bug. Generate realistic test data and give me info on what the likely output would be. Tell me that the algorithm I just wrote is O(n^2).

Those tasks are so much more applicable to matching against existing data, something LLMs should be extremely good at, rather than trying to get them to write something novel, which so far they've been mostly bad at, at least in my experience.