Hopefully the Tilderen aren't completely sick of the topic of AI writing.

To summarize my take: I think LLM agents are profoundly useful. And problematic. And I really really dislike AI writing when I encounter it outside of documentation. It's not only that it usually has a poor signal to noise ratio and a lot of annoying tropes, it also breaks a contract that has existed since writing was invented. And I dislike that it displaces human writing. Human writing is beautiful, we need more of it, not less.

By breaking a contract I mean that human writing implies an investment. The author invested time in creating it and has (sometimes) invested their reputation by publishing it. Whereas LLM prose requires minimal effort and thought. One great summary is:

"If you didn't take the time to write it, why should I take the time to read it?"

But it's not just about time or preference. We rely on signals of investment and competence to judge the value of information both personally and professionally. LLMs short circuit that completely. They incorporate signals of effort despite there being no effort involved. Signals that we could once rely on that someone has a deep understanding of a subject, or did extensive research, or is reasonably intelligent, or interesting, don't have any meaning when it's LLM prose.

Recently, while investigating Fable failure modes (while working in Claude Code), it outputted a summary that I think captures the situation really well: "My polish costs nothing".

Which is to say that LLM prose folds in signals of authority, competence, rigor and insight without any of those things needing to be present in the content. A fluent human is often worth listening to while a fluent LLM is just the default, the fluency carries no information. I think that's part of the reason for AI psychosis, I think it effectively uses, now outdated, expectations to hijack perception.

The reason that I sometimes take the time to investigate why an agent went wrong is partly to refine the scaffolding and partly to remind myself how dumb they are. Each generation is a little bit better, confidently right a little more than it's confidently wrong, so I make a point not to let myself fall into the trap of trusting their output too much.

I suppose I'm just posting to pass on the reminder.