My polish costs nothing
Hopefully the Tilderen aren't completely sick of the topic of AI writing.
To summarize my take: I think LLM agents are profoundly useful. And problematic. And I really really dislike AI writing when I encounter it outside of documentation. It's not only that it usually has a poor signal to noise ratio and a lot of annoying tropes, it also breaks a contract that has existed since writing was invented. And I dislike that it displaces human writing. Human writing is beautiful, we need more of it, not less.
By breaking a contract I mean that human writing implies an investment. The author invested time in creating it and has (sometimes) invested their reputation by publishing it. Whereas LLM prose requires minimal effort and thought. One great summary is:
"If you didn't take the time to write it, why should I take the time to read it?"
But it's not just about time or preference. We rely on signals of investment and competence to judge the value of information both personally and professionally. LLMs short circuit that completely. They incorporate signals of effort despite there being no effort involved. Signals that we could once rely on that someone has a deep understanding of a subject, or did extensive research, or is reasonably intelligent, or interesting, don't have any meaning when it's LLM prose.
Recently, while investigating Fable failure modes (while working in Claude Code), it outputted a summary that I think captures the situation really well: "My polish costs nothing".
Which is to say that LLM prose folds in signals of authority, competence, rigor and insight without any of those things needing to be present in the content. A fluent human is often worth listening to while a fluent LLM is just the default, the fluency carries no information. I think that's part of the reason for AI psychosis, I think it effectively uses, now outdated, expectations to hijack perception.
The reason that I sometimes take the time to investigate why an agent went wrong is partly to refine the scaffolding and partly to remind myself how dumb they are. Each generation is a little bit better, confidently right a little more than it's confidently wrong, so I make a point not to let myself fall into the trap of trusting their output too much.
I suppose I'm just posting to pass on the reminder.
My team is quietly sliding into doing more and more programming work through Claude Code, while still mandating human reviews. I'm really struggling with this development.
When I review code written by a human, I do it via a sort of "technical empathy." If I had been given this task, how would I have approached it? What knowledge would I have needed? What mistakes would I have tried to avoid? I attempt to put myself in the other developer's shoes and understand their reasoning. Which is possible because we are both humans. When reviewing LLM-generated code, this method falls apart. There's no human reasoning to relate to. How the code was put together follows a totally different, inscrutable logic. Potential mistakes happened for other reasons and take different forms.
Even more infuriating, if I happen to find a mistake, there is no discussion as to where the mistake came from, what misunderstanding of the original problem or lack of technical knowledge led to it. There is an answer akin to "You're right. I made a mistake here -- let me rewrite that properly." There is no human exchange, understanding of each other's point of view, collaboration towards an acceptable solution, mutual growth. Just a cold, empty interaction with a black box.
There's something utterly disgusting in becoming a sort of biological appendage to systems that become more prevalent and more opaque with each passing day, and I have a hard time articulating exactly why.
Cory Doctorow has a term for this: reverse centaur. He's alarmist sometimes but he coins great terms.
Thanks, I didn't know about that concept. Gives me more fuel for the upcoming discussion I'm gonna have to have with my colleagues.
Something to think about for that conversation: IMO the conversation we should be having isn't "human coding VS agentic coding". Instead we should be talking about how to incorporate agents in ways that make sense and don't suck for humans. There's no denying that, used well, agents can speed things up and soften pain points without lowering quality (too much) or driving people insane. The problem is that there's not yet conventional wisdom about what that should look like. We're still figuring out what coding agents are best used for, and what they aren't good at. And then re-figuring it out every model generation or two.
I think the answer for the time being is to be extremely intentional when incorporating agents. And expect a lot of experience and feedback driven iteration of the process. Scaffolding makes a huge difference and I've seen a lot of attempts that miss this entirely. Formalization is another big one but I won't digress. Most importantly: until things have settled and best practices start to emerge, every org should take the time to carefully create processes that work for them. Preferrably driven largely by senior engineers rather than management.
It sounds like maybe the process isn't particularly intentional at your company, so humans are left trying to adapt, rather than adapting agents to what works for humans. It doesn't have to be that way. Though of course I know nothing about your org, so I might have that wrong. And maybe you're at a scale where intentionally building processes is really hard. The principle is sound though.
Yes I agree, finding the proper formulation and arguments is going to be the tricky part. My fundamental belief is that the whole hype is insufferable, we've collectively and almost overnight decided to stop thinking altogether, and that it's all going to come back to bite us in the ass in catastrophic and unpredictable ways 10-15 years from now. But of course that's not very constructive or actionable.
An example of this problem I think is the devaluation of covering letters or recommendation letters from teachers. The value idea to be that if someone took time to write one of those, it was because they meant it. If they write themselves, they are less valuable.
Yeah I saw someone I trust and respect just entirely use AI to answer application questions for a recommendation/reference letter without any tailoring to the individual they were writing about. It felt a bit fraudulent (what they were writing was not true as the individual was not very good) and I lost a lot of respect for both them and the validity of any reference letter I get. Sad times. I feel that soft fraud or lying/exaggerating too far has become normalised and really has weakened a lot of good things about modern institutions.
I think it's common for people who are not very good to still request and receive recommendation letters. The way I've seen this handled before is the language in these letters tends to be terse and unpolished. Theres a subtext there that says I'm saying these nice things about this candidate but I don't really mean any of them. Maybe the default AI voice is that new subtext.
Perhaps, but I think it’s harder to tell the difference between AI voice and a positive voice for a good candidate.
The examples you gave are things that I have a hard relationship with. They are both things that I think AI has allowed me to do better. They are also both things that I assign no value other than the vestigial relationship we have with them and job searching. Every person I knew who wrote recommendation letters 10+ years ago just had a form letter they filled in and most job applicants had the same with a cover letter. Perhaps at the higher levels of a career these things become more important but even the HR reps I know didn't really value a cover letter. We're probably entering an end-phase of the war of job applications where HR has been using AI to sort applicants and applicants are using AI to try and get selected. Cynically, it is highlighting that the system was always kind of dumb.
I feel the same about references for most jobs. You should be able to find three people to lie for you. But that starts getting more off-topic.
It probably depends a lot on the applicant and the job they're applying for.
I've always found cover letters really great. They make me feel like a human with a personality, rather than like a mindless application-filler or resume-submitter.
My resume is basically just a list of stats, but my cover letter is where I can just outright say what I want them to know about me. It's a chance to demonstrate my personality, explain things in my resume (like job gaps and career changes), and generally just communicate what sort of stuff I'm good at (and not so good at) and what sort of stuff I'm hoping to do in my next job.
I really struggle with formality, adhering to vague-yet-somehow-strict social 'rules' (like how to dress for interviews or how to word resumes), and selling myself, so until I get an interview, my cover letter is pretty much the only place I can loosen up and show off what I offer instead, like my creativity, flexibility, and direct communication style.
But I've mostly only applied to work at small companies that haven't publicly listed the roles they're hiring for, so they aren't slogging through hundreds of applicants. Writing a cover letter is a lot more enticing when you know people are actually going to read it — and, more importantly, that those people are your potential future managers and coworkers (and therefore pretty keen to get a sense of your personality).
I strongly agree that it depends a lot on the job one is applying for. On the applicant, I softly agree.
I think I have the opposite struggle when it comes to cover letters (and the application process as a whole). They feel the opposite of humanizing to me. I am stuck selling myself with no external feedback, trying to adhere to social constructs I barely know and certainly don't understand. I too really struggle with formality and much of the unspoken rules of office culture and, to me, the application process is the first hurdle of that labyrinth. Especially when applying to larger institutions where the hiring managers and coworkers may never see the cover letter.
When applying for smaller companies, I start to see the use of a cover letter much more. I also generally already have a relationship with individuals at the company and... well... my personality can tend to precede me, so they generally already know some of my intangibles.
Yeah, that makes sense. I think particularly when the cover letter is getting filtered through some kind of recruitment apparatus first, adhering to some kind of standard formulation for it might be necessary? (Not sure.)
For the jobs I apply for, I write it in a more casual tone, like how I'd write an email. I might even inject a bit of humor. I figure if that gets me filtered out, I probably don't want to work for whoever filtered it out; they're way too rigid for me!
...did you use em-dashes in your written correspondence prior to LLMs establishing a 'standard tone' for sentence structure?..i worry that i'm picking up bad habits from reading it so often of late...
Plot twist: RoyalHenOil fed this thread into ChatGPT to come up with a comment that would both justify LLM usage and elicit sympathy from us, and voila.
This shadow of distrust that we must now take with us into practically every encounter with text is the exact contract-breaking that post_below is talking about.
...'breaking the social contract' so succinctly articulates the offense i take from LLM slop; i'll probably incorporate that phrase into my rant-lexicon...
I've always been a big em dash user (and semi-colon user, though I don't think I've ever seen that one in LLM writing?). And, um, an over-user of parenthetical statements. I use way too much punctuation.
Believe it or not, I used to be much worse about it when I was a teenager (waaay before LLMs). Back then, I even used to abuse ellipses...
I have admittedly tried to cut back on my em dash usage since LLMs became common, but they're really useful and instinctive for me. At least I use them a slightly nonstandard way (straddled by spaces), since I was taught on AP style in school, rather than Chicago style or whatever. All of the LLM writing I've seen lacks the spaces (I assume because they've been fed on a lot of books and formal writing?).
As it happens, if you tell them not to use em-dashes, they develop a newfound love of semicolons!
(can't speak for RoyalHenOil, but I certainly did. The default style for a few of the bigger ones tends to use 'em as a means of separating off contrasting concepts, in order to highlight a point -- not, for example, interjecting in the middle of one -- so they often read weirdly to me in a very distinct, robotic way. They're like a student who has learned a stylistic crutch -- not a thorough deployment of that punctuation, understanding all of its subtleties)
(also I use a double hyphen cuz I'm old)
I've noticed that stuff written by Claude (not super sure about ChatGPT) uses em dashes in places where I would never use one; I would use a semicolon there instead (unless I already have another semicolon in the sentence, and I'm just really adverse to breaking it into two sentences for some reason).
There are plenty of people who use em dashes that way, though. Or double-hyphens. (I use true em dashes when I'm writing on my phone, like right now, or on my PC where I have an AutoHotKey script to make typing them easier; I edit a lot of technical documentation for work, so I need easier access to all my punctuation than Windows offers standard. Otherwise, I also use a double-hyphen, but I've noticed a lot of software automatically converts that to an em dash anyway.)
...every time former staff have asked me for a recommendation letter, i've taken the time to thoughtfully articulate my experiences working with them and carefully compose a survey of their strengths tailored to the specific application for which they've requested it; this isn't a trivial endeavor and can consume a half-day of my time to do the exercise justice...
...i like to think that it works well, as they've always gotten the position for which i've recommended them, but in the face of AI slop submissions and evaluations i'm inclined to no longer bother...
You're one of the good ones! My wife is asked to write letters of recommendation a few times a year and she also thoughtfully writes a letter tailored to the person and position, and even researches the company a bit to make sure she's really tailoring it well. Like you, it usually takes up a half-day of her time. She's a saint.
But I also think in the era of AI, it's a tough exercise to continue. In a lot of ways I feel like the AI boom is an extension of a "fine is good enough" mentality (rather than the producer of said mindset) and both sides, HR and Applicant, are living that ethos. It sucks that AI can write a cover letter that is "good enough" in seconds because the HR's AI is just pulling buzzwords out of the cover letter. Obviously, I think this is experience is different across career levels but being on the more entry-level side is exhausting.
I don't know how true this is, but I'm under the impression that it's already somewhat gamed, because professors will ask students to write a draft letter for them.
If so, it doesn't necessarily show that the professor remembers that much about them.
I really dislike AI writing of documentation.
Instead of one short page with clear explanations I got 3-7 (sometime 20) pages of dilutet text with repeated statements thats honestly hard to digest.
But of course all people that do not need to work with resulting documents are happy. You got proper big beautiful documents in just few minutes.
I disagree with this line here. I'll give some counterexamples:
(1) I work with quite a few people who have English as a second language. AI has been really helpful for them in composing and managing emails - it reduces the cognitive load of having to write in English and they can express themselves easier and faster. The combination of their thought process and the AI composition lets them be more fluent in email then they could be otherwise.
(2) I have about 20 years of helping small nonprofits with finance and accounting issues. I needed to draft a policy for one I'm working with at the moment. I used Claude to draft the policy - and it did a pretty good job - it essentially hit the best practices. I needed to customize and redraft a few items to meet the organizations specific context, but overall, it was 85% complete. It saved me quite a bit of time. It was also better produced then about 75% of the nonprofit financial policies I've read in my lifetime. The raw AI output wasn't as good as if I had hand drafted it, but most nonprofits don't have someone like me in their corner. The fluency of the AI's policy is better then what is available to most nonprofits.
Ehhh... I don't think I agree. If the premise was that what consulting firms sell is information and experience, then yeah, I'd 1000% agree with you. They're finished. Almost all of what they do is draw upon a huge internal library of pretty standard info, customize it a bit, and deliver it as a product to a customer. The bigger customers get a bit of actual analysis and teams of people to do more in depth customization along with more white glove treatment, but it's mostly the same thing.
That's not actually the value proposition of management consulting firms though. If it was, they probably would have already been replaced by Google.
What they're really selling is authority. Its a arrow in an executive's quiver. If an executive wants to do something big, they need to convince someone of it. If it's a director, they need to convince the COO and CFO. If it's another C level, they need to convince the CEO. If it's the CEO, they need to convince the board.
They can't do that effectively without hard data and numbers. They go to a consulting firm because they almost always already have a conclusion in mind. The firms job is basically to justify that conclusion. If they can't justify it, the executive either gets a second opinion of someone who does, just does it live and conveniently leaves out the consultants report, or if they're lawful good, just drops the idea and comes up with another idea that doesn't suck.
The value that the firm brings isn't the actual info though, it's being able to go to the board saying "McKinsey said this was a good idea". That's why the big three are the big three. Everyone knows their name, and most people trust them. That name is their entire value proposition, and is the reason why they get six figures to put together some boilerplate PDFs.
Even if an AI tool gave you the exact same content, it would be worthless, because a CEO going to a board with an idea and defending it with "chatGPT said this was a good idea" would be laughed out of the room.
I don't see that changing any time soon (unfortunately)
I work in consultancy, though not as a prototypical consultant more of a secondment type of deal just doing your regular dev stuff as part of a client team. But, other departments of my company certainly fit the bill of what people think of. Anyway, reading OPs post I couldn't help but think that part of what they encounter is something I have been dealing with for years now from your stereotypical consultants.
Specifically this bit
Did actually prompt a jaded chuckle on my part as many consultants and consultancy firm have been doing exactly this for decades already. And historically people have been really bad at picking it up. Frankly speaking, depending on the firm (or department in the company I work for) I'd be more inclined to trust an LLM over many subjects as they, in theory, are at least building on a wide array of training data.
I am mostly seeing a pivot towards AI (as everywhere) with the angle being that they are of course the best bet in properly implementing AI solutions with all the safeguards, etc, etc. But yeah, I have heard rumblings that certain areas of these companies are struggling more already.
Yes - I broadly agree. A lot of the strategy work was already under pressure since so many of the strategy consulting tools and knowledge are well known. AI probably accelerates that except for the truly novel problems.
IBM's latest returns showed that the business process outsourcing work is really under pressure right now. AI agents are going to eat that portion of their business and automate what was probably automatable.
The ERP systems are probably quite busy with AI projects and attaching AI capabilities and use cases to the core systems.
Even though in modern tools, translation and text generation follow the exact same pipeline, to me, they're still fundementally different tasks that, to me, even LLMs handle fundementally differently.
For text generation, the goal is to replicate a convincing approximation of what a human being would write, given the prompt. For translation, the goal is to, as accurately as possible, move the exact ideas given from one language to another.
One of them is basically an exercise in trying to fool humans into thinking the writer is also human, the other one is an exercise in trying to fool the speakers of one language that a speaker of a different language speaks the same language as them.
The first one is much more blatant of a lie than the second, and requires way more outright fabricated bullshit. The second is being guided by the exact words and meaning behind them being given to them.
I don't know if LLMs go through different training pipelines between these tasks, but when I read AI translated text versus AI generated text, the former feels far more natural and human, and doesnt have that innate immediate offensive feeling that the latter does.
Most of the workflows that I've seen are less that they write in their native tongue and then translate, it's more that they have five bullet points and ask the AI to create paragraphs, or they write the full email and then ask the AI to clean up their grammar.
I guess I find it odd that anyone gets offended about ai generated text. I've read so much human generated crap that I don't get particularly excited about machine generated. Maybe my set point is higher than most people's or my one emotion is already permanently bruised.
If all they have to say can be summed up in five bullet points, I would much rather read an email with five bullet points than five paragraphs of nothing. There can be a lot of information lost going from bullet points -> AI paragraphs -> my own interpretation of those 5 points. If you just present the five points up front, there's a lot less going on, and less room for me to misinterpret your (literal) points.
Every time something is translated or rewritten (whether it be by a human or a tool), some information is lost or altered, even without accounting for the possibility that mistakes . Sometimes, it's acceptable or even good (e.g. changing "In my personal opinion, I think that it could be said that blah" to "I think blah"), and sometimes it's a very bad change. It's like playing a game of telephone, or using Google translate to go between languages a few times before you end up with unintelligible garbage, except with AI-assisted tools, it still seems coherent, so it's harder to tell that something's wrong.
For example, I ran the sentence "This sentence could not possibly be unfalse" through Hypertranslate, and the result is the complete opposite "This sentence cannot be false." Now, you may say that "unfalse" isn't a word, and it got translated to "false" along the way. And that's probably correct. But if a non-native speaker is trying to say something, and uses a word that isn't quite right, most English speakers can recognize what they mean, even if the word is wrong. Computers don't always have that nuance.
TL;DR:
Using five bullet points might be your (and honestly my) communication preference, but it's certainly not a universal norm. I've worked for bosses that prefer bullets and others that prefer paragraphs. Some corporate cultures - Amazon for instance - are notoriously text heavy. Others work via face to face discussion. Most people don't get to dictate those preferences all the time and have to adapt to the people around them.
Fair enough. I probably wouldn't work well in an environment where a boss wanted to dictate the style of internal informational emails, but I can fully accept that those bosses exist. Text-heavy emails just feels like another bullshit metric that some high-level exec decided was a good proxy for productivity, even though it's the opposite. People waste time writing lots of unnecessary words, and then (presumably) people waste time reading those words.
In places that do everything face to face, it's always good to send a follow up email summarizing the key points so that (1) there's a paper trail, and (2) if something was misinterpreted, there's an opportunity to correct it.
Likely there are different RF segments for these but the important thing is the prompt/context. "Translate this" results in inference that's going to try harder to stay close to the source material. "Write about this" is asking for fabrication, which is a free pass for all the tropes introduced in fine tuning to come out.
I think its similar to how many AI slop software we have. I already see AI writing everywhere, it's easy to recognize once you catch on the few phrases it often repeats.
I'm less worried about what I work with but more worried with society as a whole. A lot of non dev folks already have a hard time recognizing what is an AI slop website and what is a AI-assisted website. I'm honestly not sure what the solution is except for educating people and trying to improve the aggregate quality bar for both writing and websites.
A large percentage of the population will likely just not be able to tell the difference, and they'll be the source of a lot of false positives and false negatives.
Before AI-generated imagery, people were constantly wrongly accusing photographers of Photoshopping their photos (e.g., due to shadows appearing to go in different directions, even though that's just how perspective works, or due to the existence of JPG artifacts) while missing very blatant Photoshop tells (e.g., inconsistent resolution in different parts of the image).
Now that image-based genAI is a thing, suddenly nobody's accusing each other of Photoshopping images, even though Photoshop is still very much being used to doctor photos.
Exactly mate. Gotta try to instill principles in epople and call out AI slop, its kinda the only safe for this. Of course, its not like a few of us can shift the entire culture but I'd like to believe that at least we're doing our part in creating a dent in the universe. :)
It wouldn't be the first time, or even the 50th, that people with non-mainstream tech insight influenced culture. A pretty high percentage of life has been happening online for most of two decades.
Not to say that I necessarily like the odds. Slop is going to have an impact one way or another.
I have mixed feelings about this because I prefer polished prose, but writing well is also a sign of privilege. Either you actually write that well or you had help. Now getting help is cheaper.
We lose a signal, but maybe it's more democratic if everyone has access to an inexpensive ghostwriter?
/offtopic
Yes, it's catching on!
It's got an old school dark children's story vibe to it
I personally prefer ...
But, of course, the more, the merrier
I was thinking Tilderiñ@s to make it gender-neutral