Recent overuse by language models has led many to declare it bad writing. I'm not so sure. Nobody called JFK a lazy writer when he said, "ask not what your country can do for you – ask what you can do for your country." Negative parallelism is a rhetorical device, and any rhetorical device is only as lazy or inspired as what it contains.
[...]
Now, we have AI detectors that claim to protect you from the witch hunt by looking for these patterns. You take your own writing and you run it through Grammarly, which will analyze word patterns that AI detectors might flag. Then it offers ideas for how to change them, which a) gives Grammarly the power to write for you and b) makes your writing lose any sense of rhythm or intent.
[...]
Defining reasoning the way it has been used in LLMs assumes that the point of asking a question is to get an answer, that answers can be verified, and that nothing is lost in immediate closure. This has real effects on writing, and the openness to doubt is something we lose in the rapid prototyping of thought that occurs with a language model. Ambiguity, doubt, and uncertainty matter more to some ways of thinking than any immediate answer. The inner life grows in the spaces between the industrial complexes that harness every remnant of our externalized thought.
Nonetheless, the language we use in these states is the same. When AI detectors flag text as AI-generated, is it because it follows a certain structural pattern of that reasoning? Pangram and reasoning models both detect structural patterns based on how humans reason when writing. Pangram's model is trained on pre-2021 data; it then inserts AI-generated versions of the same text into its training.
So, if we publicly shame people whose text looks like it might have been written by a machine – because it mimics the language used for human reasoning – and people stop writing in ways that they internalize as "AI writing" out of fear of false detection, it sends a signal that your language for reasoning must be policed, or you too could be held up to public scrutiny.
In the end, shaming people for writing that gets flagged as AI can lead people to sidestep structures the model has learned from us: structures that are effective tools for argumentation. We take the tools of critical thinking out of the kit at the time we most need them.
[...]
I'm not convinced by the old "if you haven't done anything wrong, you don't have anything to worry about" line. I've seen 99.8% cited as a measure of accuracy in automated surveillance systems since 2018. As Arvind Narayanan has noted, that is on a per-paper basis, which compounds every time we use it. So up to 10% of college students could be falsely accused. If we collectively run every bit of text through an AI model to check whether it is AI-generated, we will generate false positives on an even larger scale.
[...]
We create a culture of self-censorship and AI-detector-pressured rewriting and paraphrasing as people strive to avoid these witch hunts. That is the opposite of protecting human expression. We should resist normalizing a trust in any machine's ability to determine matters of guilt. If using AI to write is, at its worst, an industrialization of the mind, then AI detection, at its worst, becomes a surveillance system for thought.
This is a solid essay, the author frames post training/fine tuning accurately, which is somewhat rare at the moment. Generally I agree with all the parts, but I don't agree about what they add up...
This is a solid essay, the author frames post training/fine tuning accurately, which is somewhat rare at the moment. Generally I agree with all the parts, but I don't agree about what they add up to.
It's true that pangram and grammerly are imperfect, the latter especially. It doesn't make sense to trust their output. Pangram is good enough to be considered a signal, but it's not good enough to be considered definitive proof.
It's likely true that post training favors negative parallelism because that language structure is an artifact of training models to "reason". That's the most interesting part of the essay IMO. The models achieve their ability to mimic reasoning through language, so innocuous phrases like "but wait" and "I'm reconsidering" have functional value to LLMs far in excess of their value in language from a human perspective. For an LLM they're like the equivalent of cognitive processes.
Negative parallelism: "It's not X, it's Y" is potentially a representation of "most obvious inference" but instead "somewhat less obvious but possibly more correct inference". And it's true that similar reasoning is useful for people.
But I don't think this follows:
In the end, shaming people for writing that gets flagged as AI can lead people to sidestep structures the model has learned from us: structures that are effective tools for argumentation. We take the tools of critical thinking out of the kit at the time we most need them.
I think this encapsulates the logical leap of faith you need to make to get from the premise to the conclusion that we shouldn't shame people for using AI to write. I'd be more likely to agree if AI writing was closer to human writing, but if that were the case it would be almost impossible to detect and there'd be nothing to talk about.
The models did learn these things from our language, so it's fair to say "from us", but then after that they went through vast amounts of fine tuning that morph what they learned from us into something substantively different. In part because, while our reasoning is greatly influenced by language, it doesn't happen solely through language. We have dedicated circuits for it, so to speak.
The idea that avoiding LLMisms will cause people to lose important reasoning tools is silly IMO. The current generation of LLMisms doesn't cut into the toolbox in any meaningful way. I think that's pretty seriously mischaracterizing human cognition and exaggerating the scope of LLMisms.
Post training causes LLMs to grossly overuse a limited list of rhetorical tricks in obnoxious ways. If that causes people to avoid being similarly obnoxious I think that's a good thing!
I hope people continue to call out LLM writing when authors attempt to pass it off as their own work. That's how a culture has the conversation about what we're going to accept and what we're going to reject. I think that's an important conversation where LLM writing is concerned.
That said, witch hunts are no good and it's not always obvious whether or not something is LLM written. Em-dashes are the most famous example of this. For a while now they've been almost guaranteed to draw AI accusations on social media even though, used judiciously, they're perfectly reasonable punctuation that humans have been using forever. Just not anywhere near as much as LLMs use them. Probably in part because they don't show up overtly in most keyboard layouts.
I agree with the spirit of the essay in the sense that we should be mindful about how we approach the issue, rather than reflexively calling things AI or trusting tools like pangram. However, the larger part of LLM generated writing is obvious for all sorts of reasons beyond em-dashes and negative parallelism, not so much because of specific tells, but because —negative parallelism!— of the volume and consistency in a given piece.
I think we'll get better over time at telling the difference between human and LLM writing as we consume more slop because the patterns will become intuitive. We're all carrying around an exhaustive list of language rules and structure we couldn't fully articulate and often don't even know we're aware of. Learning to identify (current) LLM writing isn't a difficult task for our language centers to manage.
If the LLM’s remain the same, their tells will become more and more recognizable. But I think this is more likely to be a phase? I don’t think Claude says “you’re absolutely right” anymore?
If the LLM’s remain the same, their tells will become more and more recognizable. But I think this is more likely to be a phase? I don’t think Claude says “you’re absolutely right” anymore?
From the article:
[...]
[...]
[...]
[...]
This is a solid essay, the author frames post training/fine tuning accurately, which is somewhat rare at the moment. Generally I agree with all the parts, but I don't agree about what they add up to.
It's true that pangram and grammerly are imperfect, the latter especially. It doesn't make sense to trust their output. Pangram is good enough to be considered a signal, but it's not good enough to be considered definitive proof.
It's likely true that post training favors negative parallelism because that language structure is an artifact of training models to "reason". That's the most interesting part of the essay IMO. The models achieve their ability to mimic reasoning through language, so innocuous phrases like "but wait" and "I'm reconsidering" have functional value to LLMs far in excess of their value in language from a human perspective. For an LLM they're like the equivalent of cognitive processes.
Negative parallelism: "It's not X, it's Y" is potentially a representation of "most obvious inference" but instead "somewhat less obvious but possibly more correct inference". And it's true that similar reasoning is useful for people.
But I don't think this follows:
I think this encapsulates the logical leap of faith you need to make to get from the premise to the conclusion that we shouldn't shame people for using AI to write. I'd be more likely to agree if AI writing was closer to human writing, but if that were the case it would be almost impossible to detect and there'd be nothing to talk about.
The models did learn these things from our language, so it's fair to say "from us", but then after that they went through vast amounts of fine tuning that morph what they learned from us into something substantively different. In part because, while our reasoning is greatly influenced by language, it doesn't happen solely through language. We have dedicated circuits for it, so to speak.
The idea that avoiding LLMisms will cause people to lose important reasoning tools is silly IMO. The current generation of LLMisms doesn't cut into the toolbox in any meaningful way. I think that's pretty seriously mischaracterizing human cognition and exaggerating the scope of LLMisms.
Post training causes LLMs to grossly overuse a limited list of rhetorical tricks in obnoxious ways. If that causes people to avoid being similarly obnoxious I think that's a good thing!
I hope people continue to call out LLM writing when authors attempt to pass it off as their own work. That's how a culture has the conversation about what we're going to accept and what we're going to reject. I think that's an important conversation where LLM writing is concerned.
That said, witch hunts are no good and it's not always obvious whether or not something is LLM written. Em-dashes are the most famous example of this. For a while now they've been almost guaranteed to draw AI accusations on social media even though, used judiciously, they're perfectly reasonable punctuation that humans have been using forever. Just not anywhere near as much as LLMs use them. Probably in part because they don't show up overtly in most keyboard layouts.
I agree with the spirit of the essay in the sense that we should be mindful about how we approach the issue, rather than reflexively calling things AI or trusting tools like pangram. However, the larger part of LLM generated writing is obvious for all sorts of reasons beyond em-dashes and negative parallelism, not so much because of specific tells, but because —negative parallelism!— of the volume and consistency in a given piece.
I think we'll get better over time at telling the difference between human and LLM writing as we consume more slop because the patterns will become intuitive. We're all carrying around an exhaustive list of language rules and structure we couldn't fully articulate and often don't even know we're aware of. Learning to identify (current) LLM writing isn't a difficult task for our language centers to manage.
If the LLM’s remain the same, their tells will become more and more recognizable. But I think this is more likely to be a phase? I don’t think Claude says “you’re absolutely right” anymore?