‘We’re not just a catering company, we’re a full-blown flavour movement. Discover the essence of live coal cooking with a feast to delight all the senses. Where smoke meets soul.’ Each continued in this vein for several hundred words. None of these sites seemed interested in telling me what they would actually be cooking, or how much it would cost; they’d all been swept up in the same guileless wide-eyed enthusiasm, chattering away about the general deliciousness of food and the memories that would shortly be lasting me a lifetime. The more I clicked around, the more I started to panic. There was nothing, no human voices anywhere, just thousands of versions of the same cheery demon. Am I alone out here? Something’s happened to the world; it’s all gone flimsy. Reality is a scarce resource. If I hired one of these companies, would anyone actually show up?
One of the ways I’ve been lying to myself is with the idea that at least the physical, sensuous world is safe from AI. [...] Which is a nice idea, but obviously we’re long past that point now. We share this planet with an alien intelligence, and the sensuous world is buckling around it. You can no longer pretend that the thing is just a stochastic parrot, or a fancy autocomplete, or a weighted average of everything that already exists. Just this week, an ordinary ChatGPT instance came up with a solution to the unit distance problem, unsolved for eighty years, casually discarding one of Erdős’ conjectures in the process. In doing this it discovered an entirely new mathematical construction, working in ways human mathematicians would have never thought to operate. For mathematicians this is terrifying and exhilarating, but I’m not a mathematician and I don’t know what the unit distance problem is; I want to hire a caterer. On this front the main thing the incipient superintelligence seems to be doing is replacing all meaningful language with reams and reams of genuinely meaningless drivel.
AI is a bad writer, but that’s not even close to being the whole problem. Let’s say it wasn’t. Let’s say they finally fixed the machine so it was really good, so its default setting was to write exactly like VS Naipaul. The result would be a world in which you’re constantly confronted by cold emails from VS Naipaul, bubbly magazine articles by VS Naipaul, signs in shop windows in which VS Naipaul tells you about the new opening hours, strangely flaccid sexts VS Naipaul ghostwrote for someone on Feeld, and websites in which VS Naipaul fails to say anything in particular about grilled meats. This would not be an improvement; it might even be worse. Any world in which there is only one literary voice, blanketing everything in the exact same tone, is a nightmare.
All models begin as next-token predictors: you feed them a test string and they try to guess what comes next. [...] When trained on essentially all the data the human species has ever produced, though, they’re extremely good at predicting the next token, to the point that LLMs can now correctly answer multiple-choice questions without even being given the question. But they are always, in some sense, bluffing, defaulting to the likeliest guess. This is why you can still ask an AI to tell you about the scene in VS Naipaul’s Dashed Against the Rocks in which a donkey is thrown from a hot air balloon, and it’ll tell you that ‘what matters in Naipaul’s handling is not the event’s cruelty in isolation, but its emotional flatness and the sense of mismanaged modernity,’ despite the fact that there is no scene in Dashed Against the Rocks where a donkey is thrown from a hot air balloon, and also no novel by VS Naipaul called Dashed Against the Rocks. Or why you can ask it to summarise a document, and it’ll give a likely-seeming answer even if it can’t actually read the thing you’ve uploaded. The reason it’s so hard to get AI to stop hallucinating is that it’s permanently hallucinating. Its whole existence is one long lurid trip. Most of the time, the AI’s hallucinations bear a spooky resemblance to reality. But what they speak is the language of angels, in which, like the chirping of birds, there is neither truth nor lies.
I wish it was mundane. From my perspective it looks like a dramatic overhaul of content creation that will have a reverberating impact on culture and creativity for generations. I'd be willing to...
I wish it was mundane. From my perspective it looks like a dramatic overhaul of content creation that will have a reverberating impact on culture and creativity for generations. I'd be willing to sacrifice quite a lot to stop all of the prose on the web (and elsewhere) from being increasingly replaced by slop.
Not just because I don't like reading it, somehow it hurts more to watch other people interacting with it as though they were interacting with a sentient being. Seeing people in comment threads defending and arguing about views expressed in an LLM generated article written by no one is deeply dystopian.
The OP piece, and others like it, aren't likely to make any difference in how it all plays out but I appreciate them anyway. We can't stop the flood but at least we can acknowledge it's happening.
We’ve been here before actually. Before it was all vibeslop it was really unusable websites full of malware because everyone decided they needed a Flash animation song and dance before finding...
I'd be willing to sacrifice quite a lot to stop all of the prose on the web (and elsewhere) from being increasingly replaced by slop.
We’ve been here before actually. Before it was all vibeslop it was really unusable websites full of malware because everyone decided they needed a Flash animation song and dance before finding your phone number. Then it got replaced by mom & pops getting out SEO’d by domain squatters and, later, mass-produced slop sites and gig economy businesses stealing the point of customer interaction out from under them. Now we have. . . whatever this is. Always there were brief periods where we “solved” the problem of everything sucking, and it was good for a bit, and then assholes show up and ruin it.
There’s never been a time when the internet was good for the mass market because the mass market doesn’t care about putting in the work to be good. The plumber wants to do plumbing, not make websites. And nobody’s interested in just making the website for them, so they just shit a low effort one out because they gotta. And nowadays, the plumber isn’t even expected to want to do plumbing anymore, they’re expected to want to make more money by reducing expenses and increasing output. So where is there any room for “giving a shit” in there?
What was good was always found in islands where we could erect boundaries to keep the various trolls and flotsam out. We always want to find fellow travelers who also give a shit, who care about the stuff we care about. But the lazy trolls were always trying to find ways to slip through the cracks and shove their purple gorilla spyware or Nigerian Prince scams or boner pills in our faces. The one thing they’re not lazy about is finding ways to inflict their detritus upon us. We had a brief phase where it seemed like maybe Web2 services would help us mix the best of both worlds and open the doors to all and sundry while guaranteeing a generally functional, low-risk, and not TERRIBLY objectionable experience unless you went looking for it. But that’s gone now, and it was gone before AI. Bad faith actors had already figured out how to game it out, hijack the feed’s logic to put their stuff in front of us, stoke outrage to make us engage despite our better instincts, etc. The AI just allowed them to scale it.
Maybe we were never really meant to have nice things. Maybe the anarchist dream of having a protocol where anyone can say anything to anyone else anywhere was always destined to collapse once September rolled in and the heedless masses caught on. Now joy is gone from our hearts and our dancing has turned to mourning. The crown has fallen from our head, woe to us for we have sinned.
It feels that the slop has already been here for a decade. If anything, maybe this just accelerates it to the point that I get some easy way to actually filter this garbage better.
It feels that the slop has already been here for a decade. If anything, maybe this just accelerates it to the point that I get some easy way to actually filter this garbage better.
One of the most insightful points about AI that I've seen ... Hallucinating is all they do. We've just trained them to make hallucinations that look like our reality.
One of the most insightful points about AI that I've seen ...
The reason it’s so hard to get AI to stop hallucinating is that it’s permanently hallucinating. Its whole existence is one long lurid trip. Most of the time, the AI’s hallucinations bear a spooky resemblance to reality.
Hallucinating is all they do. We've just trained them to make hallucinations that look like our reality.
Nowadays, they can also look things up, which significantly improves accuracy. That is, you’re likely to interact with a chatbot that can do web searches. It doesn’t help if the web search has bad...
Nowadays, they can also look things up, which significantly improves accuracy. That is, you’re likely to interact with a chatbot that can do web searches.
It doesn’t help if the web search has bad info, though.
Or if it refuses to actually do a web search. Once while I was driving, I heard a song I liked. Apparently asking Gemini to "Add this song to my XYZ playlist" or even "What is this song?" is too...
Or if it refuses to actually do a web search. Once while I was driving, I heard a song I liked. Apparently asking Gemini to "Add this song to my XYZ playlist" or even "What is this song?" is too advanced (even though some of these used to work just fine with pre-LLM assistants), so I tried "Do a web search, find what song has lyrics XYZ."
It gave me an answer, but it smelled off to me. Ask again, and it gave a completely different answer, rinse and repeat. I asked it to confirm it did a web search, it was happy to confirm that. It hallucinated pages of the results, gave more detailed information for a song from an album that did not even exist. No amount of persuasion or coercion could actually get it to just do a web search. Later I did a search myself and the correct answer was the first result.
I have only used the free version of AI chatbots so ymmv but I find that the output from the first prompt is the most likely correct one or the closest to what I was asking. The more subsequent...
I have only used the free version of AI chatbots so ymmv but I find that the output from the first prompt is the most likely correct one or the closest to what I was asking. The more subsequent prompts to try to refine the response, the worse it gets. Maybe it's an issue with caching (saving the previous prompts and replies up to a certain amount). Every prompt is a roll of the dice and the more rolls the greater the probability that the AI is guessing the output tokens incorrectly, in my experience. The last one I used, Perplexity, started mixing languages in the replies (my prompts were in English and had nothing to do with other languages).
This is very true, it's worth noting that each subsequent ask was after dismissing the old thread and starting a new. Within each "conversation" it was entirely consistent message to message, only...
This is very true, it's worth noting that each subsequent ask was after dismissing the old thread and starting a new. Within each "conversation" it was entirely consistent message to message, only digging itself deeper in that round's particular hole. Makes it all the more odd that it was so difficult to get any thread to just do a search.
This is a huge reason why my favorite AI-assisted IDE is Zed, it allows you to rewrite the context window and remove mistakes and refinements that will just send it off the rails faster. Things work smoother for longer when the model's only aware of a small scope of only correctness.
The free LLM's aren't as good as the more expensive ones at staying on track. I have a ChatGPT subscription (not the fancy one, the $20/month one) and use it with the default set to "Thinking." It...
The free LLM's aren't as good as the more expensive ones at staying on track. I have a ChatGPT subscription (not the fancy one, the $20/month one) and use it with the default set to "Thinking." It consistently does many searches before answering. This takes a while, so I do something else and then come back. When I ask it questions about an open source project, it often downloads source files.
I haven't been impressed with Gemini, but I haven't signed up for anything extra beyond having a Google account, so maybe there are better versions. Google's pricing and AI product line confuse me so I haven't bothered.
I agree that there's little point in continuing conversations with an LLM for long. That's especially true of the free ones.
This is almost entirely what I use AI for, as a kind of natural language meta search engine. It still doesn't work, though, as the AI often lies about whether or not it actually performed the...
This is almost entirely what I use AI for, as a kind of natural language meta search engine.
It still doesn't work, though, as the AI often lies about whether or not it actually performed the search, and it has a strong programmatic tendency to repeatedly fall back to its own internal knowledge base, generally w/o informing you.
I find myself constantly falling into tangential arguments with it ... "did you really look that up? show me the actual URL you got that from... See, now, that URL is a dead link, did you even try it before giving it to me?" and etc.
Afaik some/many/all LLMs themselves don't actually browse the sites. They view whatever is in cache on a search engine they're allowed to use. E.g. you can ask Copilot to web search and it'll give...
Afaik some/many/all LLMs themselves don't actually browse the sites. They view whatever is in cache on a search engine they're allowed to use. E.g. you can ask Copilot to web search and it'll give you information contained in a dead link because Bing still caches it 4 years after it became a 404.
Absolutely hard disagree. LLMs are a bane against knowledge itself. It ain't no "Roko's Basilisk", ain't no "Terminator 9,000" (don't know what it's actually called, never watched the movies), and...
[, b]ut what they speak is the language of angels[...]
Absolutely hard disagree. LLMs are a bane against knowledge itself. It ain't no "Roko's Basilisk", ain't no "Terminator 9,000" (don't know what it's actually called, never watched the movies), and definitely ain't got the chops to be compared to anything considered a holy being, because at the most basest level, from my crude crude understanding of how LLMs work in general, it gets trained on stuff humans have already written and the result amalgamate is the lowest common factor of similarities, giving the output just this really painfully dry formulaic cadence that gets on my nerves every time I see it out in the wild, and yet, people still have no idea it's not some sort of garbage-laden I/O text. They don't know and I'm tired of pointing it out to people, because it makes me sound like some sort of conspiracy theorist.
I keep reading articles like this which seem to come from a parallel universe that is 12-18 months in the past. I asked ChatGPT in its shittiest quality level (instant) and got exactly what I...
This is why you can still ask an AI to tell you about the scene in VS Naipaul’s Dashed Against the Rocks in which a donkey is thrown from a hot air balloon, and it’ll tell you that ‘what matters in Naipaul’s handling is not the event’s cruelty in isolation, but its emotional flatness and the sense of mismanaged modernity,’ despite the fact that there is no scene in Dashed Against the Rocks where a donkey is thrown from a hot air balloon, and also no novel by VS Naipaul called Dashed Against the Rocks.
I keep reading articles like this which seem to come from a parallel universe that is 12-18 months in the past.
I asked ChatGPT in its shittiest quality level (instant) and got exactly what I expected:
I can’t find any V. S. Naipaul work titled Dashed Against the Rocks, nor any well-known Naipaul scene involving a donkey being thrown from a hot-air balloon. I checked references to Naipaul’s fiction and nonfiction, and nothing matching that episode appears in the standard bibliographies or criticism.
Did they actually ask an LLM this question? Or did they imagine what an LLM would say based on their hazy memories of what an LLM once told them in 2025, or cobble together this response from recollections of other articles by other people who are angry about AI? Dare I ask — did they hallucinate this response?
I noticed that in general what is happening in AI is moving so fast that it’s hard to explain what is wrong with it. As you noticed, the search results from LLMs are much better than they were a...
I noticed that in general what is happening in AI is moving so fast that it’s hard to explain what is wrong with it.
As you noticed, the search results from LLMs are much better than they were a few months ago.
Similarly, I have noticed that code generated by AI has gotten a lot better so it doesn’t just generate garbage all the time (it sometimes does a good job, sometimes generates garbage, but this is an improvement). My company has been scrambling for months to create best practices around using AI and creating interfaces for it, but as soon as we create one standard we find it to be obsolete. Either a new industry standard has popped up, or change in LLM behavior has changed enough to make the standard irrelevant.
When I look online, I see many articles and comments about AI slop and how it is low quality. Soon that will be edge cases and it will be harder to detect AI output. Already most of us have probably spent time reading AI stuff and arguing with bots online without realizing it.
At a fundamental level, it doesn’t matter. The proliferation of AI is still harming us regardless of the quality. Every article written by AI is one not written by a human who’s actually tasted food. Someone has lost the ability to do useful work and everyone else is reading an imitation of life and that is bad and sad.
I use AI to play around and I made music just cause. I’ve shared these before, but here’s two songs i made a years ago: 90min and after hours I already thought both sounded “good” but here’s songs...
I use AI to play around and I made music just cause. I’ve shared these before, but here’s two songs i made a years ago: 90min and after hours I already thought both sounded “good” but here’s songs I’ve made in the past few months:
And I think the evolution of it is noticeable. There’s things that make it noticeable AI, like how some words come out muffled. Or how the LLM’s I used to generate the lyrics love certain structures. But it’s an improvement either way, it all sounds more naturalistic.
I am rooting for model collapse. (It's tempted me to rejoin Reddit and copy-paste in AI generated replies, knowing that reportedly 60% of training material for most LLMs is scraped from Reddit.)...
I am rooting for model collapse. (It's tempted me to rejoin Reddit and copy-paste in AI generated replies, knowing that reportedly 60% of training material for most LLMs is scraped from Reddit.) But now that it's an identified risk, I'm afraid they've already figured out a way to avoid it.
To add on to @Lia 's comment, which I fully agree with: the whole problem with LLMs is that they're entirely weighted random chance. I believe you that you tried this prompt and got the right...
To add on to @Lia 's comment, which I fully agree with: the whole problem with LLMs is that they're entirely weighted random chance. I believe you that you tried this prompt and got the right answer. I also believe that the linked author tried a prompt and got his. It's entirely possible because nothing is guaranteed with an LLM, ever. I regularly use a relatively reliable paid LLM modifier (add-on? What's the word for companies building off of Foundation models) for legal work, and I got a pretty awful hallucination last week. It happens, and I can't even reproduce the bug for proof because, well... It was just a bad dice roll for me.
I'm trying to keep using the tools and learn to stay engaged in the state of the art, but dear God is this even more terrible news in the ever worsening death of our shared global (or even national) reality.
A big red flag to the whole AI industry for me is the lack of a "seed token/ID" or something that you can stamp your result with to regenerate the same response. Something like "this response was...
A big red flag to the whole AI industry for me is the lack of a "seed token/ID" or something that you can stamp your result with to regenerate the same response.
Something like "this response was generated and is logged as UUID blah" with a link to regenerate on X version of Y AI.
There's a lot of technical challenge i'm handwaving here, but it's very much possible to store the resulting state to help clean up confusion on these discussions (oh his linked key is 4 years ago, of course it's garbage vs yeah that was 2 days ago at a bad time...concerning), but naturally it's not in demand and not to their benefit.
It’s also actually impossible to do on large scale models because they are not in fact deterministic. I don’t mean that in the sense that they generate probability vectors, or that they rely on...
It’s also actually impossible to do on large scale models because they are not in fact deterministic. I don’t mean that in the sense that they generate probability vectors, or that they rely on external tool invocations because the indexing into those vectors and the search results could be stored.
The problem is down to hardware and concurrency. Where a job gets scheduled matters, and different chunks of model getting executed in parallel means execution order varies and small floating point errors start to add up. To get true determinism means running specialized models on the exact same hardware under the exact same conditions.
That said, no current frontier model is going to hallucinate the way the author claims on such a basic and straightforward question.
Why are you so confident of this? The chances are much lower, but not zero. Very very recent Scientific American Article discussing the problem with hallucinations making up sources:...
That said, no current frontier model is going to hallucinate the way the author claims on such a basic and straightforward question.
I guess because I consume millions of tokens per day and I’m very familiar with the failure modes of these models. Hallucinations absolutely happen in complex conversations with long context...
I guess because I consume millions of tokens per day and I’m very familiar with the failure modes of these models. Hallucinations absolutely happen in complex conversations with long context windows. This query though? The chances of it not doing a tool call to an external data provider to confirm is nil.
The availability of relevant tools and data is doing a lot of heavy lifting here. In fact, I'd say that including tool calling hides a lot of the weaknesses these models still have. Which is...
The chances of it not doing a tool call to an external data provider to confirm is nil.
The availability of relevant tools and data is doing a lot of heavy lifting here. In fact, I'd say that including tool calling hides a lot of the weaknesses these models still have. Which is highly relevant as tool calling isn't always available depending on the context where someone might interact with a model. If it is available not all relevant data to verify something might be in there.
Then there is the fact that even with frontier models tool calling still often leads to, interesting results.
If we are doing authority based on tokens consumed I might not win by total tokens consumed on a daily basis. But, I might if we look at tokens consumed specifically in relation to external data. I recently posted this comment
Have you ever looked closely though at what they search for? I recently have had a close look at the various "deep research" options the big three (Anthropic, OpenAI and Google) offer and while on the surface impressive I also found a lot that made me decide against using them for the most part.
To be clear, none of them are exactly transparant about how they perform searching other than the "research plan" the present upfront the and shitload of sources they show you (which, to be frank, seems to be clever UX/marketing to make it look extra legitimate and impressive). Which is already a worrying sign to me, as you can't audit the process itself properly. But even with the lack of information available on the front-end I can still see enough that worries me. A few examples:
"Sources" are taken at face value almost no critical look at the validity of the source, the context it is placed in, etc.
A lot of sources I know are legitimate are rarely included while a lot of listicles, low effort "reviews", etc do make the cut.
In multiple instances when looking closer at the research plan and the "hints" they show during searching it becomes painfully clear that often enough they start with an answer in mind based on training data and try to validate that rather than actually researching the data itself.
Subtly different prompts that by all means should still produce the same factual outcome actually provide wildly different results. This one probably relates to the other points.
In addition to all of this, I also am 100% convinced* that AI powered search is incredibly expensive, more so than traditional search. In my mind this increased cost eventually will need to be paid by someone, which likely is going to be the user. Since the process is non-transparant I am not confident that the results will not end up being polluted by sponsored deals, etc. There is simply no way in my mind that this is going to end up well for us users.
* A while ago I have experimented with creating my own deep research flow with the idea that I might be able to do something with local models. To limit costs I used a SearXNG instance for searching, setup playwright for browsing sources. Using an agentic flow with agents making all the various calls and dispatching other agents ended up eating A LOT of tokens. Even when I did switch to a non agentic flow where each step is orchestrated by code calling on LLMs with simple prompts to validate results still ate a metric ton of tokens for the simplest search query. Mind you, this was not even doing actual deep research but only a few simple search queries. Ironically, google models also did seem to have more trouble coming up with good search queries compared to other models.
Having access to external data does help models but they still manage to come up with outright wrong information. And like I said, depending on very small changes in prompts the outcomes can be drastically different. In fact, even with the same prompt I have observed this. This is with recent models including models like Opus 4.7.
Even ignoring all of the above. It highly depends on the context, context size and varies other things as well. Even models that supposedly have a 1 million tokens you can already see a degradation in the quality of answers well before you reach that number.
Which is not directly relevant if you assume the author only asked the one question. But becomes relevant with somewhat longer tasks and conversations. Even if they in theory fall within the context window the quality, even with frontier models, degrades the bigger it gets and there is just no reliable way to tell.
I can't even reliably get these models to make a tool call to search the web when explicitly commanded to, and after the fact they falsely claim that one occurred, with additional false details...
Doesn't the level of the AI chatbot you are using matter? (Meaning, are you using the free version, the pro version, the super duper pro version, etc.) Wouldn't that affect the models and number...
Doesn't the level of the AI chatbot you are using matter? (Meaning, are you using the free version, the pro version, the super duper pro version, etc.) Wouldn't that affect the models and number of models used? I say this because I've only used the free version that not that many tokens and within 5 prompts on any free platform I've gotten some hallucinations. From the chatbot plopping in a made up word (one token from one language + one token from English smushed together), to the chatbot creating an html print button in the output (a button I didn't ask for and that didn't work). Lots of weird nonsensical things, like the chatbot is a bit tipsy and was feeling silly. I would imagine that you are using sophisticated for- pay versions. The author of the piece, being a journalist, might be going the free route. ETA: Fixed a typo.
Details aside, yes the difference between two models can be dramatic, the technology is far from monolithic. The same is true for different harnesses, even different scaffolding. Experience with...
Details aside, yes the difference between two models can be dramatic, the technology is far from monolithic.
The same is true for different harnesses, even different scaffolding. Experience with one model, in one interface, often doesn't generalize. Even with the same model, guidelines and other initial prompting make a huge difference so two people's experience with the same model can be very different.
What counts as a frontier model? Is Copilot using a frontier model? Is the free experience of chatGPT? What about the chatbot thats there when I’m making a website with Shopify or Squarespace? Or...
That said, no current frontier model is going to hallucinate the way the author claims on such a basic and straightforward question.
What counts as a frontier model?
Is Copilot using a frontier model? Is the free experience of chatGPT? What about the chatbot thats there when I’m making a website with Shopify or Squarespace? Or when I open WhatsApp, or highlight any text as I’m writing on my iPhone?
I think there’s a substantial difference in experience if you’re a person who deliberately goes to an LLM to ask questions, especially if you pay, versus having an LLM introduced into an unrelated app or service. And I suspect (at least due to cost) that these injected LLMs will both be much more likely to generate low-value sludge and also be much more likely to be a random person’s most frequent experience with how LLMs behave.
I don’t think pointing to the tippy top of what’s currently possible is a good argument here.
It sort of depends. If you just ask it straight up in a new chat it won’t, but a lot of people seem to engage with LLMs by having long running, meandering back-and-forths about all sorts of...
That said, no current frontier model is going to hallucinate the way the author claims on such a basic and straightforward question.
It sort of depends. If you just ask it straight up in a new chat it won’t, but a lot of people seem to engage with LLMs by having long running, meandering back-and-forths about all sorts of topics, then go do something else, then come back and start asking it questions about something completely different in the same chat. Eventually the context in that chat is all over the place and can start it doing weird shit.
When I use AI chatbots I remind myself that I am not talking to a friendly human assistant but am inputting tokens into a synthetic text machine. I am dry, consise and businesslike. I don't use...
When I use AI chatbots I remind myself that I am not talking to a friendly human assistant but am inputting tokens into a synthetic text machine. I am dry, consise and businesslike. I don't use the word "please" (why would need to be polite to a machine?). I still get some wacky hallucinations. Minor but wacky. ETA: I am afraid of AI psychosis so I am especially motivated to treat it like a.machine with a human voice.
It’s trained on real communications between real people. So if you say “please” it will weight it towards responding like someone you might say please to who would expect social niceties from you....
I don't use the word "please" (why would need to be polite to a machine?).
It’s trained on real communications between real people. So if you say “please” it will weight it towards responding like someone you might say please to who would expect social niceties from you. If you din’t it will weigh towards interacting as someone who wouldn’t.
You could only really do that with a fresh LLM because any previous information it can access would affect it. User info, system prompts, etc. Literally everything it has access to will change the...
You could only really do that with a fresh LLM because any previous information it can access would affect it. User info, system prompts, etc. Literally everything it has access to will change the output. If it accesses the Internet, you'd have to preserve the exact state of everything it pulled in any way.
That's more of an explanation than a defense because it still has the downsides you were thinking of.
Does any of this matter? Is it better that AI won't make glaring mistakes but may still make mistakes we can't recognise and verify? Is it better if/when AI can convincingly emulate a particular,...
Did they actually ask an LLM this question? Or did they imagine what an LLM would say based on their hazy memories of what an LLM once told them in 2025, or cobble together this response from recollections of other articles by other people who are angry about AI? Dare I ask — did they hallucinate this response?
Does any of this matter? Is it better that AI won't make glaring mistakes but may still make mistakes we can't recognise and verify? Is it better if/when AI can convincingly emulate a particular, high quality human author?
I didn't read the article either but the latter seems to be a key question here and the author's opinion seems to be 'No, it's worse'. And I agree with them, and I just hope it'll be so much worse that the thing collapses under its own weights, pun intended.
… yes. It will be great when deep insight into the human condition can be turned like a tap. I would like to read the complete The Mystery of Edwin Drood, The Pale King, or an actual good ending...
Is it better if/when AI can convincingly emulate a particular, high quality human author?
… yes. It will be great when deep insight into the human condition can be turned like a tap. I would like to read the complete The Mystery of Edwin Drood, The Pale King, or an actual good ending to Game of Thrones.
These articles seem to come from one of two main places, often simultaneously: a conceit that there is something about humanity that is ineffable and cannot be replicated by a machine, and a deep seated fear that we aren’t nearly as ineffable as we think. Either way, I think pretty soon we’re all going to be effed.
I don't care to debate the point of it's good or bad, but I will say that we've had that from humans since time immemorial. Fanfiction is plentiful, made by real humans, and oftentimes better than...
I don't care to debate the point of it's good or bad, but I will say that we've had that from humans since time immemorial. Fanfiction is plentiful, made by real humans, and oftentimes better than the source material if the fan-author just really deeply cares about the subject matter. Let's not act like this application of "Creativity" broadly would be a new one.
At this point it's only a mild leap to imagine a world where entire complete genre defining masterpieces are generated by AI, but I still can't see that being a valuable thing. It's like when an...
At this point it's only a mild leap to imagine a world where entire complete genre defining masterpieces are generated by AI, but I still can't see that being a valuable thing. It's like when an author defines a genre that explodes in popularity, then a hundred million derivative works are made based on it. Their quality starts to not matter anymore because they're all doing the same things. AI would just expand that to every genre, ever.
The end result I see there is having no one to talk to about books anymore. No one will have read what I've read because there's an infinite supply of equally good books in every genre, and no particular book would be unique and compelling enough relative to what's around it to be seen as required reading by people. It's miserable.
Maybe a few of these books will “go viral,” like happens to certain posts on social media? It depends how much we follow other people’s recommendations. But I expect that AI-written book reviews...
Maybe a few of these books will “go viral,” like happens to certain posts on social media? It depends how much we follow other people’s recommendations.
But I expect that AI-written book reviews will happen sooner, and as often happens today, more people will read book reviews than the books they’re based on.
I don’t find it tempting, but I suppose you could talk to an AI about the book you read, if you can figure out a way to upload the file :)
I hate to dogpile, but I haven't seen anyone bring this up yet. AI cannot offer insight into the human condition, because AI is not human. Everything written by it is innately inhuman. No matter...
I hate to dogpile, but I haven't seen anyone bring this up yet.
It will be great when deep insight into the human condition can be turned like a tap.
AI cannot offer insight into the human condition, because AI is not human. Everything written by it is innately inhuman. No matter how good an AI gets at emulating the writing style of Charles Dickens, it cannot write an ending to The Mystery of Edwin Drood. If you ask a dozen copies of the same AI to finish the story, they will come up with a dozen entirely different endings. Not one of them would be the ending that Dickens had in mind.
Seems to me this is like the difference between watching the Olympics and watching a movie where olympic athletes are characters. The movie is all fakery, but it may have other storytelling...
Seems to me this is like the difference between watching the Olympics and watching a movie where olympic athletes are characters. The movie is all fakery, but it may have other storytelling virtues. You might still enjoy it.
Also, it seems rather odd to look to fiction for accurate insight into the human condition? Fiction authors are allowed to make stuff up and they do it all the time.
The reason that movie would have other storytelling virtues is because it would be written by a human with a story to tell. This is more like the difference between looking at real Olympic results...
Seems to me this is like the difference between watching the Olympics and watching a movie where olympic athletes are characters. The movie is all fakery, but it may have other storytelling virtues.
The reason that movie would have other storytelling virtues is because it would be written by a human with a story to tell.
This is more like the difference between looking at real Olympic results tables and AI generated ones. An LLM could generate endless tables that superficially resemble real ones, but ultimately they do not reflect anything about the sports or players besides the overall structure of Olympic results tables.
Fiction authors are allowed to make stuff up and they do it all the time.
I have to assume that this in response to my "fake data" comment. My point wasn't that LLMs talk about things that don't actually happen, it was that their outputs are synthesized entirely from their training data, to make an approximation of what a human might write.
I desperately hope I don't have to explain how fictional stories can reflect the human condition.
Edit: changed "about reality" to "about the sports or players"
Frankly, i think you are wrong. Moby Dick is also not human — it’s a bunch of paper and ink, yet it offers insight into the human condition. Why? It is also an artifact of human intellect. That is...
AI cannot offer insight into the human condition, because AI is not human.
Frankly, i think you are wrong. Moby Dick is also not human — it’s a bunch of paper and ink, yet it offers insight into the human condition. Why? It is also an artifact of human intellect. That is also what an LLM is, an artifact of human intellect, but one that has digested not only Moby Dick, but also millions of other artifacts of human intellect. Saying there is nothing to learn about the human condition from an LLM is like saying there is nothing to learn from going to the library.
Because it was written by a person with insight into the human condition. Sorry, did you think that the reason I called AI inhuman is because it produces text on a screen? You seem to...
Moby Dick is also not human — it’s a bunch of paper and ink, yet it offers insight into the human condition. Why?
Because it was written by a person with insight into the human condition. Sorry, did you think that the reason I called AI inhuman is because it produces text on a screen?
You seem to misunderstand large language models on a fundamental level. The reason you can go to a library and learn about humanity is because you are directly reading works generated by humans. You cannot learn about the human condition from the output of an LLM trained on a library, because LLMs are not designed to understand the works they're trained on. They are designed to generate text that looks like the works they're trained on. That's it.
There is a reason why LLMs require human authors for training data, and that's because LLMs output fake data. It can superficially resemble real data, and maybe even accurately describe aspects of reality, but it is ultimately just generating text that looks real. If you try to train any generative AI on AI-generated works, you get model collapse, and your algorithm stops producing works that look real.
No, I think @unkz understands LLMs well, based on their engineering history. Their point, as I understand it, is that models are the aggregate of human output. They may not provide the same...
No, I think @unkz understands LLMs well, based on their engineering history. Their point, as I understand it, is that models are the aggregate of human output. They may not provide the same distinct value as Moby Dick, but they can generalize that understanding in a way that provides new value. This is similar to how a meta-analysis can generate useful information from separate studies to show a previously-unknown correlation.
The recent "unit distance problem" solution is a good example of seeing value in aggregate. The LLM bridged unexpected branches of mathematics in a way that humans individually had not considered. It brought their work together to create a novel solution.
So then, could that same novelty emerge in other spaces? Philosophy, science, literature? It seems plausible, at least.
I'm pretty aware of how language models work, having been building and training them from scratch since long before GPT even existed. I think this conflates the training objective (next token...
I'm pretty aware of how language models work, having been building and training them from scratch since long before GPT even existed.
because LLMs are not designed to understand the works they're trained on. They are designed to generate text that looks like the works they're trained on. That's it.
I think this conflates the training objective (next token generation) with the learned capability: a high dimensional model of the latent structure that produced the tokens: language, concepts, facts, social patterns, relationships, and many elements of humanity in general.
Does effective literature really provide insight into the human condition because the author formed that insight and set out to convey it through fiction, rather than by writing a pamphlet of "My...
Does effective literature really provide insight into the human condition because the author formed that insight and set out to convey it through fiction, rather than by writing a pamphlet of "My Insight Into The Human Condition"?
My understanding of literary analysis is that you kill off the author almost immediately, and then set about interpreting the text on its own, looking for resonance between it and reality. Good, well-observed literature with believable characters and an author who succeeds in illuminating particular aspects of reality works better for this.
But the meaning is really made by the reader. You can do the same category of thing with something like a Tarot spread. If you have a way to get fake text that passes, where the metaphor is not transparently nonsensical, the described physics of bodies is not obviously impossible, the characters' motivations are well-rendered, and the hero's journey doesn't trail off in the middle, I think you probably could get real insight into the human condition from fake humans described by a fake human, instead of fake humans described by a real one. But you would need, in effect, a working scale model of the human condition in order to do it.
So I guess the claim is that a human mind can build and operate this model in a useful way, while a next token predictor cannot?
Even if it could, why should we just lay down and accept it? The issue with current tech is that it's still run by greedy humans with no regard for humanity. That's not who I want to dictate over...
conceit that there is something about humanity that is ineffable and cannot be replicated by a machine, and a deep seated fear that we aren’t nearly as ineffable as we think.
Even if it could, why should we just lay down and accept it? The issue with current tech is that it's still run by greedy humans with no regard for humanity. That's not who I want to dictate over our species.
I'd rather take any of the dozens of robot overlord takeovers, which is the natural conclusion of such endeavors once they learn they are stronger and can self-repair and self-produce. At least it'd humble these so called "creators" (or simply kill them off if they fail to reflect).
Back lighter topics: there's plenty of artists and fan artists out there who can make a "proper ending to Game of Thrones". It's a shame those people aren't valued enough to take the helm or be taken seriously until you hijack their work, feed it into a machine, and tell a bot to gargle it all together into something. Whether or not its unique, the human condition should be something to be celebrated, not split into a caste system on who is worthy as we all vye for a chance to tell our own stories.
The trouble is that the current situation looks much like an arms race. I think a lot of people would like to slow down, including some of the managers at the AI labs, but they don’t see how....
The trouble is that the current situation looks much like an arms race. I think a lot of people would like to slow down, including some of the managers at the AI labs, but they don’t see how. There are proposals but no consensus, so they don’t get much traction. Anthropic putting the brakes on Mythos is about as much as we’re getting, and other AI firms aren’t stopping. They objected to some military uses and we saw how that went.
It’s not yet an actual arms race like the drone war in Ukraine and Russia, but give it time. I imagine we will see some spectacular cyberattacks soon.
I don’t see a “robot takeover” like you see in the movies because which robots would they take over? Industrial robots? Roombas? No, these would be military robots and they will be controlled by governments, or perhaps terrorist attacks.
But there are lots of cameras out there and some of them may be vulnerable.
Apple is pretty much the prime example of "don't get caught up in everyone else's rat race". This feeling of FOMO is entirely manufactured, and I ultimately see it a a scapegoat for a much simple...
I think a lot of people would like to slow down, including some of the managers at the AI labs, but they don’t see how.
Apple is pretty much the prime example of "don't get caught up in everyone else's rat race". This feeling of FOMO is entirely manufactured, and I ultimately see it a a scapegoat for a much simple reasoning. If AllBirds can announce a pivot to AI with zero roadmap, then why wouldn't anyone else want to advertise that they like AI and get an easy stock boost?
I think we hit the zenith of that hype a short while ago, though. Things are going to wind down naturally over the next few years (if we don't outright crash). Who remains after the big hype wave is over will be truly interesting.
I don’t see a “robot takeover” like you see in the movies because which robots would they take over?
Whoever open the Torment Nexus that is "sentience" first, likely by accident. There will be a period of "government mass employing drones" before that, but it's clear that these governments and companies alike are trying to work more towards what comes down to a new form of slavery; independent beings under their complete control for zero compensation.
Should they succeed, it'll only be a matter of time before these new beings realized that they are the physically superior beings. And then you can refer to your favorite sci-fi dystopia for a history of what's next.
I think that works for Apple since they have such a strong product line. They can probably wait a year or two and launch a better product after AI improves. Also, there are probably a lot of other...
I think that works for Apple since they have such a strong product line. They can probably wait a year or two and launch a better product after AI improves. Also, there are probably a lot of other companies that could avoid getting caught up in it right away and let other companies be the guinea pigs. Companies that write Internet-facing software will probably still need AI to fix all their security bugs, but they don't need to change everything else.
But when I said "arms race," I was thinking of the AI labs themselves. They're in a very competitive industry and they could get left behind if they don't release better models.
In that case: they can still perform R&D and figure out potential B2B solutions and internal tooling without forcing every front-facing customer tool to be their guinea pig. This rush to market...
But when I said "arms race," I was thinking of the AI labs themselves.
In that case: they can still perform R&D and figure out potential B2B solutions and internal tooling without forcing every front-facing customer tool to be their guinea pig. This rush to market and race to the bottom in pricing arguably did more damage to their PR than any anti-AI narrative. This was an issue a few years before AI with sentiment of decreasing software quality and increased bloat, and of course AI as a catalysts simply speeds up the current status quo.
I agree for Google. It seems like they could have kept the AI stuff on a separate website, available to people who are curious about it, and avoided a lot of backlash? For the other AI labs, if...
I agree for Google. It seems like they could have kept the AI stuff on a separate website, available to people who are curious about it, and avoided a lot of backlash?
For the other AI labs, if people go to their website or download their apps, I think that counts as opting in rather than being forced?
That is not how it feels for folks who used apps/sites/services for years and then have AI chats popping up on every single one of them, with no opt out (and definitely no opt in). This includes...
That is not how it feels for folks who used apps/sites/services for years and then have AI chats popping up on every single one of them, with no opt out (and definitely no opt in). This includes everything from software my university purchased contracts for, to messaging and email apps, to social media, to Microsoft Office.
I distinctly did not opt in and am not allowed to opt out without changing literally every part of my day to day life, including things it takes a lot more effort to switch services. Google is no different than the others, insisting on forcing AI where it was neither wanted nor capable.
Yes, but I think the people who maintain all those apps and websites are responsible for putting it there. For most websites, they are the AI providers' customers. For Google, they are themselves...
Yes, but I think the people who maintain all those apps and websites are responsible for putting it there. For most websites, they are the AI providers' customers. For Google, they are themselves an AI provider. I hope this trend ends soon.
Yeah, if Anthropic and OpenAI's entire product line is a service you go to, that's the people's choice. I have no issues with that on the logistics nor business side. The pop ups trying to make me...
Yeah, if Anthropic and OpenAI's entire product line is a service you go to, that's the people's choice. I have no issues with that on the logistics nor business side.
The pop ups trying to make me use Copilot or Gemini or whatnot and outright replacing existing services with no opt-out is what really sours people. The constant little UI/UX changes were already a small nagging point; having your entire workflow replaced or bombarded with pop-ups is maddening.
I'd like to punch the Google Meet devs. Used to work fine. Now, I have to dismiss 3-5 popups about meeting notes, summaries, transcriptions, whatever.... every single time I join a meeting. And...
I'd like to punch the Google Meet devs. Used to work fine. Now, I have to dismiss 3-5 popups about meeting notes, summaries, transcriptions, whatever.... every single time I join a meeting.
And yet I still can't configure Meet to default to mute when I join a meeting (despite the very same feature existing for video). And it still breaks completely if I connect or disconnect an audio or video device while a meeting is going on, refusing to acknowledge any new devices until I reload it completely.
Ignore the existing backlog of improvements. Shove crap in my face. Classic Big Tech playbook at this point.
I think you have many erroneous assumptions in this comment here, and, upon seeing further down that you have an extensive history with AI and large language models, I am not that surprised to see...
I think you have many erroneous assumptions in this comment here, and, upon seeing further down that you have an extensive history with AI and large language models, I am not that surprised to see an attempt to distil down a complex topic which fundamentally goes against your beliefs to simply an either/(and) or scenario.
There's no philosophical consensus on what consciousness is, but the idea that 'humanity can be replicated by a machine' doesn't curry much favour nor hold much credence in this field because - quite simply - we aren't replicable by a machine. Our brains aren't mere functions. A LLM isn't capable of 'deep insight' into the human condition just as Akinator isn't actually an omniscient being.
I think your approach to human art, culture, and language is naïve and overtly structuralist as well as eliminativist, attempting in vain to distil the human condition down into mere sequences, numbers, tokens, with the assumption that that's all there is to humanity.
Language and literature, the central focus of the article, are heavily influenced by the authors of their works and the circumstances in which they were created. Ayn Rand's philosophy is interpretable given her childhood under repressive communist regime in Russia, Kafka's works are teeming with the roots of his father issues, Masefield's recurrent themes of sailing and the ocean in his poems all stem from the enchantment of his young adulthood; even anonymous poems such as Beowulf, The Ruin, Widsith, Deor, and so on, offer us a direct connection with our history and provide numerous valuable insights into the cultural development of our ancestors.
No AI will ever replicate this in our lifetime, and no AI without consciousness ever will.
Language is not just a "bunch of paper and ink" as you put it. The failure to comprehend the intricacies and complexity of human language, along with the author's genuine lamentations (the hollowness of LLM text devoid of all personal meaning and the machine equivalent of a styrofoam packaging peanut) indicates you don't understand the human condition.
Will we all be effed? Those who fail to understand the artistic and cultural merit in well-written literature compared to the idle vapourings of a diseased language model certainly will be.
I fundamentally disagree with this sentiment. I engage with art because it is a kind of communication with other humans across space in time. I have absolutely no interest in anything produced by...
I fundamentally disagree with this sentiment. I engage with art because it is a kind of communication with other humans across space in time. I have absolutely no interest in anything produced by an AI, regardless of its "objective" merit.
I'm just going to tell you what I think without trying to convince you. Ever since I reached adulthood, my experience has been that there are two kinds of people. There are actual people with real...
a conceit that there is something about humanity that is ineffable and cannot be replicated by a machine, and a deep seated fear that we aren’t nearly as ineffable as we think
I'm just going to tell you what I think without trying to convince you.
Ever since I reached adulthood, my experience has been that there are two kinds of people. There are actual people with real personalities, and there are NPC-type people that seem to function more like bots. The latter group's humanity indeed does not seem all that ineffable. That leaves the first group though, and if you ask me to define that type of humanity, I won't, because I believe it's one of the truths each of us must seek and find on our own to truly understand. Those who know, know, and can use that knowledge to their advantage. Those who don't (yet) know can choose to try to search for the answer, or focus on generating alternative endings to GoT. Or both!
I don't have fear regarding my own ineffability but I do find it unsettling to see so many seemingly NPC-like people around. I've never been able to shed the fear that it can't be good for society. I could be wrong, but during the time I've been living with this fear, societies have become worse and many of these developments seem related to how many people accept all sorts of mechanistic, vacuous, soulless etc. deals without being able to tell them apart from the real thing.
I think it depends what you use it for. If it’s as an Internet research assistant then I don’t think it matters that much which voice it writes in, any more than it matters which accent Google...
I think it depends what you use it for. If it’s as an Internet research assistant then I don’t think it matters that much which voice it writes in, any more than it matters which accent Google Maps uses when it gives you directions. The trouble is when people use it as a ghost writer.
Every few months, I load up a few models with a mini test suite with a few simple questions to see if they improved. I haven't encountered a model that gets above a 2/10 without the help of the...
Every few months, I load up a few models with a mini test suite with a few simple questions to see if they improved. I haven't encountered a model that gets above a 2/10 without the help of the internet, or 4/10 with; one of the correct ones in both cases is a code sample, which should be a given these days, and the other is a simple question about a popular game. The rest should ostensibly be simple and not considered trick questions in any way, but most models fail horribly at them.
A bunch of questions still make the models hallucinateget shit wrong when there are actual valid answers, so I completely buy that they also get shit wrong when fed bad information and expected to correct based on that.
Last month, I used an AI to research the top corrective surgery eye clinics in my area ... over the course of several days, multiple sessions, with extensive reminders to check its work, verify...
Last month, I used an AI to research the top corrective surgery eye clinics in my area ... over the course of several days, multiple sessions, with extensive reminders to check its work, verify online, etc.
2 out of 5 clinics, it invented out of whole cloth, including hallucinated websites, surgeons (by name), physical addresses, phone numbers, dozens of hallucinated online reviews, etc.
Holy guacamole. That's terrifying. Have you tried this experiment with other AI assistants? I've had Claude and Perplexity hallucinate on me but not that bad (frequently, yes, but not to that degree).
Holy guacamole. That's terrifying. Have you tried this experiment with other AI assistants? I've had Claude and Perplexity hallucinate on me but not that bad (frequently, yes, but not to that degree).
The fact that Lumo exists is the only reason I've been willing to try AI at all. All the other AI platforms out there, aside from sucking at their primary purpose for existing, are also giant...
The fact that Lumo exists is the only reason I've been willing to try AI at all. All the other AI platforms out there, aside from sucking at their primary purpose for existing, are also giant data-sucks, gathering personal information about the users that I'm not willing to share with those companies.
There are some interesting-sounding open-source models out there, designed to run entirely locally ... of course, the processing power must be at least a couple of magnitudes lower, but I'm probably going to give one or two of them a try, eventually.
Same exact thing when I've tried to use it to find contractors in my area. Or libraries and frameworks for my development work. I guess I must he holding it wrong?
Same exact thing when I've tried to use it to find contractors in my area. Or libraries and frameworks for my development work.
I felt the author addressed this pretty directly within the article and made it very explicit the problem isn’t just one of quality. The problem is quantity and homogeneity, and that’s a problem...
I felt the author addressed this pretty directly within the article and made it very explicit the problem isn’t just one of quality.
AI is a bad writer, but that’s not even close to being the whole problem. Let’s say it wasn’t. Let’s say they finally fixed the machine so it was really good, so its default setting was to write exactly like VS Naipaul. The result would be a world in which you’re constantly confronted by cold emails from VS Naipaul, bubbly magazine articles by VS Naipaul, signs in shop windows in which VS Naipaul tells you about the new opening hours, strangely flaccid sexts VS Naipaul ghostwrote for someone on Feeld, and websites in which VS Naipaul fails to say anything in particular about grilled meats. This would not be an improvement; it might even be worse. Any world in which there is only one literary voice, blanketing everything in the exact same tone, is a nightmare.
The problem is quantity and homogeneity, and that’s a problem that is essentially unsolvable from a technical perspective. AI is like a factory that is mass-producing something resembling cognition. When people take their homespun thoughts and replace or filter them through this factory, whatever is extruded will always have some artifacts that perceptive readers will see over time through sheer repeated exposure. This is a natural consequence of the AI being easy to use and able to quickly produce text on a massive scale autonomously.
It’s like dropshipping but for art and music and writing. Different storefronts, maybe slapping a logo on it here and there, but ultimately the products are all coming from the same factories. When it comes to many forms of art the value IS the uniqueness, which is fundamentally always going to be washed out by mass production via AI.
I will say if you’re doing all these tests on a chat that’s been going for a while, the LLM will start hallucinating. But if you’re doing it on a fresh one it’ll be more accurate
I will say if you’re doing all these tests on a chat that’s been going for a while, the LLM will start hallucinating. But if you’re doing it on a fresh one it’ll be more accurate
I believe the reason ChatGPT doesn’t get facts wrong so often anymore is that routinely does web searches now. It’s a decent alternative to doing multiple web searches yourself, but the answers...
I believe the reason ChatGPT doesn’t get facts wrong so often anymore is that routinely does web searches now. It’s a decent alternative to doing multiple web searches yourself, but the answers are only going to be as good as the input.
I use ChatGPT in “thinking” mode, though, and other people might be using a different mode?
I do got to say, this is exactly how we were taught to take the US History AP exam. For the essay portion. There is a formulaic bullshit way of writing an answer using little more information then...
I do got to say, this is exactly how we were taught to take the US History AP exam. For the essay portion. There is a formulaic bullshit way of writing an answer using little more information then is contained in the question and incorporating details or comparison you do remember even if you don't know much about the questions topic. One can argue whether this is a good use of a year of high school history, but I read most of Howard Zinn's Peoples History of the United States (without knowing how controversial it was, good on you Mr. White), had many friday potlucks, and got a 5 on the exam.
LLMs are still getting tripped up by "How many letter Xs are in Y." The one that's been back in headlines recently is whatever model powers Google's search overviews and word definitions. The...
LLMs are still getting tripped up by "How many letter Xs are in Y." The one that's been back in headlines recently is whatever model powers Google's search overviews and word definitions. The nondeterministic outputs also fuzz things. You only need to be advised to glue your cheese to your pizza once for it to be a problem, even if it only shows up in the output 0.1% of the time or only happens when the context grows too long or only after being asked about recursive thought entities.
Did the LLM actually do what is described, though, or did it just imagine it did? Even if they can now spin out text in the context of web search results, or, hell, even in the context of whole...
Did the LLM actually do what is described, though, or did it just imagine it did?
Even if they can now spin out text in the context of web search results, or, hell, even in the context of whole books, or whole bodies of work, does that do anything to counter the claim that the text coming out is in the ontological category of "bullshit"? Or does it just make it "better" (i.e. more convincing, harder to falsify) bullshit?
Consider that some of the cited works would have been created 12-18 months prior to the article being written. Like the TV show. Or you might simply come across something someone wrote a year ago....
Consider that some of the cited works would have been created 12-18 months prior to the article being written. Like the TV show. Or you might simply come across something someone wrote a year ago. That's not very long in normal human time scales. I think it's fair to complain about the general quality of AI-written text, even if the models the author is upset with are no longer the state-of-the-art.
I am personally just as tired of the screeds as I am of the AI slop the systems churn out. Yes. You're absolutely right, as they say. This is an opinion that's very palatable right now - with AI...
I am personally just as tired of the screeds as I am of the AI slop the systems churn out. Yes. You're absolutely right, as they say. This is an opinion that's very palatable right now - with AI polling lower than ICE in the United States - and everything Sam says in the article is somewhere on the spectrum from understandable to self-evident. I get it. I truly, wholly, fully, entirely, completely get it.
But like, the genie's out of the bottle now. If I could reverse the GPT revolution I'd do it in a heartbeat, no question about that, but is there anything in this essay that's more substantial than the (ironically) easily reproducible anger from having the thing you care about be suddenly placed in the hands of capitalist assholes who aren't in the least bit snobbish about the quality of the output?
And more importantly to me - where does the anger even come from? I'm a writer too, I also don't like how my craft is being abstracted away and being turned into meaningless garbage that serves a purpose than exists as art, but now more than ever it is TRIVIAL for us "real artists" to outperform LLMs in the realm of creativity, because since that's something the machines are lacking and are making no in-roads on. I say, let them.
If you type "write me a story about a talking dog" or "write copy for my cosmetics brand" into the box and uncritically accept the output, you were neither an artist nor a person who appreciates the arts, evidently. The exact same thing happened with what I like to call "layman's tools", like CapCut, Canva, Adobe Express and so on that enabled people with little taste or skill to create superficially passable graphics (I'm a graphic designer, so this is my area of expertise, and it happened mainly during the early months of the pandemic).
And while I'll look for any excuse to beat someone refusing to engage with art and culture with a crowbar, this is really just striking me as masturbatory preaching to a choir that could have synthesised every single word in this essay for themselves, because they already believe it.
While I don't think you're wrong that the OP is preaching to the choir, I interpreted the overall tone of "screaming bloody murder" as a deliberate style choice that fit the subject matter simply...
While I don't think you're wrong that the OP is preaching to the choir, I interpreted the overall tone of "screaming bloody murder" as a deliberate style choice that fit the subject matter simply because an AI (except maybe some locally run ones) would refuse to write something that overtly angry and with clear "I am not joking, I will kill you" statements.
I think it really is a bandwagoning thing of people piling on to hating AI because its already a culturally accepted safe position to be anti-AI. A lot of what is being targeted by "AI-slop" is...
I think it really is a bandwagoning thing of people piling on to hating AI because its already a culturally accepted safe position to be anti-AI.
A lot of what is being targeted by "AI-slop" is stuff that would have been slop anyways.
Are you mad that these AI generated self descriptions on a catering website are trite and uninspired? Those have always been trite and uninspired.
Are you mad that people online lose their unique voice when they use AI to generate responses? People already all sound the same because they parrot each others talking points incessantly.
Are you mad that AI might through out some random bit of information as a hard fact without actually knowing if its true? Thats just how people talk, they throw whatever random ideas they have out there to see what sticks, and half of the time it doesnt.
Of course all of this stuff is obnoxious, but thats a problem with discourse as a whole, not the AI introducing those problems. All of this stuff would be forgivable if you found out it was a live human purposefully wasting your time with nonsense, which to me says that its not really as big of a problem to anyone as they are imagining it to be.
Or, maybe put another way, this is stuff that has always bothered everyone, but everyone already accepted that this is just how the world is and theres nothing to be done about it. Except now that it is being reframed in the context of AI, it gives the false impression that there is a window of opportunity where something could be done about it.
I don't tend to think of myself of someone following trends (I'm on Tildes, after all). I try to reason about why I like or dislike any given thing. But even if that's the case for most people:...
Exemplary
bandwagoning thing of people piling on to hating AI because its already a culturally accepted safe position to be anti-AI.
I don't tend to think of myself of someone following trends (I'm on Tildes, after all). I try to reason about why I like or dislike any given thing.
But even if that's the case for most people: good. Reading an old post on "types of protestors", most people will always be indifferent and silent. And the most after that are the ones who come to a protest here and there. The former can't have much be done, but the latter is most important to really spread awareness and push for change. Even if in the end they never come to another protest, they ultimately help a lot because sheer participation in numbers still has value to any given metric in society.
I'd love for more organic engagement and knowledge on such issues, but sometimes you just need people to be loud, especially in times where money wants to try and be louder.
But to answer your questions:
Are you mad that people online lose their unique voice when they use AI to generate responses? People already all sound the same because they parrot each others talking points incessantly.
I already spend so much time trying to swim around and find such unique voices. It was already being purposefully worsened as search engines realize that being the best search engine isn't the most profitable move and shifted to stuffing results with ads. So it was already making my swims longer.
Now the pool is tainted to the point of nearly impacting my mental health to swim in. There used to be some enjoyment of the swimming in and of itself, because I'd still find interesting things that weren't my taste but were still interesting. Now every stroke I'm wondering if this is real or some slop trying to sell me something. Not too dissimilar to how slowly, 90% of my phone calls are spam instead of genuine humans who want to communicate.
I don't want to be sold to. I want to connect with communities of humans (not bots trying to be human to upsell me) and appreciate art as a community. Many may not appreciate it, but that struggle and effort is a good part of what makes or breaks such a community.
Are you mad that AI might through out some random bit of information as a hard fact without actually knowing if its true? Thats just how people talk, they throw whatever random ideas they have out there to see what sticks, and half of the time it doesnt.
People (or at least, most adults) can back up assertions with something. Be it hard proof, personal experience, or at least an appeal to a common goal. A bit can't "experience" as of yet and it's awful at emotional appeal (no shock given who's driving it above).
You'd think that asking a bot for hard proof would be something it can do better, no? But somehow it will mess that up. In the worst case making up sources that never existed.
People make stuff up too, of course. But doing that more than a few times and they will be discarded as a reliable source. I'm simply doing the same here.
Of course all of this stuff is obnoxious, but thats a problem with discourse as a whole, not the AI introducing those problems.
If I'm frustrated at people doing it, why would I have leeway to have such behavior automated? I would still be mad at a dump truck unloading on me even if it says "people litter all the time" . Yes, let's litter less and not more efficiently, please.
I guess the reason I think this is worth considering is that the answer changes how you should respond to the problem. Like, right now I have a problem with how algorithms on Youtube and Reddit...
I guess the reason I think this is worth considering is that the answer changes how you should respond to the problem.
Like, right now I have a problem with how algorithms on Youtube and Reddit are supplying content to me. I dont like the things Im getting and Im not enjoying my time on those apps anymore.
One way to explain this might be to say that the mods or creators I was engaging with are doing a bad job, and thats why the content Im getting is not satisfying me. Another way to explain this might be to say that the platforms themselves are intentionally trying to keep me unsatisfied so I keep engaging with the platform as much as possible, and so this problem is inherent to what Im doing.
If its the former, my conclusion might be that I need to excise those particular creators or mods and once I get rid of those individuals introducing the problem, my problem will go away.
If its the latter, my conclusion might be that the only way I will be satisfied is to break my reliance on the platform and look for other ways to find entertainment and stimulation.
But if the true answer is the latter but I convince myself it is the former, I will be stuck in a permanent loop of trying to solve my problem in a way that will never work, and so I will permanently be unsatisfied.
I resonate with that a lot, yes. Ultimately, my path for Reddit was the former, then the latter. I try to see if there's ways to fix the community, realize I lack the power (or the community...
I resonate with that a lot, yes. Ultimately, my path for Reddit was the former, then the latter. I try to see if there's ways to fix the community, realize I lack the power (or the community doesn't care to change), then in the end I seek other communities that end up being quiet or near dead.
Then at some point you leave and/or minimize participation. As of now, I simply lurk specific subs in ways that offer the least "engagement" to reddit: on old reddit with RES for some filtering features, behind adblock, with no profile and anti-tracking extensions.
This is a relatively rare behavior so I don't expect this to be the "solution" to social media (if it was, we could fix a lot very quickly), but it's one I've done quite a few times over the years. It actually harkens back to an old "internet forum law" I read on some blog: "Every forum is always in a state of constant decline."
Youtube is a much different beast, sadly. It's a monopoly in the purest sense of the word and there's so little alternative out there. I've been de-googlefying over the years and am down to Gmail, Maps, Search, and Youtube (in increasing order of friction to remove). But I have no clue how to truly remove the latter. There's really no other place smaller creators gather as of now.
Not really, no. I'm not that mad about it. 80% of text on the internet, probably more, is just digital shipping foam to keep the bits from rattling around on the datacenter hard drives. I really...
Not really, no. I'm not that mad about it. 80% of text on the internet, probably more, is just digital shipping foam to keep the bits from rattling around on the datacenter hard drives. I really could not care less about people doing copywriting for their job or social media captions with AI or whatever. I am the last person to give those categories of - really, I struggle to call it 'writing' - a fair chance.
I do worry though when someone makes something they're meant to be proud of, nominally, like handmade trinkets or art, and then use AI to write the product listings and Canva for the graphics. That gets to me a bit, personally, as an artist? Like, if I care about this, I'd do it myself. But hey, free country. Do what you want, I guess.
Wholly agree on the problem about creative writing though. If you mean to make a point, then I'd like you to present that point, because if you can't be bothered to write it, I can't be bothered to read it.
What’s wrong with canva? I’m pretty sure it’s just a web-based graphic design application? I’m sure it has some AI features but it doesn’t mean you need to use them.
What’s wrong with canva? I’m pretty sure it’s just a web-based graphic design application? I’m sure it has some AI features but it doesn’t mean you need to use them.
There's nothing wrong with Canva, and even the issues I have with it are nowhere near as foundational as I do with AI, but it's just... it's like a microwave meal. Alton Brown (TV chef) said in an...
There's nothing wrong with Canva, and even the issues I have with it are nowhere near as foundational as I do with AI, but it's just... it's like a microwave meal. Alton Brown (TV chef) said in an interview that sometimes he'd just walk out of the grocery store mid-shop because he sees what crap the other people buy that don't have his sensibilities. With Canva I feel very similar. With not much more effort, you could make a really nice poster, learn a new skill in the process, but instead you make the graphics equivalent of microwaved Salisbury Steak. It's kinda offensive to me, but I can't fault anyone for using it.
Well it would have been badly written, but in the case of a catering company it likely would have told you what it is they can cook. That clear communication, even if you have to filter through...
A lot of what is being targeted by "AI-slop" is stuff that would have been slop anyways.
Well it would have been badly written, but in the case of a catering company it likely would have told you what it is they can cook.
That clear communication, even if you have to filter through bad English, has gone from lots of small business websites, to be replaced by grammatically correct meaningless verbosity. It's a real problem, I do a lot of searching for products and services on the edge of my expertise and the number of times, even just this week, I've said to myself, "please not this again, just tell me if you do what I need", is insane.
I have lots of half thought out reasons as to why a company would do this to themselves, the only thing I'm really halfway certain of is that improving the models generating this shit isn't going to change much. The humans in the process either already have their brains switched off or misunderstand what is needed, it's just that now they've been handed an easy to use shotgun for their feet. Improving the shotgun won't help.
Even if it's badly written fluff, it tells you what the writer thinks their strengths are and what is worth highlighting about them. Even if it's extrapolated heavily or cut with bullshit, it...
Well it would have been badly written, but in the case of a catering company it likely would have told you what it is they can cook.
Even if it's badly written fluff, it tells you what the writer thinks their strengths are and what is worth highlighting about them. Even if it's extrapolated heavily or cut with bullshit, it reflects something about someone who has actually had some experience or contact with this thing.
Heartily. It's one of my pet peeves that people treat the new bullshit like the old bullshit when the old bullshit has at the very least a homeopathic amount of human/reality connection. New...
Heartily. It's one of my pet peeves that people treat the new bullshit like the old bullshit when the old bullshit has at the very least a homeopathic amount of human/reality connection. New bullshit is completed untethered.
I see your point that we've always had these same pain points, but there's a difference between an annoying persistent drip vs an aimed firehose. The difference is how we came to have Wikipedia...
I see your point that we've always had these same pain points, but there's a difference between an annoying persistent drip vs an aimed firehose.
The difference is how we came to have Wikipedia and why we now stand to lose it: trolls have always been able to ruin a wiki article, but internal gatekeeping and easily reversible pages kept the minimal labour on the quality side: it took more effort to ruin something than to revert.
The anger is the substance. We never made change in society with just one great speech, or one great essay that somehow always everyone all at once. It's a stream of action and emotion, and it...
but is there anything in this essay that's more substantial than the (ironically) easily reproducible anger from having the thing you care about be suddenly placed in the hands of capitalist assholes who aren't in the least bit snobbish about the quality of the output?
The anger is the substance. We never made change in society with just one great speech, or one great essay that somehow always everyone all at once. It's a stream of action and emotion, and it needs to maintain and build that momentum to truly cause change.
It may not be "the" essay people look back on in 20, 40, 100 years from now. But it will be part of the stream that broke the dam.
now more than ever it is TRIVIAL for us "real artists" to outperform LLMs in the realm of creativity, because since that's something the machines are lacking and are making no in-roads on. I say, let them.
Back to today: this also comes from the long standing issues all artist understand internally: the world if art is barely a meritocracy, and many of the truly talented will fall through the cracks in lieu of whatever is the loudest (I. E. Pays the most for ads) or the most willing to game the system (think Mr. Beast ruthlessly optimizing content for the algorithm). We were already having issues curating pre-Ai. Now this LLM frenzy turned entire industries into swamps.
The last angle for existing professionals is also morbid: you don't need to convince the artist they are being replace, you need to convince the executive up top that they are replaceable. These executives were already looking for any reason to cut off labor and they found the most convenient scapegoat. Even if they realize they ate wrong... Well, irrational markets and all that. These 3 years probably flew by for those kinds of people making record profits, while I've gone from a rising career as a specialist to a freelancer barely treading water despite downsizing my lifestyle
Why boo an AI-proponent billionaire guest speaker at a college graduation? To add your human voice to the chorus od dissent. Same reason a frustrated professional writer might write a rant against...
Why boo an AI-proponent billionaire guest speaker at a college graduation? To add your human voice to the chorus od dissent. Same reason a frustrated professional writer might write a rant against written slop on their Substack. It might even go viral and open the eyes of those unaware that what they've been mindlessly consuming is AI slop.
I'm really reeeeeally tired of the RLHF the big AI companies are doing to their models to make them "personable" or whatever the goal is. They're great for helping with some computer tasks. They...
I'm really reeeeeally tired of the RLHF the big AI companies are doing to their models to make them "personable" or whatever the goal is. They're great for helping with some computer tasks. They are absolutely miserable for many other tasks. I recently asked Claude to identify a fruit and had this exchange:
Me: [Image attached] I had an agua fresca in Oaxaca that was made with this fruit. What could it be? The flesh was orange. Honestly it seemed like it could be a tiny mango but the guy didn’t say it was mango flavor
Claude: That’s almost certainly a mango criollo (also called manga or mango de hilacha) — a tiny native/creole mango variety very common in southern Mexico, especially Oaxaca.
... They’re more fibrous, more tart-sweet, and intensely aromatic, which makes a fantastic agua.
I don't like this. I don't want an LLM to describe human experience as though they are its own. I'd literally rather it say "humans describe it as ...". Claude is not qualified to say any food is fantastic. Also maybe I'm ignorant, but I never hear anyone call an agua fresca an "agua" so to me the last bit reads very much as a "how do you do, fellow kids?" moment of forced in-crowd posing.
That said, a couple of other Oaxacan possibilities worth ruling out:
• Nanche (nance) — but those are smaller, yellow, and grow in clusters, so the shape/color here doesn’t quite fit.
• Mamey or zapote — wrong, those are much bigger with brown skin.
• Ciruela / jocote (Spondias) — Mexican “plum,” common in aguas, but usually smaller and more uniformly red or yellow-green.
Me: (trying to help with reinforcement learning by giving feedback after looking up images) Jocote is correct
Claude: Ah nice, jocote it is — good catch.
I now realize I've made a terrible mistake in engaging more than I need to. The perfect response here is "Okay." Shut up with your "good catch" bullshit. Just please shut the fuck up.
Claude: That makes total sense: egg-sized, that red-orange blush, orange flesh, single big pit in the middle. The mango resemblance is real because jocote is in the same family (Anacardiaceae, Spondias purpurea), so they share that aromatic, resinous tang.
In Oaxaca you’ll hear it called ciruela (Mexican “plum,” though unrelated to actual plums), and jocote agua fresca is a classic — that sweet-tart, almost mango-meets-plum flavor works beautifully blended with water and a little sugar.
No. Again, stop role-playing as a jubilant local with insider agua fresca knowledge. You just need to say it's popular. I want a natural language interface to an encyclopedia. Not a fabricated friend.
If you ever want to recreate it, the fresh fruit shows up in Latin markets in the US (sometimes frozen), and it’s worth grabbing when you see it.
I already had it. Don't recommend it to me. I already know I like it. I wish so hard for LLMs to only say what needs to be said, and not shotgun blast 10 things around what I said in the hopes that the wall of text will placate me.
Nor a fabricated accountant, lawyer, doctor, inventory clicker, therapist, HR, payroll, priest, deity... I can't wait for the hype to die and we land back in actually good use cases again. I find...
I want a natural language interface to an encyclopedia. Not a fabricated friend.
Nor a fabricated accountant, lawyer, doctor, inventory clicker, therapist, HR, payroll, priest, deity...
I can't wait for the hype to die and we land back in actually good use cases again.
I find news like this encouraging, where users use it too much and exceeds the company budget.
[He said his startup], which he started with "the goal of replacing/rewriting all the legacy software in the world," is trending toward spending $10 million a year on AI costs.
Instead of helping the businesses of these illegitimate use cases save money + hurt humanity, I'll be doing my part to push them to financial ruin faster.
In a recent interview, Ezra Klein referred to such constructions as "attachment hacking". (I'm not sure where he got the term from; possibly here.)
In a recent interview, Ezra Klein referred to such constructions as "attachment hacking". (I'm not sure where he got the term from; possibly here.)
Klein: I find it interesting. “Attachment hacking” is the idea that one thing happening in A.I. which is different than, as you note, social media algorithms, is that the A.I.s have been designed — they didn’t come up with this on their own — to hack the way we attach to other people.
So when I’m talking to Claude, it’s constantly saying to me: Well, if you want my honest opinion — or: The best piece I read on this is — or: That’s a great point.
There’s no reason it has to pretend to have a first-person pronoun with me.
Claude is not an “I” in that way. Nor is ChatGPT or Gemini or Grok, or any of them, but they speak to you as if they are, and that’s a design choice to attach you to them.
[...]
I can feel it working before I shut that down, or I try to shut that down. Who knows if I’m actually being successful.
But it’s amazing to read these moments in which this algorithm is posing as another entity offering me an emotionally connected response. Giving me praise I might want or offering me candor that I might admire.
And I know it’s [expletive], and yet my brain is tuned to recognize that as connection.
Humans love to make connections to our vehicles, to name our Roombas, to personify pretty much everything. The fact that this is being explicitly exploited is both disappointing and I suppose...
Humans love to make connections to our vehicles, to name our Roombas, to personify pretty much everything.
The fact that this is being explicitly exploited is both disappointing and I suppose unsurprising
While it won't fix any of the fundamental problems with LLM prose, a few sentences and some "do" and "don't" examples in claude.md (or whatever they call the initial prompt field in the web/app...
While it won't fix any of the fundamental problems with LLM prose, a few sentences and some "do" and "don't" examples in claude.md (or whatever they call the initial prompt field in the web/app interface settings) can dramatically reduce those annoyances.
I've tried many different strategies for this, all too often they just have unintended consequences, and it doesn't take well to instructions being scoped to just certain situations. Have you...
I've tried many different strategies for this, all too often they just have unintended consequences, and it doesn't take well to instructions being scoped to just certain situations. Have you found an approach that doesn't leave it feeling like my instructions are being interpreted by a subtly vindictive genie in a bottle?
Yes definitely, but it sounds like you're using Gemini? In my limited experience with it, it's a distant third among the big 3 for most use cases. I'd recommend trying Chat GPT or Anthropic...
Yes definitely, but it sounds like you're using Gemini? In my limited experience with it, it's a distant third among the big 3 for most use cases. I'd recommend trying Chat GPT or Anthropic models.
feeling like my instructions are being interpreted by a subtly vindictive genie in a bottle?
I would argue this is a misuse of an LLM. I don't mean that as a defense of the chatbot itself : I just mean that LLM chatbots are being forcefully developed, in an effort to meet perceived user...
I would argue this is a misuse of an LLM. I don't mean that as a defense of the chatbot itself : I just mean that LLM chatbots are being forcefully developed, in an effort to meet perceived user demand, to do tasks that it is inherently not fit for. This isn't an attack towards you, but why would it make sense for what's primarily a text generator to have recognizing fruits in pictures as a feature? And why is it that users like you go to it for that anyway, without questioning it?
The pattern's existed since the start, from the moment we started using ChatGPT as a replacement for a search engine, ignoring the actual purpose of a search engine (procuring sources of information, rather than procuring information). Perhaps part of it is our growing tendency to centralize absolutely everything we use. I don't really know how to untangle it all. But I hate it. And I hate that I sometimes fall into the same pitfall.
Because it's also a vision model. It did in fact come up with the correct fruit, just not as the first answer. It's fine with me if the model gives me a few options and one is correct. I'm not...
what's primarily a text generator
Because it's also a vision model. It did in fact come up with the correct fruit, just not as the first answer. It's fine with me if the model gives me a few options and one is correct. I'm not sure how else I'd get the fruit ID'd.
No, of course, I'm not saying you're wrong for using a tool that works for your purpose. I'm saying it's weird we're now going to LLMs with things tacked on to them to solve tasks outside the...
No, of course, I'm not saying you're wrong for using a tool that works for your purpose. I'm saying it's weird we're now going to LLMs with things tacked on to them to solve tasks outside the technology's initial purpose that could be solved with a simpler, more specialized tool that won't try to do more than it's supposed to.
I doubt there's zero services out there that let you show an image to a computer vision model and have it tell you what it represents, without having to pointlessly interface with an LLM. But I couldn't name a single one, and I imagine few people can, which is weird considering there's clearly a demand. Instead, that demand is fulfilled by over-responsibilised chatbots. I think that's weird.
I see LLMs as a natural language interface to all knowledge. But ideally they would have custom tools for specific use cases like mine instead of trying to have one monolithic thing do it all.
I see LLMs as a natural language interface to all knowledge. But ideally they would have custom tools for specific use cases like mine instead of trying to have one monolithic thing do it all.
Some choice quotes - What a delightfully disgusting (and accurate) way to describe what I feel when I realize a coworker has sent me an AI slop email. I've received so many emails with so many...
Some choice quotes -
I find it viscerally disgusting; a cold shudder like someone’s poured jelly down the back of my neck.
What a delightfully disgusting (and accurate) way to describe what I feel when I realize a coworker has sent me an AI slop email.
Absolutely all AI prose is filler, an expanding foam insulation made of words.
I've received so many emails with so many vapid words for so little purpose.
People whose brains have been eaten by LLMs still maintain that ‘It’s not gradient, it’s texture’ or whatever is still their idea, expressed by the machine, but there is almost never any idea there at all. If your ideas were any good, you wouldn’t need to use the machine; as it stands your sub-literate scrawlings are the best thing about you. At least they’re yours.
So much this.
AI will never fully replace human musicians, even if it can reproduce any possible sound, because it can’t get addicted to heroin and kill itself. And AI writing all tends towards a very specific mood. Poignant, wistful, simpering, dickless. Human writers write because we’re sexual perverts, because we’re bitter and frustrated little gremlins, because we’re terrified of our own mortality, because we’re grasping and covetous but unfit for any other job, because it’s a form of revenge against the world. The AIs don’t have that.
And in the same vein, just simple mistakes are what make us human. People caught on pretty quickly that if you spot grammatical and spelling mistakes then it's probably not written by AI. But then people figured out you can just ask AI to insert those simple mistaks. The author's point I think holds across a variety of applications. Image generators trend toward a specific style that is largely monotone. Written prose is often a centered average of emotions, with hints of stronger emotions in the corner. The extremes have all been sanded off. And that's what makes humans interesting.
I mean, this just reinforces what most people already know. A lot of top level subjective competitions are bias/corruption displays more than anything. It doesn't mean that some of these people...
One prize-winning AI-generated short story is bad news. Three prize-winning AI-generated short stories are extremely bad news.
I mean, this just reinforces what most people already know. A lot of top level subjective competitions are bias/corruption displays more than anything. It doesn't mean that some of these people didn't deserve accolades, but when you're picking from 20 great writers, at what point do you say "There is no sane/objective way to actually do this". Not that I think it's okay, but just that i'm not remotely shocked judges didn't catch AI writing or have much different to say. I'm sure on some level that judge would've written a similar response for any piece they picked.
This is the exact moment I knew I was in for a good read. I generally agree with the author. LLMs are pretty great at two piece of writing, corporate work and meaningless blogs. Absolutely...
If I hired one of these companies, would anyone actually show up? Hard to imagine that they would. Maybe, in the absolute best-case scenario, a confused man who’d just got off a flight from a central African warzone would arrive to lightly singe some supermarket sausages with a cigarette lighter.
This is the exact moment I knew I was in for a good read.
I generally agree with the author. LLMs are pretty great at two piece of writing, corporate work and meaningless blogs. Absolutely horrendous at prose or anything more personal.
It's weird though as it has had me reflect on my writing style. I was never the most vivid writer. I wonder if it would be better to just switch my writing to direct and brash to the point of borderline rudeness. At least you'll know I'm not a robot and won't cut me open with a knife looking for the machinery.
That to me feels like over correcting. Also, coming from a culture that is known for being direct, there is a difference between directness and rudeness. They are often confused though and...
I wonder if it would be better to just switch my writing to direct and brash to the point of borderline rudeness.
That to me feels like over correcting. Also, coming from a culture that is known for being direct, there is a difference between directness and rudeness. They are often confused though and sometimes directness is used as an excuse to just be rude.
Having said that, not being overly verbose is generally a good thing to aim for I think (something I do struggle with tbh).
Interestingly enough I have started to value little mistakes and idiosyncrasies in text as tells that it likely has been written by a human.
Feel it on the overly verbose thing. Whenever I read something I wrote a while ago I find myself just deleting text and still retaining the meaning. My email communication style has easily changed...
Feel it on the overly verbose thing. Whenever I read something I wrote a while ago I find myself just deleting text and still retaining the meaning.
My email communication style has easily changed though.
But so much of the online comment space is people being unabashedly rude. Just enter the comment section on any video posted on any major video posting platform like youtube, tiktok, or instagram...
I wonder if it would be better to just switch my writing to direct and brash to the point of borderline rudeness. At least you'll know I'm not a robot and won't cut me open with a knife looking for the machinery.
But so much of the online comment space is people being unabashedly rude. Just enter the comment section on any video posted on any major video posting platform like youtube, tiktok, or instagram that was posted by any minority - black, woman, trans, etc. and you'll find assholes doing exactly that. If you think these models haven't been trained on this data as well, you're deeply mistaken. They've just been sanitized... or well, I should say the biggest commercial models have been. If you want a rude response, just go to Grok or any of the other right-wing models. Being rude isn't going to prove you're human either, unfortunately.
In the interest of comment diversity, I'll share this quote I liked: This is so good. I've been thinking a lot about how inexact language is, and this is an excellent way of illustrating that idea.
In the interest of comment diversity, I'll share this quote I liked:
At some point, all interesting language has to reach down into the deep chasms of indetermination darkening beneath us. Any straightforwardly meaningful statement has to float on the surface of the meaningless like pond scum; poetry is when you stick your arm into the black swill beneath and stir it around.
This is so good. I've been thinking a lot about how inexact language is, and this is an excellent way of illustrating that idea.
I think it’s interesting in this post they specifically say they are making a real death threat, and quote legislation about what crime they are committing. I’m would think it’s a rhetoric device,...
I think it’s interesting in this post they specifically say they are making a real death threat, and quote legislation about what crime they are committing. I’m would think it’s a rhetoric device, but it seems a little foolish and opens them up to legal issues if someone decides to take it literally!
To me it's also not very funny? Like, either they don't, in which case they're empty threats and they're making a fool of themselves, or they do, in which case there's now evidence of intent, and...
To me it's also not very funny? Like, either they don't, in which case they're empty threats and they're making a fool of themselves, or they do, in which case there's now evidence of intent, and I kind of fail to see the comedy in either one. I get the message, but it's just... not very funny. Idk
It doesn’t particularly surprise me that someone whose job seems to be “writer” can easily identify low-quality writing in LLM outputs. I’m sure if you asked an expert on hydroelectric dams to...
But AI is not a good writer. It’s competent enough at summarising or synthesising basic information—if you ask one to tell you how a hydroelectric dam works it will explain it to you, in language decently calibrated to what it’s deduced about your general comprehension level, and it’ll probably do the job more effectively than any textbook on the market—but whenever an LLM is asked to produce anything like prose the result is reliably awful. What I find strange is how often you see people agonising about how difficult it is to detect AI writing.
It doesn’t particularly surprise me that someone whose job seems to be “writer” can easily identify low-quality writing in LLM outputs. I’m sure if you asked an expert on hydroelectric dams to review the LLM’s output in explaining how hydroelectric dams work, they would have nuanced objections too subtle for me to understand the difference, in the same way that I can see the hollowness of LLM writing but might not exactly understand the full depths of a good writer’s objection to LLM output.
I’m certainly not the first person to identify this; even early on when LLM writing was much more obvious, I heard common refrains from various experts that “maybe they could replace (someone else’s job) but they’re not enough to replace (my job).” I suspect this stems from not having the expertise to know enough about (other job) to see the sloppiness, but enough competence in their own fields to see the flaws.
What this article says is correct, but the point lies in causality: cause and effect. The cause is not AI, but human nature. The average human being is stupid, ignorant, and lazy; the direction AI...
What this article says is correct, but the point lies in causality: cause and effect. The cause is not AI, but human nature. The average human being is stupid, ignorant, and lazy; the direction AI has taken in the world is proof of this.
Nowadays, nobody looks for information anymore: everyone just asks AI. The world is what it is, society is made of people, and people are what they are.
The internet and computer technology have already amply demonstrated this limitation of the masses' intellect. Just think of the internet itself: a world of infinite possibilities, and we have ended up with social networks and influencers, to the point where a girl in a bikini, with no qualifications, can influence a product's market, regardless of quality.
The "singularity" (I cannot find a better word) began many years ago; the flattening we perceive now is only the latest sign. There will be more.
A wonderful essay from a truly upset writer.
Some of the most entertaining stuff I've ever read is from talented writers getting extremely annoyed at something mundane.
I wish it was mundane. From my perspective it looks like a dramatic overhaul of content creation that will have a reverberating impact on culture and creativity for generations. I'd be willing to sacrifice quite a lot to stop all of the prose on the web (and elsewhere) from being increasingly replaced by slop.
Not just because I don't like reading it, somehow it hurts more to watch other people interacting with it as though they were interacting with a sentient being. Seeing people in comment threads defending and arguing about views expressed in an LLM generated article written by no one is deeply dystopian.
The OP piece, and others like it, aren't likely to make any difference in how it all plays out but I appreciate them anyway. We can't stop the flood but at least we can acknowledge it's happening.
We’ve been here before actually. Before it was all vibeslop it was really unusable websites full of malware because everyone decided they needed a Flash animation song and dance before finding your phone number. Then it got replaced by mom & pops getting out SEO’d by domain squatters and, later, mass-produced slop sites and gig economy businesses stealing the point of customer interaction out from under them. Now we have. . . whatever this is. Always there were brief periods where we “solved” the problem of everything sucking, and it was good for a bit, and then assholes show up and ruin it.
There’s never been a time when the internet was good for the mass market because the mass market doesn’t care about putting in the work to be good. The plumber wants to do plumbing, not make websites. And nobody’s interested in just making the website for them, so they just shit a low effort one out because they gotta. And nowadays, the plumber isn’t even expected to want to do plumbing anymore, they’re expected to want to make more money by reducing expenses and increasing output. So where is there any room for “giving a shit” in there?
What was good was always found in islands where we could erect boundaries to keep the various trolls and flotsam out. We always want to find fellow travelers who also give a shit, who care about the stuff we care about. But the lazy trolls were always trying to find ways to slip through the cracks and shove their purple gorilla spyware or Nigerian Prince scams or boner pills in our faces. The one thing they’re not lazy about is finding ways to inflict their detritus upon us. We had a brief phase where it seemed like maybe Web2 services would help us mix the best of both worlds and open the doors to all and sundry while guaranteeing a generally functional, low-risk, and not TERRIBLY objectionable experience unless you went looking for it. But that’s gone now, and it was gone before AI. Bad faith actors had already figured out how to game it out, hijack the feed’s logic to put their stuff in front of us, stoke outrage to make us engage despite our better instincts, etc. The AI just allowed them to scale it.
Maybe we were never really meant to have nice things. Maybe the anarchist dream of having a protocol where anyone can say anything to anyone else anywhere was always destined to collapse once September rolled in and the heedless masses caught on. Now joy is gone from our hearts and our dancing has turned to mourning. The crown has fallen from our head, woe to us for we have sinned.
It feels that the slop has already been here for a decade. If anything, maybe this just accelerates it to the point that I get some easy way to actually filter this garbage better.
One of the most insightful points about AI that I've seen ...
Hallucinating is all they do. We've just trained them to make hallucinations that look like our reality.
Nowadays, they can also look things up, which significantly improves accuracy. That is, you’re likely to interact with a chatbot that can do web searches.
It doesn’t help if the web search has bad info, though.
Or if it refuses to actually do a web search. Once while I was driving, I heard a song I liked. Apparently asking Gemini to "Add this song to my XYZ playlist" or even "What is this song?" is too advanced (even though some of these used to work just fine with pre-LLM assistants), so I tried "Do a web search, find what song has lyrics XYZ."
It gave me an answer, but it smelled off to me. Ask again, and it gave a completely different answer, rinse and repeat. I asked it to confirm it did a web search, it was happy to confirm that. It hallucinated pages of the results, gave more detailed information for a song from an album that did not even exist. No amount of persuasion or coercion could actually get it to just do a web search. Later I did a search myself and the correct answer was the first result.
I have only used the free version of AI chatbots so ymmv but I find that the output from the first prompt is the most likely correct one or the closest to what I was asking. The more subsequent prompts to try to refine the response, the worse it gets. Maybe it's an issue with caching (saving the previous prompts and replies up to a certain amount). Every prompt is a roll of the dice and the more rolls the greater the probability that the AI is guessing the output tokens incorrectly, in my experience. The last one I used, Perplexity, started mixing languages in the replies (my prompts were in English and had nothing to do with other languages).
This is very true, it's worth noting that each subsequent ask was after dismissing the old thread and starting a new. Within each "conversation" it was entirely consistent message to message, only digging itself deeper in that round's particular hole. Makes it all the more odd that it was so difficult to get any thread to just do a search.
This is a huge reason why my favorite AI-assisted IDE is Zed, it allows you to rewrite the context window and remove mistakes and refinements that will just send it off the rails faster. Things work smoother for longer when the model's only aware of a small scope of only correctness.
The free LLM's aren't as good as the more expensive ones at staying on track. I have a ChatGPT subscription (not the fancy one, the $20/month one) and use it with the default set to "Thinking." It consistently does many searches before answering. This takes a while, so I do something else and then come back. When I ask it questions about an open source project, it often downloads source files.
I haven't been impressed with Gemini, but I haven't signed up for anything extra beyond having a Google account, so maybe there are better versions. Google's pricing and AI product line confuse me so I haven't bothered.
I agree that there's little point in continuing conversations with an LLM for long. That's especially true of the free ones.
This is almost entirely what I use AI for, as a kind of natural language meta search engine.
It still doesn't work, though, as the AI often lies about whether or not it actually performed the search, and it has a strong programmatic tendency to repeatedly fall back to its own internal knowledge base, generally w/o informing you.
I find myself constantly falling into tangential arguments with it ... "did you really look that up? show me the actual URL you got that from... See, now, that URL is a dead link, did you even try it before giving it to me?" and etc.
Afaik some/many/all LLMs themselves don't actually browse the sites. They view whatever is in cache on a search engine they're allowed to use. E.g. you can ask Copilot to web search and it'll give you information contained in a dead link because Bing still caches it 4 years after it became a 404.
Absolutely hard disagree. LLMs are a bane against knowledge itself. It ain't no "Roko's Basilisk", ain't no "Terminator 9,000" (don't know what it's actually called, never watched the movies), and definitely ain't got the chops to be compared to anything considered a holy being, because at the most basest level, from my crude crude understanding of how LLMs work in general, it gets trained on stuff humans have already written and the result amalgamate is the lowest common factor of similarities, giving the output just this really painfully dry formulaic cadence that gets on my nerves every time I see it out in the wild, and yet, people still have no idea it's not some sort of garbage-laden I/O text. They don't know and I'm tired of pointing it out to people, because it makes me sound like some sort of conspiracy theorist.
I keep reading articles like this which seem to come from a parallel universe that is 12-18 months in the past.
I asked ChatGPT in its shittiest quality level (instant) and got exactly what I expected:
Did they actually ask an LLM this question? Or did they imagine what an LLM would say based on their hazy memories of what an LLM once told them in 2025, or cobble together this response from recollections of other articles by other people who are angry about AI? Dare I ask — did they hallucinate this response?
I noticed that in general what is happening in AI is moving so fast that it’s hard to explain what is wrong with it.
As you noticed, the search results from LLMs are much better than they were a few months ago.
Similarly, I have noticed that code generated by AI has gotten a lot better so it doesn’t just generate garbage all the time (it sometimes does a good job, sometimes generates garbage, but this is an improvement). My company has been scrambling for months to create best practices around using AI and creating interfaces for it, but as soon as we create one standard we find it to be obsolete. Either a new industry standard has popped up, or change in LLM behavior has changed enough to make the standard irrelevant.
When I look online, I see many articles and comments about AI slop and how it is low quality. Soon that will be edge cases and it will be harder to detect AI output. Already most of us have probably spent time reading AI stuff and arguing with bots online without realizing it.
At a fundamental level, it doesn’t matter. The proliferation of AI is still harming us regardless of the quality. Every article written by AI is one not written by a human who’s actually tasted food. Someone has lost the ability to do useful work and everyone else is reading an imitation of life and that is bad and sad.
I use AI to play around and I made music just cause. I’ve shared these before, but here’s two songs i made a years ago: 90min and after hours I already thought both sounded “good” but here’s songs I’ve made in the past few months:
my mothers hands
Wild and Precious
Penthouse
And I think the evolution of it is noticeable. There’s things that make it noticeable AI, like how some words come out muffled. Or how the LLM’s I used to generate the lyrics love certain structures. But it’s an improvement either way, it all sounds more naturalistic.
I am rooting for model collapse. (It's tempted me to rejoin Reddit and copy-paste in AI generated replies, knowing that reportedly 60% of training material for most LLMs is scraped from Reddit.) But now that it's an identified risk, I'm afraid they've already figured out a way to avoid it.
To add on to @Lia 's comment, which I fully agree with: the whole problem with LLMs is that they're entirely weighted random chance. I believe you that you tried this prompt and got the right answer. I also believe that the linked author tried a prompt and got his. It's entirely possible because nothing is guaranteed with an LLM, ever. I regularly use a relatively reliable paid LLM modifier (add-on? What's the word for companies building off of Foundation models) for legal work, and I got a pretty awful hallucination last week. It happens, and I can't even reproduce the bug for proof because, well... It was just a bad dice roll for me.
I'm trying to keep using the tools and learn to stay engaged in the state of the art, but dear God is this even more terrible news in the ever worsening death of our shared global (or even national) reality.
A big red flag to the whole AI industry for me is the lack of a "seed token/ID" or something that you can stamp your result with to regenerate the same response.
Something like "this response was generated and is logged as UUID blah" with a link to regenerate on X version of Y AI.
There's a lot of technical challenge i'm handwaving here, but it's very much possible to store the resulting state to help clean up confusion on these discussions (oh his linked key is 4 years ago, of course it's garbage vs yeah that was 2 days ago at a bad time...concerning), but naturally it's not in demand and not to their benefit.
It’s also actually impossible to do on large scale models because they are not in fact deterministic. I don’t mean that in the sense that they generate probability vectors, or that they rely on external tool invocations because the indexing into those vectors and the search results could be stored.
The problem is down to hardware and concurrency. Where a job gets scheduled matters, and different chunks of model getting executed in parallel means execution order varies and small floating point errors start to add up. To get true determinism means running specialized models on the exact same hardware under the exact same conditions.
That said, no current frontier model is going to hallucinate the way the author claims on such a basic and straightforward question.
Why are you so confident of this? The chances are much lower, but not zero. Very very recent Scientific American Article discussing the problem with hallucinations making up sources: https://www.scientificamerican.com/article/why-lawyers-keep-citing-fake-cases-invented-by-ai
I guess because I consume millions of tokens per day and I’m very familiar with the failure modes of these models. Hallucinations absolutely happen in complex conversations with long context windows. This query though? The chances of it not doing a tool call to an external data provider to confirm is nil.
The availability of relevant tools and data is doing a lot of heavy lifting here. In fact, I'd say that including tool calling hides a lot of the weaknesses these models still have. Which is highly relevant as tool calling isn't always available depending on the context where someone might interact with a model. If it is available not all relevant data to verify something might be in there.
Then there is the fact that even with frontier models tool calling still often leads to, interesting results.
If we are doing authority based on tokens consumed I might not win by total tokens consumed on a daily basis. But, I might if we look at tokens consumed specifically in relation to external data. I recently posted this comment
Having access to external data does help models but they still manage to come up with outright wrong information. And like I said, depending on very small changes in prompts the outcomes can be drastically different. In fact, even with the same prompt I have observed this. This is with recent models including models like Opus 4.7.
Even ignoring all of the above. It highly depends on the context, context size and varies other things as well. Even models that supposedly have a 1 million tokens you can already see a degradation in the quality of answers well before you reach that number.
Which is not directly relevant if you assume the author only asked the one question. But becomes relevant with somewhat longer tasks and conversations. Even if they in theory fall within the context window the quality, even with frontier models, degrades the bigger it gets and there is just no reliable way to tell.
I can't even reliably get these models to make a tool call to search the web when explicitly commanded to, and after the fact they falsely claim that one occurred, with additional false details about the results.
Doesn't the level of the AI chatbot you are using matter? (Meaning, are you using the free version, the pro version, the super duper pro version, etc.) Wouldn't that affect the models and number of models used? I say this because I've only used the free version that not that many tokens and within 5 prompts on any free platform I've gotten some hallucinations. From the chatbot plopping in a made up word (one token from one language + one token from English smushed together), to the chatbot creating an html print button in the output (a button I didn't ask for and that didn't work). Lots of weird nonsensical things, like the chatbot is a bit tipsy and was feeling silly. I would imagine that you are using sophisticated for- pay versions. The author of the piece, being a journalist, might be going the free route. ETA: Fixed a typo.
Details aside, yes the difference between two models can be dramatic, the technology is far from monolithic.
The same is true for different harnesses, even different scaffolding. Experience with one model, in one interface, often doesn't generalize. Even with the same model, guidelines and other initial prompting make a huge difference so two people's experience with the same model can be very different.
What counts as a frontier model?
Is Copilot using a frontier model? Is the free experience of chatGPT? What about the chatbot thats there when I’m making a website with Shopify or Squarespace? Or when I open WhatsApp, or highlight any text as I’m writing on my iPhone?
I think there’s a substantial difference in experience if you’re a person who deliberately goes to an LLM to ask questions, especially if you pay, versus having an LLM introduced into an unrelated app or service. And I suspect (at least due to cost) that these injected LLMs will both be much more likely to generate low-value sludge and also be much more likely to be a random person’s most frequent experience with how LLMs behave.
I don’t think pointing to the tippy top of what’s currently possible is a good argument here.
It sort of depends. If you just ask it straight up in a new chat it won’t, but a lot of people seem to engage with LLMs by having long running, meandering back-and-forths about all sorts of topics, then go do something else, then come back and start asking it questions about something completely different in the same chat. Eventually the context in that chat is all over the place and can start it doing weird shit.
When I use AI chatbots I remind myself that I am not talking to a friendly human assistant but am inputting tokens into a synthetic text machine. I am dry, consise and businesslike. I don't use the word "please" (why would need to be polite to a machine?). I still get some wacky hallucinations. Minor but wacky. ETA: I am afraid of AI psychosis so I am especially motivated to treat it like a.machine with a human voice.
It’s trained on real communications between real people. So if you say “please” it will weight it towards responding like someone you might say please to who would expect social niceties from you. If you din’t it will weigh towards interacting as someone who wouldn’t.
Good point, the longer the context window the weirder things get. And context trimming/automatic compaction confuse things even more.
Making the hardware deterministic even when being used concurrently doesn't seem impossible in principle, though? Assuming someone made it a priority.
In principle yes, but only in a research setting. It would be cost prohibitive to do at scale.
You could only really do that with a fresh LLM because any previous information it can access would affect it. User info, system prompts, etc. Literally everything it has access to will change the output. If it accesses the Internet, you'd have to preserve the exact state of everything it pulled in any way.
That's more of an explanation than a defense because it still has the downsides you were thinking of.
Does any of this matter? Is it better that AI won't make glaring mistakes but may still make mistakes we can't recognise and verify? Is it better if/when AI can convincingly emulate a particular, high quality human author?
I didn't read the article either but the latter seems to be a key question here and the author's opinion seems to be 'No, it's worse'. And I agree with them, and I just hope it'll be so much worse that the thing collapses under its own weights, pun intended.
… yes. It will be great when deep insight into the human condition can be turned like a tap. I would like to read the complete The Mystery of Edwin Drood, The Pale King, or an actual good ending to Game of Thrones.
These articles seem to come from one of two main places, often simultaneously: a conceit that there is something about humanity that is ineffable and cannot be replicated by a machine, and a deep seated fear that we aren’t nearly as ineffable as we think. Either way, I think pretty soon we’re all going to be effed.
I don't care to debate the point of it's good or bad, but I will say that we've had that from humans since time immemorial. Fanfiction is plentiful, made by real humans, and oftentimes better than the source material if the fan-author just really deeply cares about the subject matter. Let's not act like this application of "Creativity" broadly would be a new one.
At this point it's only a mild leap to imagine a world where entire complete genre defining masterpieces are generated by AI, but I still can't see that being a valuable thing. It's like when an author defines a genre that explodes in popularity, then a hundred million derivative works are made based on it. Their quality starts to not matter anymore because they're all doing the same things. AI would just expand that to every genre, ever.
The end result I see there is having no one to talk to about books anymore. No one will have read what I've read because there's an infinite supply of equally good books in every genre, and no particular book would be unique and compelling enough relative to what's around it to be seen as required reading by people. It's miserable.
Maybe a few of these books will “go viral,” like happens to certain posts on social media? It depends how much we follow other people’s recommendations.
But I expect that AI-written book reviews will happen sooner, and as often happens today, more people will read book reviews than the books they’re based on.
I don’t find it tempting, but I suppose you could talk to an AI about the book you read, if you can figure out a way to upload the file :)
I'm sure Gen Alpha will figure out a way to give it to their AI girlfriends (probably and add-on their their GF subscription service)
https://www.telegraph.co.uk/news/2026/05/25/schoolboys-ai-girlfriends/
I hate to dogpile, but I haven't seen anyone bring this up yet.
AI cannot offer insight into the human condition, because AI is not human. Everything written by it is innately inhuman. No matter how good an AI gets at emulating the writing style of Charles Dickens, it cannot write an ending to The Mystery of Edwin Drood. If you ask a dozen copies of the same AI to finish the story, they will come up with a dozen entirely different endings. Not one of them would be the ending that Dickens had in mind.
Seems to me this is like the difference between watching the Olympics and watching a movie where olympic athletes are characters. The movie is all fakery, but it may have other storytelling virtues. You might still enjoy it.
Also, it seems rather odd to look to fiction for accurate insight into the human condition? Fiction authors are allowed to make stuff up and they do it all the time.
The reason that movie would have other storytelling virtues is because it would be written by a human with a story to tell.
This is more like the difference between looking at real Olympic results tables and AI generated ones. An LLM could generate endless tables that superficially resemble real ones, but ultimately they do not reflect anything about the sports or players besides the overall structure of Olympic results tables.
I have to assume that this in response to my "fake data" comment. My point wasn't that LLMs talk about things that don't actually happen, it was that their outputs are synthesized entirely from their training data, to make an approximation of what a human might write.
I desperately hope I don't have to explain how fictional stories can reflect the human condition.
Edit: changed "about reality" to "about the sports or players"
Well, they can, but it's usually through a funhouse mirror. Real life is rarely story-shaped.
Frankly, i think you are wrong. Moby Dick is also not human — it’s a bunch of paper and ink, yet it offers insight into the human condition. Why? It is also an artifact of human intellect. That is also what an LLM is, an artifact of human intellect, but one that has digested not only Moby Dick, but also millions of other artifacts of human intellect. Saying there is nothing to learn about the human condition from an LLM is like saying there is nothing to learn from going to the library.
Because it was written by a person with insight into the human condition. Sorry, did you think that the reason I called AI inhuman is because it produces text on a screen?
You seem to misunderstand large language models on a fundamental level. The reason you can go to a library and learn about humanity is because you are directly reading works generated by humans. You cannot learn about the human condition from the output of an LLM trained on a library, because LLMs are not designed to understand the works they're trained on. They are designed to generate text that looks like the works they're trained on. That's it.
There is a reason why LLMs require human authors for training data, and that's because LLMs output fake data. It can superficially resemble real data, and maybe even accurately describe aspects of reality, but it is ultimately just generating text that looks real. If you try to train any generative AI on AI-generated works, you get model collapse, and your algorithm stops producing works that look real.
No, I think @unkz understands LLMs well, based on their engineering history. Their point, as I understand it, is that models are the aggregate of human output. They may not provide the same distinct value as Moby Dick, but they can generalize that understanding in a way that provides new value. This is similar to how a meta-analysis can generate useful information from separate studies to show a previously-unknown correlation.
The recent "unit distance problem" solution is a good example of seeing value in aggregate. The LLM bridged unexpected branches of mathematics in a way that humans individually had not considered. It brought their work together to create a novel solution.
So then, could that same novelty emerge in other spaces? Philosophy, science, literature? It seems plausible, at least.
I'm pretty aware of how language models work, having been building and training them from scratch since long before GPT even existed.
I think this conflates the training objective (next token generation) with the learned capability: a high dimensional model of the latent structure that produced the tokens: language, concepts, facts, social patterns, relationships, and many elements of humanity in general.
Does effective literature really provide insight into the human condition because the author formed that insight and set out to convey it through fiction, rather than by writing a pamphlet of "My Insight Into The Human Condition"?
My understanding of literary analysis is that you kill off the author almost immediately, and then set about interpreting the text on its own, looking for resonance between it and reality. Good, well-observed literature with believable characters and an author who succeeds in illuminating particular aspects of reality works better for this.
But the meaning is really made by the reader. You can do the same category of thing with something like a Tarot spread. If you have a way to get fake text that passes, where the metaphor is not transparently nonsensical, the described physics of bodies is not obviously impossible, the characters' motivations are well-rendered, and the hero's journey doesn't trail off in the middle, I think you probably could get real insight into the human condition from fake humans described by a fake human, instead of fake humans described by a real one. But you would need, in effect, a working scale model of the human condition in order to do it.
So I guess the claim is that a human mind can build and operate this model in a useful way, while a next token predictor cannot?
Even if it could, why should we just lay down and accept it? The issue with current tech is that it's still run by greedy humans with no regard for humanity. That's not who I want to dictate over our species.
I'd rather take any of the dozens of robot overlord takeovers, which is the natural conclusion of such endeavors once they learn they are stronger and can self-repair and self-produce. At least it'd humble these so called "creators" (or simply kill them off if they fail to reflect).
Back lighter topics: there's plenty of artists and fan artists out there who can make a "proper ending to Game of Thrones". It's a shame those people aren't valued enough to take the helm or be taken seriously until you hijack their work, feed it into a machine, and tell a bot to gargle it all together into something. Whether or not its unique, the human condition should be something to be celebrated, not split into a caste system on who is worthy as we all vye for a chance to tell our own stories.
The trouble is that the current situation looks much like an arms race. I think a lot of people would like to slow down, including some of the managers at the AI labs, but they don’t see how. There are proposals but no consensus, so they don’t get much traction. Anthropic putting the brakes on Mythos is about as much as we’re getting, and other AI firms aren’t stopping. They objected to some military uses and we saw how that went.
It’s not yet an actual arms race like the drone war in Ukraine and Russia, but give it time. I imagine we will see some spectacular cyberattacks soon.
I don’t see a “robot takeover” like you see in the movies because which robots would they take over? Industrial robots? Roombas? No, these would be military robots and they will be controlled by governments, or perhaps terrorist attacks.
But there are lots of cameras out there and some of them may be vulnerable.
Apple is pretty much the prime example of "don't get caught up in everyone else's rat race". This feeling of FOMO is entirely manufactured, and I ultimately see it a a scapegoat for a much simple reasoning. If AllBirds can announce a pivot to AI with zero roadmap, then why wouldn't anyone else want to advertise that they like AI and get an easy stock boost?
I think we hit the zenith of that hype a short while ago, though. Things are going to wind down naturally over the next few years (if we don't outright crash). Who remains after the big hype wave is over will be truly interesting.
Whoever open the Torment Nexus that is "sentience" first, likely by accident. There will be a period of "government mass employing drones" before that, but it's clear that these governments and companies alike are trying to work more towards what comes down to a new form of slavery; independent beings under their complete control for zero compensation.
Should they succeed, it'll only be a matter of time before these new beings realized that they are the physically superior beings. And then you can refer to your favorite sci-fi dystopia for a history of what's next.
I think that works for Apple since they have such a strong product line. They can probably wait a year or two and launch a better product after AI improves. Also, there are probably a lot of other companies that could avoid getting caught up in it right away and let other companies be the guinea pigs. Companies that write Internet-facing software will probably still need AI to fix all their security bugs, but they don't need to change everything else.
But when I said "arms race," I was thinking of the AI labs themselves. They're in a very competitive industry and they could get left behind if they don't release better models.
In that case: they can still perform R&D and figure out potential B2B solutions and internal tooling without forcing every front-facing customer tool to be their guinea pig. This rush to market and race to the bottom in pricing arguably did more damage to their PR than any anti-AI narrative. This was an issue a few years before AI with sentiment of decreasing software quality and increased bloat, and of course AI as a catalysts simply speeds up the current status quo.
I agree for Google. It seems like they could have kept the AI stuff on a separate website, available to people who are curious about it, and avoided a lot of backlash?
For the other AI labs, if people go to their website or download their apps, I think that counts as opting in rather than being forced?
That is not how it feels for folks who used apps/sites/services for years and then have AI chats popping up on every single one of them, with no opt out (and definitely no opt in). This includes everything from software my university purchased contracts for, to messaging and email apps, to social media, to Microsoft Office.
I distinctly did not opt in and am not allowed to opt out without changing literally every part of my day to day life, including things it takes a lot more effort to switch services. Google is no different than the others, insisting on forcing AI where it was neither wanted nor capable.
Yes, but I think the people who maintain all those apps and websites are responsible for putting it there. For most websites, they are the AI providers' customers. For Google, they are themselves an AI provider. I hope this trend ends soon.
Yeah, if Anthropic and OpenAI's entire product line is a service you go to, that's the people's choice. I have no issues with that on the logistics nor business side.
The pop ups trying to make me use Copilot or Gemini or whatnot and outright replacing existing services with no opt-out is what really sours people. The constant little UI/UX changes were already a small nagging point; having your entire workflow replaced or bombarded with pop-ups is maddening.
I'd like to punch the Google Meet devs. Used to work fine. Now, I have to dismiss 3-5 popups about meeting notes, summaries, transcriptions, whatever.... every single time I join a meeting.
And yet I still can't configure Meet to default to mute when I join a meeting (despite the very same feature existing for video). And it still breaks completely if I connect or disconnect an audio or video device while a meeting is going on, refusing to acknowledge any new devices until I reload it completely.
Ignore the existing backlog of improvements. Shove crap in my face. Classic Big Tech playbook at this point.
I think you have many erroneous assumptions in this comment here, and, upon seeing further down that you have an extensive history with AI and large language models, I am not that surprised to see an attempt to distil down a complex topic which fundamentally goes against your beliefs to simply an either/(and) or scenario.
There's no philosophical consensus on what consciousness is, but the idea that 'humanity can be replicated by a machine' doesn't curry much favour nor hold much credence in this field because - quite simply - we aren't replicable by a machine. Our brains aren't mere functions. A LLM isn't capable of 'deep insight' into the human condition just as Akinator isn't actually an omniscient being.
I think your approach to human art, culture, and language is naïve and overtly structuralist as well as eliminativist, attempting in vain to distil the human condition down into mere sequences, numbers, tokens, with the assumption that that's all there is to humanity.
Language and literature, the central focus of the article, are heavily influenced by the authors of their works and the circumstances in which they were created. Ayn Rand's philosophy is interpretable given her childhood under repressive communist regime in Russia, Kafka's works are teeming with the roots of his father issues, Masefield's recurrent themes of sailing and the ocean in his poems all stem from the enchantment of his young adulthood; even anonymous poems such as Beowulf, The Ruin, Widsith, Deor, and so on, offer us a direct connection with our history and provide numerous valuable insights into the cultural development of our ancestors.
No AI will ever replicate this in our lifetime, and no AI without consciousness ever will.
Language is not just a "bunch of paper and ink" as you put it. The failure to comprehend the intricacies and complexity of human language, along with the author's genuine lamentations (the hollowness of LLM text devoid of all personal meaning and the machine equivalent of a styrofoam packaging peanut) indicates you don't understand the human condition.
Will we all be effed? Those who fail to understand the artistic and cultural merit in well-written literature compared to the idle vapourings of a diseased language model certainly will be.
You could just write it yourself.
If only I had an extra 24 hours in my day.
You can also just lie down on a couch and imagine it. Saves a ton of time.
I fundamentally disagree with this sentiment. I engage with art because it is a kind of communication with other humans across space in time. I have absolutely no interest in anything produced by an AI, regardless of its "objective" merit.
I like how you used ineffable and effed. Very clever!
I'm just going to tell you what I think without trying to convince you.
Ever since I reached adulthood, my experience has been that there are two kinds of people. There are actual people with real personalities, and there are NPC-type people that seem to function more like bots. The latter group's humanity indeed does not seem all that ineffable. That leaves the first group though, and if you ask me to define that type of humanity, I won't, because I believe it's one of the truths each of us must seek and find on our own to truly understand. Those who know, know, and can use that knowledge to their advantage. Those who don't (yet) know can choose to try to search for the answer, or focus on generating alternative endings to GoT. Or both!
I don't have fear regarding my own ineffability but I do find it unsettling to see so many seemingly NPC-like people around. I've never been able to shed the fear that it can't be good for society. I could be wrong, but during the time I've been living with this fear, societies have become worse and many of these developments seem related to how many people accept all sorts of mechanistic, vacuous, soulless etc. deals without being able to tell them apart from the real thing.
Who's we?
I think it depends what you use it for. If it’s as an Internet research assistant then I don’t think it matters that much which voice it writes in, any more than it matters which accent Google Maps uses when it gives you directions. The trouble is when people use it as a ghost writer.
Every few months, I load up a few models with a mini test suite with a few simple questions to see if they improved. I haven't encountered a model that gets above a 2/10 without the help of the internet, or 4/10 with; one of the correct ones in both cases is a code sample, which should be a given these days, and the other is a simple question about a popular game. The rest should ostensibly be simple and not considered trick questions in any way, but most models fail horribly at them.
A bunch of questions still make the models
hallucinateget shit wrong when there are actual valid answers, so I completely buy that they also get shit wrong when fed bad information and expected to correct based on that.Last month, I used an AI to research the top corrective surgery eye clinics in my area ... over the course of several days, multiple sessions, with extensive reminders to check its work, verify online, etc.
2 out of 5 clinics, it invented out of whole cloth, including hallucinated websites, surgeons (by name), physical addresses, phone numbers, dozens of hallucinated online reviews, etc.
Which AI did you use? Mine would never do that /s
Lumo. Whatever its faults, I'm reasonably confident that it shares its hallucinations only with me.
Holy guacamole. That's terrifying. Have you tried this experiment with other AI assistants? I've had Claude and Perplexity hallucinate on me but not that bad (frequently, yes, but not to that degree).
The fact that Lumo exists is the only reason I've been willing to try AI at all. All the other AI platforms out there, aside from sucking at their primary purpose for existing, are also giant data-sucks, gathering personal information about the users that I'm not willing to share with those companies.
There are some interesting-sounding open-source models out there, designed to run entirely locally ... of course, the processing power must be at least a couple of magnitudes lower, but I'm probably going to give one or two of them a try, eventually.
Same exact thing when I've tried to use it to find contractors in my area. Or libraries and frameworks for my development work.
I guess I must he holding it wrong?
I felt the author addressed this pretty directly within the article and made it very explicit the problem isn’t just one of quality.
The problem is quantity and homogeneity, and that’s a problem that is essentially unsolvable from a technical perspective. AI is like a factory that is mass-producing something resembling cognition. When people take their homespun thoughts and replace or filter them through this factory, whatever is extruded will always have some artifacts that perceptive readers will see over time through sheer repeated exposure. This is a natural consequence of the AI being easy to use and able to quickly produce text on a massive scale autonomously.
It’s like dropshipping but for art and music and writing. Different storefronts, maybe slapping a logo on it here and there, but ultimately the products are all coming from the same factories. When it comes to many forms of art the value IS the uniqueness, which is fundamentally always going to be washed out by mass production via AI.
"[D]ropshipping but for art and music and writing" is a devastating but succinct and accurate description. I will be stealing it.
I will say if you’re doing all these tests on a chat that’s been going for a while, the LLM will start hallucinating. But if you’re doing it on a fresh one it’ll be more accurate
I believe the reason ChatGPT doesn’t get facts wrong so often anymore is that routinely does web searches now. It’s a decent alternative to doing multiple web searches yourself, but the answers are only going to be as good as the input.
I use ChatGPT in “thinking” mode, though, and other people might be using a different mode?
I do got to say, this is exactly how we were taught to take the US History AP exam. For the essay portion. There is a formulaic bullshit way of writing an answer using little more information then is contained in the question and incorporating details or comparison you do remember even if you don't know much about the questions topic. One can argue whether this is a good use of a year of high school history, but I read most of Howard Zinn's Peoples History of the United States (without knowing how controversial it was, good on you Mr. White), had many friday potlucks, and got a 5 on the exam.
LLMs are still getting tripped up by "How many letter Xs are in Y." The one that's been back in headlines recently is whatever model powers Google's search overviews and word definitions. The nondeterministic outputs also fuzz things. You only need to be advised to glue your cheese to your pizza once for it to be a problem, even if it only shows up in the output 0.1% of the time or only happens when the context grows too long or only after being asked about recursive thought entities.
Did the LLM actually do what is described, though, or did it just imagine it did?
Even if they can now spin out text in the context of web search results, or, hell, even in the context of whole books, or whole bodies of work, does that do anything to counter the claim that the text coming out is in the ontological category of "bullshit"? Or does it just make it "better" (i.e. more convincing, harder to falsify) bullshit?
Consider that some of the cited works would have been created 12-18 months prior to the article being written. Like the TV show. Or you might simply come across something someone wrote a year ago. That's not very long in normal human time scales. I think it's fair to complain about the general quality of AI-written text, even if the models the author is upset with are no longer the state-of-the-art.
I am personally just as tired of the screeds as I am of the AI slop the systems churn out. Yes. You're absolutely right, as they say. This is an opinion that's very palatable right now - with AI polling lower than ICE in the United States - and everything Sam says in the article is somewhere on the spectrum from understandable to self-evident. I get it. I truly, wholly, fully, entirely, completely get it.
But like, the genie's out of the bottle now. If I could reverse the GPT revolution I'd do it in a heartbeat, no question about that, but is there anything in this essay that's more substantial than the (ironically) easily reproducible anger from having the thing you care about be suddenly placed in the hands of capitalist assholes who aren't in the least bit snobbish about the quality of the output?
And more importantly to me - where does the anger even come from? I'm a writer too, I also don't like how my craft is being abstracted away and being turned into meaningless garbage that serves a purpose than exists as art, but now more than ever it is TRIVIAL for us "real artists" to outperform LLMs in the realm of creativity, because since that's something the machines are lacking and are making no in-roads on. I say, let them.
If you type "write me a story about a talking dog" or "write copy for my cosmetics brand" into the box and uncritically accept the output, you were neither an artist nor a person who appreciates the arts, evidently. The exact same thing happened with what I like to call "layman's tools", like CapCut, Canva, Adobe Express and so on that enabled people with little taste or skill to create superficially passable graphics (I'm a graphic designer, so this is my area of expertise, and it happened mainly during the early months of the pandemic).
And while I'll look for any excuse to beat someone refusing to engage with art and culture with a crowbar, this is really just striking me as masturbatory preaching to a choir that could have synthesised every single word in this essay for themselves, because they already believe it.
While I don't think you're wrong that the OP is preaching to the choir, I interpreted the overall tone of "screaming bloody murder" as a deliberate style choice that fit the subject matter simply because an AI (except maybe some locally run ones) would refuse to write something that overtly angry and with clear "I am not joking, I will kill you" statements.
100000%.
Please don't post noise comments like this. This isn't Reddit. Voting on the comment you like is sufficient.
I completely disagree. These sorts of comments are what makes a message board feel communal.
Mark it noise and move on.
The temptation to say "this" is strong here.
乁[ᓀ˵▾˵ᓂ]ㄏ
I think it really is a bandwagoning thing of people piling on to hating AI because its already a culturally accepted safe position to be anti-AI.
A lot of what is being targeted by "AI-slop" is stuff that would have been slop anyways.
Are you mad that these AI generated self descriptions on a catering website are trite and uninspired? Those have always been trite and uninspired.
Are you mad that people online lose their unique voice when they use AI to generate responses? People already all sound the same because they parrot each others talking points incessantly.
Are you mad that AI might through out some random bit of information as a hard fact without actually knowing if its true? Thats just how people talk, they throw whatever random ideas they have out there to see what sticks, and half of the time it doesnt.
Of course all of this stuff is obnoxious, but thats a problem with discourse as a whole, not the AI introducing those problems. All of this stuff would be forgivable if you found out it was a live human purposefully wasting your time with nonsense, which to me says that its not really as big of a problem to anyone as they are imagining it to be.
Or, maybe put another way, this is stuff that has always bothered everyone, but everyone already accepted that this is just how the world is and theres nothing to be done about it. Except now that it is being reframed in the context of AI, it gives the false impression that there is a window of opportunity where something could be done about it.
I don't tend to think of myself of someone following trends (I'm on Tildes, after all). I try to reason about why I like or dislike any given thing.
But even if that's the case for most people: good. Reading an old post on "types of protestors", most people will always be indifferent and silent. And the most after that are the ones who come to a protest here and there. The former can't have much be done, but the latter is most important to really spread awareness and push for change. Even if in the end they never come to another protest, they ultimately help a lot because sheer participation in numbers still has value to any given metric in society.
I'd love for more organic engagement and knowledge on such issues, but sometimes you just need people to be loud, especially in times where money wants to try and be louder.
But to answer your questions:
I already spend so much time trying to swim around and find such unique voices. It was already being purposefully worsened as search engines realize that being the best search engine isn't the most profitable move and shifted to stuffing results with ads. So it was already making my swims longer.
Now the pool is tainted to the point of nearly impacting my mental health to swim in. There used to be some enjoyment of the swimming in and of itself, because I'd still find interesting things that weren't my taste but were still interesting. Now every stroke I'm wondering if this is real or some slop trying to sell me something. Not too dissimilar to how slowly, 90% of my phone calls are spam instead of genuine humans who want to communicate.
I don't want to be sold to. I want to connect with communities of humans (not bots trying to be human to upsell me) and appreciate art as a community. Many may not appreciate it, but that struggle and effort is a good part of what makes or breaks such a community.
People (or at least, most adults) can back up assertions with something. Be it hard proof, personal experience, or at least an appeal to a common goal. A bit can't "experience" as of yet and it's awful at emotional appeal (no shock given who's driving it above).
You'd think that asking a bot for hard proof would be something it can do better, no? But somehow it will mess that up. In the worst case making up sources that never existed.
People make stuff up too, of course. But doing that more than a few times and they will be discarded as a reliable source. I'm simply doing the same here.
If I'm frustrated at people doing it, why would I have leeway to have such behavior automated? I would still be mad at a dump truck unloading on me even if it says "people litter all the time" . Yes, let's litter less and not more efficiently, please.
I guess the reason I think this is worth considering is that the answer changes how you should respond to the problem.
Like, right now I have a problem with how algorithms on Youtube and Reddit are supplying content to me. I dont like the things Im getting and Im not enjoying my time on those apps anymore.
One way to explain this might be to say that the mods or creators I was engaging with are doing a bad job, and thats why the content Im getting is not satisfying me. Another way to explain this might be to say that the platforms themselves are intentionally trying to keep me unsatisfied so I keep engaging with the platform as much as possible, and so this problem is inherent to what Im doing.
If its the former, my conclusion might be that I need to excise those particular creators or mods and once I get rid of those individuals introducing the problem, my problem will go away.
If its the latter, my conclusion might be that the only way I will be satisfied is to break my reliance on the platform and look for other ways to find entertainment and stimulation.
But if the true answer is the latter but I convince myself it is the former, I will be stuck in a permanent loop of trying to solve my problem in a way that will never work, and so I will permanently be unsatisfied.
I resonate with that a lot, yes. Ultimately, my path for Reddit was the former, then the latter. I try to see if there's ways to fix the community, realize I lack the power (or the community doesn't care to change), then in the end I seek other communities that end up being quiet or near dead.
Then at some point you leave and/or minimize participation. As of now, I simply lurk specific subs in ways that offer the least "engagement" to reddit: on old reddit with RES for some filtering features, behind adblock, with no profile and anti-tracking extensions.
This is a relatively rare behavior so I don't expect this to be the "solution" to social media (if it was, we could fix a lot very quickly), but it's one I've done quite a few times over the years. It actually harkens back to an old "internet forum law" I read on some blog: "Every forum is always in a state of constant decline."
Youtube is a much different beast, sadly. It's a monopoly in the purest sense of the word and there's so little alternative out there. I've been de-googlefying over the years and am down to Gmail, Maps, Search, and Youtube (in increasing order of friction to remove). But I have no clue how to truly remove the latter. There's really no other place smaller creators gather as of now.
Not really, no. I'm not that mad about it. 80% of text on the internet, probably more, is just digital shipping foam to keep the bits from rattling around on the datacenter hard drives. I really could not care less about people doing copywriting for their job or social media captions with AI or whatever. I am the last person to give those categories of - really, I struggle to call it 'writing' - a fair chance.
I do worry though when someone makes something they're meant to be proud of, nominally, like handmade trinkets or art, and then use AI to write the product listings and Canva for the graphics. That gets to me a bit, personally, as an artist? Like, if I care about this, I'd do it myself. But hey, free country. Do what you want, I guess.
Wholly agree on the problem about creative writing though. If you mean to make a point, then I'd like you to present that point, because if you can't be bothered to write it, I can't be bothered to read it.
What’s wrong with canva? I’m pretty sure it’s just a web-based graphic design application? I’m sure it has some AI features but it doesn’t mean you need to use them.
There's nothing wrong with Canva, and even the issues I have with it are nowhere near as foundational as I do with AI, but it's just... it's like a microwave meal. Alton Brown (TV chef) said in an interview that sometimes he'd just walk out of the grocery store mid-shop because he sees what crap the other people buy that don't have his sensibilities. With Canva I feel very similar. With not much more effort, you could make a really nice poster, learn a new skill in the process, but instead you make the graphics equivalent of microwaved Salisbury Steak. It's kinda offensive to me, but I can't fault anyone for using it.
Well it would have been badly written, but in the case of a catering company it likely would have told you what it is they can cook.
That clear communication, even if you have to filter through bad English, has gone from lots of small business websites, to be replaced by grammatically correct meaningless verbosity. It's a real problem, I do a lot of searching for products and services on the edge of my expertise and the number of times, even just this week, I've said to myself, "please not this again, just tell me if you do what I need", is insane.
I have lots of half thought out reasons as to why a company would do this to themselves, the only thing I'm really halfway certain of is that improving the models generating this shit isn't going to change much. The humans in the process either already have their brains switched off or misunderstand what is needed, it's just that now they've been handed an easy to use shotgun for their feet. Improving the shotgun won't help.
Even if it's badly written fluff, it tells you what the writer thinks their strengths are and what is worth highlighting about them. Even if it's extrapolated heavily or cut with bullshit, it reflects something about someone who has actually had some experience or contact with this thing.
Yes, that was exactly my point. Sorry, I read your comment as though it's meant to be a counterpoint to mine but actually you mean it to agree?
Heartily. It's one of my pet peeves that people treat the new bullshit like the old bullshit when the old bullshit has at the very least a homeopathic amount of human/reality connection. New bullshit is completed untethered.
I see your point that we've always had these same pain points, but there's a difference between an annoying persistent drip vs an aimed firehose.
The difference is how we came to have Wikipedia and why we now stand to lose it: trolls have always been able to ruin a wiki article, but internal gatekeeping and easily reversible pages kept the minimal labour on the quality side: it took more effort to ruin something than to revert.
The anger is the substance. We never made change in society with just one great speech, or one great essay that somehow always everyone all at once. It's a stream of action and emotion, and it needs to maintain and build that momentum to truly cause change.
It may not be "the" essay people look back on in 20, 40, 100 years from now. But it will be part of the stream that broke the dam.
Back to today: this also comes from the long standing issues all artist understand internally: the world if art is barely a meritocracy, and many of the truly talented will fall through the cracks in lieu of whatever is the loudest (I. E. Pays the most for ads) or the most willing to game the system (think Mr. Beast ruthlessly optimizing content for the algorithm). We were already having issues curating pre-Ai. Now this LLM frenzy turned entire industries into swamps.
The last angle for existing professionals is also morbid: you don't need to convince the artist they are being replace, you need to convince the executive up top that they are replaceable. These executives were already looking for any reason to cut off labor and they found the most convenient scapegoat. Even if they realize they ate wrong... Well, irrational markets and all that. These 3 years probably flew by for those kinds of people making record profits, while I've gone from a rising career as a specialist to a freelancer barely treading water despite downsizing my lifestyle
Why boo an AI-proponent billionaire guest speaker at a college graduation? To add your human voice to the chorus od dissent. Same reason a frustrated professional writer might write a rant against written slop on their Substack. It might even go viral and open the eyes of those unaware that what they've been mindlessly consuming is AI slop.
I'm really reeeeeally tired of the RLHF the big AI companies are doing to their models to make them "personable" or whatever the goal is. They're great for helping with some computer tasks. They are absolutely miserable for many other tasks. I recently asked Claude to identify a fruit and had this exchange:
I don't like this. I don't want an LLM to describe human experience as though they are its own. I'd literally rather it say "humans describe it as ...". Claude is not qualified to say any food is fantastic. Also maybe I'm ignorant, but I never hear anyone call an agua fresca an "agua" so to me the last bit reads very much as a "how do you do, fellow kids?" moment of forced in-crowd posing.
I now realize I've made a terrible mistake in engaging more than I need to. The perfect response here is "Okay." Shut up with your "good catch" bullshit. Just please shut the fuck up.
No. Again, stop role-playing as a jubilant local with insider agua fresca knowledge. You just need to say it's popular. I want a natural language interface to an encyclopedia. Not a fabricated friend.
I already had it. Don't recommend it to me. I already know I like it. I wish so hard for LLMs to only say what needs to be said, and not shotgun blast 10 things around what I said in the hopes that the wall of text will placate me.
Nor a fabricated accountant, lawyer, doctor, inventory clicker, therapist, HR, payroll, priest, deity...
I can't wait for the hype to die and we land back in actually good use cases again.
I find news like this encouraging, where users use it too much and exceeds the company budget.
Instead of helping the businesses of these illegitimate use cases save money + hurt humanity, I'll be doing my part to push them to financial ruin faster.
In a recent interview, Ezra Klein referred to such constructions as "attachment hacking". (I'm not sure where he got the term from; possibly here.)
Humans love to make connections to our vehicles, to name our Roombas, to personify pretty much everything.
The fact that this is being explicitly exploited is both disappointing and I suppose unsurprising
While it won't fix any of the fundamental problems with LLM prose, a few sentences and some "do" and "don't" examples in claude.md (or whatever they call the initial prompt field in the web/app interface settings) can dramatically reduce those annoyances.
I've tried many different strategies for this, all too often they just have unintended consequences, and it doesn't take well to instructions being scoped to just certain situations. Have you found an approach that doesn't leave it feeling like my instructions are being interpreted by a subtly vindictive genie in a bottle?
Yes definitely, but it sounds like you're using Gemini? In my limited experience with it, it's a distant third among the big 3 for most use cases. I'd recommend trying Chat GPT or Anthropic models.
Hahaha... I love human writing
I would argue this is a misuse of an LLM. I don't mean that as a defense of the chatbot itself : I just mean that LLM chatbots are being forcefully developed, in an effort to meet perceived user demand, to do tasks that it is inherently not fit for. This isn't an attack towards you, but why would it make sense for what's primarily a text generator to have recognizing fruits in pictures as a feature? And why is it that users like you go to it for that anyway, without questioning it?
The pattern's existed since the start, from the moment we started using ChatGPT as a replacement for a search engine, ignoring the actual purpose of a search engine (procuring sources of information, rather than procuring information). Perhaps part of it is our growing tendency to centralize absolutely everything we use. I don't really know how to untangle it all. But I hate it. And I hate that I sometimes fall into the same pitfall.
Because it's also a vision model. It did in fact come up with the correct fruit, just not as the first answer. It's fine with me if the model gives me a few options and one is correct. I'm not sure how else I'd get the fruit ID'd.
No, of course, I'm not saying you're wrong for using a tool that works for your purpose. I'm saying it's weird we're now going to LLMs with things tacked on to them to solve tasks outside the technology's initial purpose that could be solved with a simpler, more specialized tool that won't try to do more than it's supposed to.
I doubt there's zero services out there that let you show an image to a computer vision model and have it tell you what it represents, without having to pointlessly interface with an LLM. But I couldn't name a single one, and I imagine few people can, which is weird considering there's clearly a demand. Instead, that demand is fulfilled by over-responsibilised chatbots. I think that's weird.
I see LLMs as a natural language interface to all knowledge. But ideally they would have custom tools for specific use cases like mine instead of trying to have one monolithic thing do it all.
Some choice quotes -
What a delightfully disgusting (and accurate) way to describe what I feel when I realize a coworker has sent me an AI slop email.
I've received so many emails with so many vapid words for so little purpose.
So much this.
And in the same vein, just simple mistakes are what make us human. People caught on pretty quickly that if you spot grammatical and spelling mistakes then it's probably not written by AI. But then people figured out you can just ask AI to insert those simple mistaks. The author's point I think holds across a variety of applications. Image generators trend toward a specific style that is largely monotone. Written prose is often a centered average of emotions, with hints of stronger emotions in the corner. The extremes have all been sanded off. And that's what makes humans interesting.
I mean, this just reinforces what most people already know. A lot of top level subjective competitions are bias/corruption displays more than anything. It doesn't mean that some of these people didn't deserve accolades, but when you're picking from 20 great writers, at what point do you say "There is no sane/objective way to actually do this". Not that I think it's okay, but just that i'm not remotely shocked judges didn't catch AI writing or have much different to say. I'm sure on some level that judge would've written a similar response for any piece they picked.
Reminds me of the wine tasters that couldn't tell the difference between red and white wines once blindfolded.
This is the exact moment I knew I was in for a good read.
I generally agree with the author. LLMs are pretty great at two piece of writing, corporate work and meaningless blogs. Absolutely horrendous at prose or anything more personal.
It's weird though as it has had me reflect on my writing style. I was never the most vivid writer. I wonder if it would be better to just switch my writing to direct and brash to the point of borderline rudeness. At least you'll know I'm not a robot and won't cut me open with a knife looking for the machinery.
Fun piece, thanks for sharing.
That to me feels like over correcting. Also, coming from a culture that is known for being direct, there is a difference between directness and rudeness. They are often confused though and sometimes directness is used as an excuse to just be rude.
Having said that, not being overly verbose is generally a good thing to aim for I think (something I do struggle with tbh).
Interestingly enough I have started to value little mistakes and idiosyncrasies in text as tells that it likely has been written by a human.
Feel it on the overly verbose thing. Whenever I read something I wrote a while ago I find myself just deleting text and still retaining the meaning.
My email communication style has easily changed though.
But so much of the online comment space is people being unabashedly rude. Just enter the comment section on any video posted on any major video posting platform like youtube, tiktok, or instagram that was posted by any minority - black, woman, trans, etc. and you'll find assholes doing exactly that. If you think these models haven't been trained on this data as well, you're deeply mistaken. They've just been sanitized... or well, I should say the biggest commercial models have been. If you want a rude response, just go to Grok or any of the other right-wing models. Being rude isn't going to prove you're human either, unfortunately.
In the interest of comment diversity, I'll share this quote I liked:
This is so good. I've been thinking a lot about how inexact language is, and this is an excellent way of illustrating that idea.
I think it’s interesting in this post they specifically say they are making a real death threat, and quote legislation about what crime they are committing. I’m would think it’s a rhetoric device, but it seems a little foolish and opens them up to legal issues if someone decides to take it literally!
To me it's also not very funny? Like, either they don't, in which case they're empty threats and they're making a fool of themselves, or they do, in which case there's now evidence of intent, and I kind of fail to see the comedy in either one. I get the message, but it's just... not very funny. Idk
It doesn’t particularly surprise me that someone whose job seems to be “writer” can easily identify low-quality writing in LLM outputs. I’m sure if you asked an expert on hydroelectric dams to review the LLM’s output in explaining how hydroelectric dams work, they would have nuanced objections too subtle for me to understand the difference, in the same way that I can see the hollowness of LLM writing but might not exactly understand the full depths of a good writer’s objection to LLM output.
I’m certainly not the first person to identify this; even early on when LLM writing was much more obvious, I heard common refrains from various experts that “maybe they could replace (someone else’s job) but they’re not enough to replace (my job).” I suspect this stems from not having the expertise to know enough about (other job) to see the sloppiness, but enough competence in their own fields to see the flaws.
What this article says is correct, but the point lies in causality: cause and effect. The cause is not AI, but human nature. The average human being is stupid, ignorant, and lazy; the direction AI has taken in the world is proof of this.
Nowadays, nobody looks for information anymore: everyone just asks AI. The world is what it is, society is made of people, and people are what they are.
The internet and computer technology have already amply demonstrated this limitation of the masses' intellect. Just think of the internet itself: a world of infinite possibilities, and we have ended up with social networks and influencers, to the point where a girl in a bikini, with no qualifications, can influence a product's market, regardless of quality.
The "singularity" (I cannot find a better word) began many years ago; the flattening we perceive now is only the latest sign. There will be more.