I think the crux of the author's argument is that LLMs are bad because people use them for offloading both their thought and their acquisition of knowledge? The example they give is for education:...
I think the crux of the author's argument is that LLMs are bad because people use them for offloading both their thought and their acquisition of knowledge? The example they give is for education:
Look at education. ChatGPT may be the world’s largest learning platform, but students are using it en masse to avoid the effortful thinking that’s necessary to build their durable knowledge. Evidence continues to mount demonstrating the negative impact of AI tools within education settings (and Gen Z generally hates it). A recent study from China revealed that thousands of students essentially stopped doing their homework once they started using AI (which substantially harmed their learning). There’s even evidence that when students use large language models for supposed learning purposes, they become habituated to relaxing their judgment and critical thinking in other contexts.
I think I disagree pretty strongly? It moreso seems like, if you create a game where students are forced to get a high score by any means necessary or else have their futures taken from them (either via school placement or academic achievement-linked scholarships), they're going to optimize for the presented objective. Presumably, that's why wealthy people bribe their way through the system in some cases, or pay for improved tutoring in others. They aren't learning because it's such a beautiful and wonderful way to embody the human experience, it's so that they can claw their way up out of whatever hole they were born into (see: children pursuing sports with the sole goal of escaping poverty, or joining the military in regions where no other employer exists).
Furthermore, I'd disagree with the notion that delegating thought doesn't happen widely now. People groupthink all the time, and lean on biases or stereotypes just to get through the week. The most common complaint I hear about people w.r.t. doing anything productive for society is that they don't have the spare thought for it: why research electoral candidates; they're all the same and I'm too tired -- I'll just vote the same way my parents did/I did last time. Why look for ways to reduce my carbon impact; it's all so difficult to understand, and there's already too much to do (and some bald guy on the internet said it doesn't work anyways). Why understand why anything doesn't work; it's too complicated, and the most recent attack ad I saw said brown people are causing it anyways.
So yeah, firm disagree that the problem here is the technology of LLMs. It's companies shoving it into every corner of our lives in order to optimize their personal game, which is increasing profits (or increasing the perception of a probability of future profits).
I do agree with a lot of this, especially that LLMs and social media are probably more symptoms of deeper problems than the root cause themselves. If you build systems where people are heavily...
I do agree with a lot of this, especially that LLMs and social media are probably more symptoms of deeper problems than the root cause themselves. If you build systems where people are heavily incentivized to optimize for grades, productivity, money, etc. Of course they're going to use whatever tools let them do that as efficiently as possible. And obviously outsourcing thought, relying on heuristics, groupthink, authority, stereotypes, etc. existed long before LLMs. Where I disagree is with the idea that this means the technology itself isn't also part of the problem.
Something doesn't have to create a human tendency in order to amplify it. We already tend to look for easier, faster, more comfortable ways of doing things, so giving everyone a tool that can instantly research, summarize, reason through problems, and produce answers for them is obviously going to make cognitive offloading much easier and more attractive than it was before. That doesn't make LLMs inherently bad, any more than the fact that cars make it easier not to walk makes cars inherently bad. But their affordances do change our behavior, and I don't think we can completely separate the tool from the way it encourages or enables certain habits.
So to me the answer isn't "LLMs are the problem," but it also isn't "LLMs have nothing to do with the problem." We need to learn how to use them responsibly and, probably more importantly, teach people when using them helps them think better and when it's just replacing the thinking they actually need to do.
Ah, certainly! Similar to my response to creesch, I'd note that my message was laser focused on dismantling the article author's thesis rather than discussing the topic in general. I would say...
Ah, certainly! Similar to my response to creesch, I'd note that my message was laser focused on dismantling the article author's thesis rather than discussing the topic in general.
I would say that, IMO, the jury is still out on how best to apply these tools, so I'm not certain about what we should be teaching people even to start with.
Yeah, exactly. It actually reminds me a bit of tobacco. Early on, it was advertised as beneficial or harmless, while the companies selling it had every incentive to emphasize the positives. With...
Yeah, exactly. It actually reminds me a bit of tobacco. Early on, it was advertised as beneficial or harmless, while the companies selling it had every incentive to emphasize the positives. With AI, we're hearing a similar constant message that it's "the future" and that we should integrate it into everything.
Obviously the comparison isn't perfect, but I think the broader lesson still applies. We often don't fully understand the downsides of a technology until it's already widespread and people have built habits around it. By the time the harms become clearer, it can be much harder to change those habits.
So even if we can't yet say, "Don't do this with AI for this specific thing, but use it for that" but I think there's still value in openly promoting caution and awareness now that we're starting to see side-effects. We can acknowledge that these tools are useful while also encouraging people to pay attention to when they're augmenting their thinking versus simply replacing it, specially on the schools.
Overall agreed! I suppose, I would prefer if LLMs were pulled into disciplines rather than being pushed, if that makes sense: run active campaigns to pair up technologists (or, like, literally...
Overall agreed! I suppose, I would prefer if LLMs were pulled into disciplines rather than being pushed, if that makes sense: run active campaigns to pair up technologists (or, like, literally LLMs in a chat interface) with practitioners (educators, admins, maintenance staff, etc.) to figure out where the tools can have an impact, or where minor systems-level changes could help them better leverage the tools. Ideally, those people on the ground could directly influence the design of the tools that would be made for them, instead of diluting their input across reams of meddling middle managers.
In sum: I don't think that Sam Altman or Elon Musk know the first thing about the fields onto which their tools are being deployed, but that doesn't stop them from pushing to do so anyhow. And they'll succeed, since swinging an enormous pile of money in front of nearly anyone in a position of power will get them to agree to anything. It makes a lot of business sense if, for instance, you need to demonstrate your AI business' absurd profitability and lack all morality: blitz entire fields semi-autonomously with your infinite wealth and nearly-but-not-quite useful tools to put all your competitors out of business, then crank the prices up after they're all locked into using your platform.
If you decide to purely focus on education you might have a point. I also do agree that the root cause is companies shoving it everywhere and having (for now) unlimited funds to offer these models...
If you decide to purely focus on education you might have a point. I also do agree that the root cause is companies shoving it everywhere and having (for now) unlimited funds to offer these models for next to nothing.
However, that doesn't negate the fact that using of LLMs in itself can and often has a very negative impact on us individually. Something I have seen happen all around me. Something has indeed been changing. I have seen an alarming increase of lazy non critical use of LLM tools by people who should know better. Code spanning dozens of line trying to solve something that should only take one line. Code that completely ignores and conventions or design paradigms put in place. Code that goes directly against security practices. Suddenly downgraded dependency versions (because the models training data doesn't include the latest version). Verbatim answers from co-claude and friends and much more.
It really is basic human psychology as far as I am concerned. Our minds are wired to take the easy approach wherever they can, which historically makes a lot of sense. But that also means that many people will go down the lazy route of using LLMs to one degree or another. Is this amplified by how education works and many work environments? Possibly, but I firmly believe it is still an issue even without those factors in the mix.
Just as an aside, it's not my preference to focus on education. I re-read the author's sprawling blog post (archive link), and "LLMs are bad for the children" is the only concrete critique I could...
If you decide to purely focus on education you might have a point.
Just as an aside, it's not my preference to focus on education. I re-read the author's sprawling blog post (archive link), and "LLMs are bad for the children" is the only concrete critique I could find, hence my bringing it up. I would have crafted a more comprehensive argument were this a discussion post, and not a shared link 😅. In point form, the author wrote this:
Spun a tale to literally interpret statements about the mind == computer as being false, since it's not a Von Neumann or Harvard architecture CPU, ipso facto squishy brain != silicon calculator
Language != thought, asserted via citation
Alluding to evolutionary psychology via a metaphorical comparison to hot dogs [1] and calorie seeking behaviour, claims that AI == junk food, and since junk food == bad, AI == bad ("clogging our capacity to develop the knowledge we need [...] to navigate the world").
Pivots to how this disproves the claim that "AI is transforming education", since per the above framing, you cannot learn via AI methods (metaphorically: you can't make a balanced meal out of hot dogs)
Dismisses dismissals of AI critique by AI boosters by reframing onto education [2], which is their strongest argument of AI boosters being fallible.
(my OP; quote about how the children can't read anymore etc. etc.)
Appeals to nature [3]
Broad claim that AI assistance for education is invalid since (1) human brain != computer, and (2) some efforts have failed
Cognitive delegation is bad, but people use it
Jab at researchers by stating that they reinvented schools
Discusses school AI bans by various jurisdictions
Points out that college students have protested it
Another appeal to nature and reassertion that human brain != computer.
... hence why I was laser focused on education, since their thesis ("AI bad, humans lazy") hinges on it. Anywho.
[professional concerns around LLM usage]
That's actually super interesting! Tbh, as I'm completely out of the software dev industry now, I've been trying to come up with ways to avoid those issues. A lot of them showed up before as coworkers would just copy paste out of stack overflow: we'd end up with insecure package versions (SO suggested it), awful coding practices (it passed tests what more do you want), and so forth. I'm trying to get my personal LLM harness + tooling set up to prevent this stuff programmatically: adding local continuous testing and code coverage enforcement, automated version checking, getting a fresh agent to enforce each code guideline as part of a merge review, etc.
Thinking back, it feels like a supercharged version of the complaints I heard when switching from a majority C++ workplace to a majority Java/C#/insert-memory-managed-language-here workplace: people stopped thinking about what their code did at runtime, and started leaning on the GC to clean up their poor practices. And it worked! The GC (and python/javascript in particular) did more to democratize software development than any amount of re-reading TAOCP or Design Patterns ever could. This is obviously more of a step function than that, but I can't help but see similarities between the two. IMO the largest differences are (1) software development is now critical to the world economy, so smashing a hole in its bottom will have ... side effects, (2) there are a tonne more people to complain now, (3) as a dual to 1, there're trillions of dollars being bet that this is worth hundreds of trillions of dollars, so there's tonnes of incentive to be extra annoying about it.
It really is basic human psychology as far as I am concerned. Our minds are wired to take the easy approach wherever they can, which historically makes a lot of sense.
Maybe? I'd point out that people still play music (despite MP3 players existing), hike intentionally difficult paths (despite cars and bikes existing), and play video games on hard (despite easy modes existing).
My more broad argument from my original post could be rephrased as: when faced with a limited set of resources, and put under pressure to perform optimally with them, people will often find more efficient strategies to accomplish their objectives. However, as those pressures recede, we find ourselves pursuing art, culture, and personal improvement for their own merits (or as part of prosocial exercises), and not in the context of an external framework that demands monetary results (and threatens corporal punishment). Perhaps it speaks to the depths to which those constraints are intertwined with our current socioeconomic systems that we feel personally aligned with them. But I have hope that human nature is not to optimize laziness, and that the false perception of such is a symptom of the world we've constructed.
...
I may have had too much coffee.
[1]
AI poses a similar sort of danger to our cognitive capabilities, by clogging our capacity to develop the knowledge we need – in our heads – to navigate the world. It’s a cognitive hot dog.
The occasional hot dog won’t harm anyone, but it sure will if it becomes a regular meal at lunch or dinner. The same is true for AI — the harm is not from its occasional use, but making it part of our mental diet.
[2]
Whatever the effects of these earlier technologies, never before have we developed and broadly deployed something so explicitly intended to supplant human thinking — that’s what AI evangelists themselves argue.
Look at education. ChatGPT may be the world’s largest learning platform, but students are using it en masse to avoid the effortful thinking [...] (see previous quotes)
[3]
No other species’ brain and body develops over such an extended period as ours, and historically, we’ve complemented that with unique cultural institutions that transmit knowledge from one generation to the next.
P.S. I try not to bring up "cap----ism" anymore since it's a bit of a trigger word that instantly devolves conversations into poop slinging contests, but feel free to substitute it out for "socioeconomic system" or "external framework" if you want to imagine the first draft of this comment.
The premise that AI offloads human thought and that this is bad only works out if you use AI for the things you shouldn't be using it for. I already offload human thought about the mindless...
The premise that AI offloads human thought and that this is bad only works out if you use AI for the things you shouldn't be using it for. I already offload human thought about the mindless garbage tasks I have to do at work by simply deferring and half-assing them, so in a way, you could make the point that AI taking over those tasks and simply doing a better job than I can be bothered to is not only making the end product measurably better but freeing my mind to think the healthy thoughts I actually would like to think.
(Hyperbole of course, not even 30% ish percent of my job is stuff I hate doing)
I really don't think that is my problem to solve, I'll be honest. Yes, this is a thought terminating cliché, but what exactly do you want the individual to do here? This is a systemic issue to...
I really don't think that is my problem to solve, I'll be honest. Yes, this is a thought terminating cliché, but what exactly do you want the individual to do here? This is a systemic issue to solve and the personal framing of the article as some problem we as individuals HAVE to solve and be cognisant of is a honestly kind of weirdly framed. We wouldn't have this issue if Sam Altman didn't have dollar signs in his eyes and decided to force one of the most specific and niche pieces of computer science in decades to be General Access and desirable by any means necessary.
Can it be both? I mean, the author's parallel regarding food applies perfectly. You're not supposed to eat ultra-processed food only or mostly, there's a gigantic industry almost force-feeding us...
Can it be both? I mean, the author's parallel regarding food applies perfectly. You're not supposed to eat ultra-processed food only or mostly, there's a gigantic industry almost force-feeding us with crap as food, and yet you could reject most of it and instead eat healthy, fresh food. Not always, not easier, but possible. And at the same time, systemic changes should be pursued to make this path easier.
I studied mathematics back in the day. Computers and calculators were widely available, but we almost never used them beyond a four function because the expectation was that you should build the...
I studied mathematics back in the day. Computers and calculators were widely available, but we almost never used them beyond a four function because the expectation was that you should build the right mental muscles to become a working mathematician. So almost everything was chalk and talk with a mixture of multi day take homes or long in class exams. There were specific electives for computing and mathematics if you were inclined that way.
This is broadly the same with education and AI - just because the tool is available for the professional, doesn't mean that it needs to be used by the student.
I think the author's point is more subtle than statements about whether AI in a good thing and the problems with using it too much, but their viewpoint isn't being communicated well enough. I...
I think the author's point is more subtle than statements about whether AI in a good thing and the problems with using it too much, but their viewpoint isn't being communicated well enough. I think they're framing AI as a cultural technology rather than a purely technological one - a framing which I believe is also true - and which explains the seemingly haphazard article.
The video below does a better job of communicating the issues with AI within this framing (though is quite long). I'll stick with discussing the article here, and the author's point about language being lossy compared to thought is crucial. Much in the same way that different media like text or art communicate ideas in their own lossy way.
One of the values of cultural technologies is how it allows us to propagate knowledge through time - memory is fickle, text and art lasts generations. But what happens when there is so much information that even cataloguing it becomes a challenge? Where do you even start?
AI has the potential to be that starting point and let us grow our society to new heights, but there are also risks and downsides to using it inappropriately. Like if students lose their capacity for complex thought, are we risking a society-wide cognitive decline in favour of lazy answers? Like the hot-dog example, will we retain our discipline of healthy food or fall into obesity and malnutrition? What about bias?What systems do we need to put in place to prevent us making these mistakes? That question ties in with cybernetics, which the author starts their thesis with.
I think the crux of the author's argument is that LLMs are bad because people use them for offloading both their thought and their acquisition of knowledge? The example they give is for education:
I think I disagree pretty strongly? It moreso seems like, if you create a game where students are forced to get a high score by any means necessary or else have their futures taken from them (either via school placement or academic achievement-linked scholarships), they're going to optimize for the presented objective. Presumably, that's why wealthy people bribe their way through the system in some cases, or pay for improved tutoring in others. They aren't learning because it's such a beautiful and wonderful way to embody the human experience, it's so that they can claw their way up out of whatever hole they were born into (see: children pursuing sports with the sole goal of escaping poverty, or joining the military in regions where no other employer exists).
Furthermore, I'd disagree with the notion that delegating thought doesn't happen widely now. People groupthink all the time, and lean on biases or stereotypes just to get through the week. The most common complaint I hear about people w.r.t. doing anything productive for society is that they don't have the spare thought for it: why research electoral candidates; they're all the same and I'm too tired -- I'll just vote the same way my parents did/I did last time. Why look for ways to reduce my carbon impact; it's all so difficult to understand, and there's already too much to do (and some bald guy on the internet said it doesn't work anyways). Why understand why anything doesn't work; it's too complicated, and the most recent attack ad I saw said brown people are causing it anyways.
So yeah, firm disagree that the problem here is the technology of LLMs. It's companies shoving it into every corner of our lives in order to optimize their personal game, which is increasing profits (or increasing the perception of a probability of future profits).
I do agree with a lot of this, especially that LLMs and social media are probably more symptoms of deeper problems than the root cause themselves. If you build systems where people are heavily incentivized to optimize for grades, productivity, money, etc. Of course they're going to use whatever tools let them do that as efficiently as possible. And obviously outsourcing thought, relying on heuristics, groupthink, authority, stereotypes, etc. existed long before LLMs. Where I disagree is with the idea that this means the technology itself isn't also part of the problem.
Something doesn't have to create a human tendency in order to amplify it. We already tend to look for easier, faster, more comfortable ways of doing things, so giving everyone a tool that can instantly research, summarize, reason through problems, and produce answers for them is obviously going to make cognitive offloading much easier and more attractive than it was before. That doesn't make LLMs inherently bad, any more than the fact that cars make it easier not to walk makes cars inherently bad. But their affordances do change our behavior, and I don't think we can completely separate the tool from the way it encourages or enables certain habits.
So to me the answer isn't "LLMs are the problem," but it also isn't "LLMs have nothing to do with the problem." We need to learn how to use them responsibly and, probably more importantly, teach people when using them helps them think better and when it's just replacing the thinking they actually need to do.
Ah, certainly! Similar to my response to creesch, I'd note that my message was laser focused on dismantling the article author's thesis rather than discussing the topic in general.
I would say that, IMO, the jury is still out on how best to apply these tools, so I'm not certain about what we should be teaching people even to start with.
Yeah, exactly. It actually reminds me a bit of tobacco. Early on, it was advertised as beneficial or harmless, while the companies selling it had every incentive to emphasize the positives. With AI, we're hearing a similar constant message that it's "the future" and that we should integrate it into everything.
Obviously the comparison isn't perfect, but I think the broader lesson still applies. We often don't fully understand the downsides of a technology until it's already widespread and people have built habits around it. By the time the harms become clearer, it can be much harder to change those habits.
So even if we can't yet say, "Don't do this with AI for this specific thing, but use it for that" but I think there's still value in openly promoting caution and awareness now that we're starting to see side-effects. We can acknowledge that these tools are useful while also encouraging people to pay attention to when they're augmenting their thinking versus simply replacing it, specially on the schools.
Overall agreed! I suppose, I would prefer if LLMs were pulled into disciplines rather than being pushed, if that makes sense: run active campaigns to pair up technologists (or, like, literally LLMs in a chat interface) with practitioners (educators, admins, maintenance staff, etc.) to figure out where the tools can have an impact, or where minor systems-level changes could help them better leverage the tools. Ideally, those people on the ground could directly influence the design of the tools that would be made for them, instead of diluting their input across reams of meddling middle managers.
In sum: I don't think that Sam Altman or Elon Musk know the first thing about the fields onto which their tools are being deployed, but that doesn't stop them from pushing to do so anyhow. And they'll succeed, since swinging an enormous pile of money in front of nearly anyone in a position of power will get them to agree to anything. It makes a lot of business sense if, for instance, you need to demonstrate your AI business' absurd profitability and lack all morality: blitz entire fields semi-autonomously with your infinite wealth and nearly-but-not-quite useful tools to put all your competitors out of business, then crank the prices up after they're all locked into using your platform.
If you decide to purely focus on education you might have a point. I also do agree that the root cause is companies shoving it everywhere and having (for now) unlimited funds to offer these models for next to nothing.
However, that doesn't negate the fact that using of LLMs in itself can and often has a very negative impact on us individually. Something I have seen happen all around me. Something has indeed been changing. I have seen an alarming increase of lazy non critical use of LLM tools by people who should know better. Code spanning dozens of line trying to solve something that should only take one line. Code that completely ignores and conventions or design paradigms put in place. Code that goes directly against security practices. Suddenly downgraded dependency versions (because the models training data doesn't include the latest version). Verbatim answers from co-claude and friends and much more.
It really is basic human psychology as far as I am concerned. Our minds are wired to take the easy approach wherever they can, which historically makes a lot of sense. But that also means that many people will go down the lazy route of using LLMs to one degree or another. Is this amplified by how education works and many work environments? Possibly, but I firmly believe it is still an issue even without those factors in the mix.
Just as an aside, it's not my preference to focus on education. I re-read the author's sprawling blog post (archive link), and "LLMs are bad for the children" is the only concrete critique I could find, hence my bringing it up. I would have crafted a more comprehensive argument were this a discussion post, and not a shared link 😅. In point form, the author wrote this:
... hence why I was laser focused on education, since their thesis ("AI bad, humans lazy") hinges on it. Anywho.
That's actually super interesting! Tbh, as I'm completely out of the software dev industry now, I've been trying to come up with ways to avoid those issues. A lot of them showed up before as coworkers would just copy paste out of stack overflow: we'd end up with insecure package versions (SO suggested it), awful coding practices (it passed tests what more do you want), and so forth. I'm trying to get my personal LLM harness + tooling set up to prevent this stuff programmatically: adding local continuous testing and code coverage enforcement, automated version checking, getting a fresh agent to enforce each code guideline as part of a merge review, etc.
Thinking back, it feels like a supercharged version of the complaints I heard when switching from a majority C++ workplace to a majority Java/C#/insert-memory-managed-language-here workplace: people stopped thinking about what their code did at runtime, and started leaning on the GC to clean up their poor practices. And it worked! The GC (and python/javascript in particular) did more to democratize software development than any amount of re-reading TAOCP or Design Patterns ever could. This is obviously more of a step function than that, but I can't help but see similarities between the two. IMO the largest differences are (1) software development is now critical to the world economy, so smashing a hole in its bottom will have ... side effects, (2) there are a tonne more people to complain now, (3) as a dual to 1, there're trillions of dollars being bet that this is worth hundreds of trillions of dollars, so there's tonnes of incentive to be extra annoying about it.
Maybe? I'd point out that people still play music (despite MP3 players existing), hike intentionally difficult paths (despite cars and bikes existing), and play video games on hard (despite easy modes existing).
My more broad argument from my original post could be rephrased as: when faced with a limited set of resources, and put under pressure to perform optimally with them, people will often find more efficient strategies to accomplish their objectives. However, as those pressures recede, we find ourselves pursuing art, culture, and personal improvement for their own merits (or as part of prosocial exercises), and not in the context of an external framework that demands monetary results (and threatens corporal punishment). Perhaps it speaks to the depths to which those constraints are intertwined with our current socioeconomic systems that we feel personally aligned with them. But I have hope that human nature is not to optimize laziness, and that the false perception of such is a symptom of the world we've constructed.
...
I may have had too much coffee.
[1]
[2]
[3]
P.S. I try not to bring up "cap----ism" anymore since it's a bit of a trigger word that instantly devolves conversations into poop slinging contests, but feel free to substitute it out for "socioeconomic system" or "external framework" if you want to imagine the first draft of this comment.
The premise that AI offloads human thought and that this is bad only works out if you use AI for the things you shouldn't be using it for. I already offload human thought about the mindless garbage tasks I have to do at work by simply deferring and half-assing them, so in a way, you could make the point that AI taking over those tasks and simply doing a better job than I can be bothered to is not only making the end product measurably better but freeing my mind to think the healthy thoughts I actually would like to think.
(Hyperbole of course, not even 30% ish percent of my job is stuff I hate doing)
That's an “you're holding it wrong” argument. Most people use AI “wrong”, this is the material reality. How do we solve this problem?
I really don't think that is my problem to solve, I'll be honest. Yes, this is a thought terminating cliché, but what exactly do you want the individual to do here? This is a systemic issue to solve and the personal framing of the article as some problem we as individuals HAVE to solve and be cognisant of is a honestly kind of weirdly framed. We wouldn't have this issue if Sam Altman didn't have dollar signs in his eyes and decided to force one of the most specific and niche pieces of computer science in decades to be General Access and desirable by any means necessary.
Can it be both? I mean, the author's parallel regarding food applies perfectly. You're not supposed to eat ultra-processed food only or mostly, there's a gigantic industry almost force-feeding us with crap as food, and yet you could reject most of it and instead eat healthy, fresh food. Not always, not easier, but possible. And at the same time, systemic changes should be pursued to make this path easier.
I studied mathematics back in the day. Computers and calculators were widely available, but we almost never used them beyond a four function because the expectation was that you should build the right mental muscles to become a working mathematician. So almost everything was chalk and talk with a mixture of multi day take homes or long in class exams. There were specific electives for computing and mathematics if you were inclined that way.
This is broadly the same with education and AI - just because the tool is available for the professional, doesn't mean that it needs to be used by the student.
I think the author's point is more subtle than statements about whether AI in a good thing and the problems with using it too much, but their viewpoint isn't being communicated well enough. I think they're framing AI as a cultural technology rather than a purely technological one - a framing which I believe is also true - and which explains the seemingly haphazard article.
The video below does a better job of communicating the issues with AI within this framing (though is quite long). I'll stick with discussing the article here, and the author's point about language being lossy compared to thought is crucial. Much in the same way that different media like text or art communicate ideas in their own lossy way.
One of the values of cultural technologies is how it allows us to propagate knowledge through time - memory is fickle, text and art lasts generations. But what happens when there is so much information that even cataloguing it becomes a challenge? Where do you even start?
AI has the potential to be that starting point and let us grow our society to new heights, but there are also risks and downsides to using it inappropriately. Like if students lose their capacity for complex thought, are we risking a society-wide cognitive decline in favour of lazy answers? Like the hot-dog example, will we retain our discipline of healthy food or fall into obesity and malnutrition? What about bias?What systems do we need to put in place to prevent us making these mistakes? That question ties in with cybernetics, which the author starts their thesis with.
https://www.youtube.com/watch?v=Km2bn0HvUwg