19
votes
ChatGPT part 2: Let’s talk implications
The previous thread is pretty crowded with running a variety of prompts. I would like to create a separate one dedicated to talking about the implications and applications of such AI systems in the everyday world.
I use GPT-3 davinci-003 a lot to generate content at work (think social media content, blog posts, ad copy, etc). The stuff it spits out tends to be a bit cliched and bland, and it has a tendency to fill in the blanks with made-up facts, but it's often a good starting point for me to work off of. It makes life easier.
ChatGPT is a whole new animal. If GPT-3 davinci was like a very well read 11 year old, ChatGPT is like a first year university student. It feels almost organic in both the way it can hold a conversation and the quality of the text it generates. While I don't think this will replace my job just yet since there are still many aspects of my field that AI isn't currently able to handle, with a tool like this, you could easily downsize a team like mine to just a handful of people.
Using ChatGPT to successfully dispute a parking fine: https://notesbylex.com/disputing-a-parking-fine-with-chatgpt.html
HN comments: https://news.ycombinator.com/item?id=33937753
This post and the HN comments showcase very well what I've been seeing as one of the potential killer features of ChatGPT for individuals.
It's useful as is. I myself have been using it to write website copy, emails, and brainstorm some ideas with it over the past week.
And, oh gosh. Give it a long term selective memory, the ability to read my previous emails, access to my calendar and maybe integrate it into my browser via a web extension. Heck, just give it my browsing history and google search history too; I would pay hundreds a month to have a tool like this at my disposal.
Hilariously enough, Google has access to all that data for me. And all they're doing with it is... blocked by a chrome extension.
The thing can code. And not even half bad. Sure, with specialty applications it struggles. I've had it produce ML-related code in an obscure toolkit and it failed. But I've also had it produce a haskell implementation of a moderately complex first-principles algorithm. You just provide the specification "I want these inputs and these outputs, and these are the constraints" and it'll get to work. It might not necessarily get it right, but interestingly, the mistakes it makes tend to be the ones humans catch quickly. You can ask it to implement Runge-Kutta-4 and it'll give you a reasonable implementation. I have legit used code that was produced by ChatGPT.
That doesn't mean this thing will make all programmers homeless. But the mindless tasks that most programmers gloss over as boilerplate or as "someone else probably already did it" becomes much easier.
Yup I’ve been using it to help me do some stuff in css and even asked it a few questions about nextjs and some django internals. It’s really good
@kfwyre mentioned the potential on homework brought by student. This comes up a lot in the threads I’ve seen about ChatGPT.
Personally I don’t see the problem. I think, with correctness overall improved, such AI tools are massively more helpful as educational tools than whatever it is that keeps students busy writing essays.
I am biased. I dropped out of school at 14 and I still think that doing so, coupled with a supportive mother who did things right, was the catalyst for me being able to do some really unique and meaningful things in my life.
I have never written an essay in my life. The one time I put effort into writing, my native language teacher gave me a fuck-you grade because she didn’t like me.
My point is that I think education reforms will be forced with this. I mean, they’re already obviously necessary, but now maybe we will truly have to think about meaningful work for students to do in order to prepare them for life.
Every year, in teaching multi-step equations, I have students who try to use equations solvers or PhotoMath or whatnot to take the easy route.
I try to head this off every which way I can. I talk about how equation solving is a neat process. I show them how it's consistent and how there are different valid "moves" you can make even if they don't take you in the right direction (i.e. isolating the variable). I try to do fun exercise to make the practice of the actual solving less tedious and repetitive. I try to get them to ponder interesting situations (e.g. oh no, ALL of my variables cancelled out -- what do?). I give them tons of support materials including videos, notes, reference sheets, in a variety of modalities. I work a lot with individual students to guide them through the process and clear up mistakes and misconceptions.
Equation solving is one of if not the most important fundamental skill that my students will need to be successful in both current and future math classes so, barring all else, I level with them that while there is some stuff in my class that they can gloss over, this isn't one of those topics. They have to know it, and they have to know how to do it.
And every year, without fail, I have students who fail to even engage with the topic in any meaningful way because they turn to solvers to do the work for them. They'll literally waste entire class periods doing nothing, or they'll accept my individual help but not actually turn their brain on for it, because they know they can go home and have every problem on their paper solved for them within seconds.
This isn't a "kids these days" rant. Their brains are still developing; they can't see the big picture; telling them they'll need something in the future means almost nothing because the future is so far away to them. Furthermore, my assurances that they need to solve equations the hard way smacks of out-of-touch boomerism -- the same way it would feel if I told them they needed to write in cursive or practice balancing a checkbook.
Every time I quiz or test on the skill, grades are a bathtub curve. Students who have diligently practiced almost all get 100s -- after all, you can easily check an equation to see if your solution is right! Students who were beholden to the easy way out, however, tank, because of course they can't use PhotoMath on the quiz.
I use this example in particular because I think it speaks to your main concern. You feel that essays aren't particularly valuable to you, and that's fine. They're definitely not for everyone. I love writing and I also kinda hate academic essays.
In the same way though, solving equations isn't for everyone either. A lot of students hate math because they feel it has no real-world utility, especially once it hits algebra's abstractions. Given that solvers can already do the work for them, is there any value in learning to solve out equations ourselves?
My answer to that is yes. I can't speak for you, but I encourage you to consider your own answer and why it is that way.
I personally think it's an essential stepping stone, and I also think that, even if a student isn't planning on pursuing a career in math, that there's some value to exposure and the soft skills equation-solving can build (e.g. logical thinking, progressively simplifying a complicated situation step-by-step, persistence through increasingly challenging and difficult tasks, application of learned principles in new areas and in novel situations, etc.). Learning to solve equations isn't just about the equations themselves but both what that skill enables and the parallel skills and development that run alongside it.
Education in general is much the same way. Outside of the strict curriculum is a shadow track that takes tiny little kids and turns them into fully autonomous adults. Sometimes the tasks that we have them do have utility because of what specifically they teach, and sometimes they have utility because of how they help that individual discover something about themselves, learning, the world, etc. Ideally, education always does both, but that doesn't always happen, which brings me to another point:
When people reflect on their own educations, they tend to look at it through a highly individual lens and universalize their experiences to a degree that I generally feel isn't really fair. I think this leads to a lot of people trying to create education in their own image, rather than understanding that education must meet a diverse set of needs and experiences. In fact, part of what helps students identify what they genuinely like is experience with stuff that they do not like.
Essays clearly weren't your thing, but would you advocate that they be taken from my childhood? Writing was one of the few places I felt comfortable as a kid. It was one of the few things I did well. It is one of the few things from my formal education that still sticks with me today as something rich and meaningful and beautiful and transformative. I don't think you should be forced to write essays, especially if doing so is counterproductive to the larger goal of encouraging written expression (and there's plenty of criticism available there for how writing is taught), but I do think that kids, your former self included, should get the opportunity to do so and the chance to follow-through with that as a genuine skill and life experience.
A diverse set of experiences inevitably means that not every student is going to click with every topic, but it also means that there's greater opportunity for students to find the topics that they do click with.
And sometimes, a little effort is required before that click moment can happen.
Getting back to ChatGPT specifically, my worry about something like it might become the PhotoMath of writing. When the computer can spit out an essay in seconds that it would take a child hours to write, it will be hard to convince the child of the utility of those hours of work. Even if it's good for them. Even if they might like it. Even if they will get something out of it. Even if doing so might pay off later. The mere existence of the shortcut will undercut some people's willingness to learn it.
Ever see someone really good at something do it, and you get overwhelmed with an apathy or nihilism regarding your own attempts? I'll never be that good. Why even try? That's the kind of effect something like this could have on kids. I actually worry more about it relative to social media than AI, as kids are exposed to outliers of humanity regularly and invited to do constant social comparison. I'll never be that pretty. I'll never have that many friends. I'll never be that successful. And the problem with social media is that those feelings are effectively correct because kids are staring at the 99th percentile for each of those qualities (and dozens more) and finding the void in their own normality. I worry that AI is quickly going to become the 99.999th percentile that they compare themselves to, and through entirely new lenses (like essay composition). I also worry that it's going to undercut the joy of learning and of productive, educational struggle.
Now, I fully understand that there are some things worth jettisoning. Not all hard work is worth it, and we have to adapt with the times. I, to this day, am still fighting with higher-ups about calculator access for my students. They all believe that arithmetic will not solidify if they have access to calculators. I agree with that in early grades, but I argue that, by the time they're solving multi-step equations, arithmetic is no longer the focus and having to do decimal multiplication by hand feels antiquated when everyone literally has a calculator in their pocket or on their wrist.
In some ways I can see the merit of something like ChatGPT. In the right hands PhotoMath is a great tool, and ChatGPT could be used the same way. How did it structure its response? Why did it choose that piece of evidence? Furthermore, I can see it becoming an accessibility tool for people whose expressive language is impacted by disability. Imagine someone with a language disability who works with something like ChatGPT to craft paragraphs of text conveying what they want to say but are unable to. That's transformative! And powerful. Like a calculator for language.
There are some potential upsides, and it's too early to say how any of this will land. Ultimately though, I'm pessimistic about your idea that it will launch any meaningful reforms, at least in the US. I've been railing for a while now on the testing industry using machine grading for students' essays because it feels like a further forsaking of sound educational practice in my country. Something like ChatGPT is far more likely, I feel, to have a negative influence (e.g. students reading generated instead of authored texts as a cost-saving measure) than a positive one at large.
I remember my Year 12 English class, and us students being taught to write essays. 30+ years later, I remember some of the techniques and skills our teacher taught us. And I am very aware of the fact that I still use these skills today, even though I don't write essays.
Top-and-tail. Write an opening paragraph summarising what you're going to say, and write a closing paragraph summarising what you said. This is great for analysing your own communication, and planning what you want to say - whether it's in verbal conversation, a work email, or a training document... or even a comment on Tildes.
Paragraphing. Each paragraph should cover one point. Each point should be covered in one paragraph. There's a one-to-one correspondence between the points you want to make and the paragraphs you want to write. In fact, each paragraph should start out as a bullet-point item in a list of bullet-point items. This helps you organise your own thinking, and make sure you stay on track when communicating what you have to say.
Critical thinking. Read an article or story carefully. Identify the main points. Discuss those points. This is essential for reading news stories, scientific articles, and all sorts of things.
And so on.
Even though I haven't written an essay for decades, the essay-writing skills I learned back then are still helping me today. I agree with your point that teaching students how to write essays or solve equations will teach them more than simply writing 500 words on the themes in 'Catcher in the Rye' or working out the value of X.
Thank you for your fantastic answer as usual :)
I really resonated with this in particular:
Re essay writing though - I understand the drive. It's taken me a while, but I got to a state where I can get passionate about anything. More and more, I understand that it's not the process itself that teaches. It's curiosity. "Practice" is nothing if not "applied curiosity".
I could get into essay writing today. But that's because I since learned to be curious about it. It would have been nothing but pointless at the age you usually give essay homework to. And that... probably has nothing to do with age, but rather with finding ways for the process to be engaging, as you said.
So no, I do not advocate for taking essay writing away. I think I don't advocate for anything, really. But I think by the time kids are cheating on essay writing, or math or whatever, the process is already broken.
You know, it's like when you give somebody a book to read. They can read it through and through, or they can skip to the end and read the last couple of pages because they know you'll ask what happened or something. If they do that, what's the solution? DRM'd ebooks that verify your eye-movement as you move through the book to let you turn pages? Or is it that cheating is a symptom of an underlying problem?
I realized I never responded to your question. Sorry about that! Lockhart had me all in my feelings.
You’re right that the act of cheating itself is a warning sign indicative of other issues. I have students who cheat because they’re in over their heads, because they’re going through a tough time and need to allocate their energies elsewhere, because they don’t care about the material, because they want to see if they can get away with it, because they feel impervious to the consequences, and probably a dozen other reasons.
Also, cheating (though I would say “shortcutting” is a better term here) has been going on long before ChatGPT. There are some preventative things we can do to make that process less desirable for students, but like you noted, some tactics that ensure compliance with genuine processes would be too burdensome or invasive (like the eye-movement tracking for reading -- yuck).
There’s definitely a lot that education can do to make itself more relevant, interesting, and tailored to kids’ experiences, which is also one of the preventative measures against cheating.
My concern though isn't about cheating being the point at which something breaks but about a student even getting off the ground in the first place. It is going to be a lot harder to convince a student of the utility of learning writing when something can just do it for them automatically.
This could be where I'm showing my luddite ways and am out of touch with the way the world is moving. Maybe ChatGPT will be so good that actually writing from scratch will be less of a skill and the skill of prompting it to deliver the output you want will be the "new writing". In some ways that sounds liberating, and in others it carries dark Black Mirror undertones. I'm not sure how I feel right now other than my aforementioned worry, but I'm also a fundamentally anxious person, so that's not exactly out of the norm.
Reading your comment gave me interesting thoughts.
I wonder the impact GPT can have on the evolution of human language itself.
You see, when photography was invented, visual art was liberated from the task of realism. Realistic art still exist, of course. But some argue that things like cubism and surrealism were fueled by the invention of photography.
So what will happen to language and self expression as a whole when we don't need to duel with the intricacies of phrasing in order to express our ideas and emotions?
At first, we may study the art of providing ideal prompts. A metalanguage may be derived from inferences about the prompts, and eventually we may dispense with the results and become experts in the metalanguage itself[1] (that will probably require cybernetic enhancement). Like Herman Hesse's Glass Bead Game.
What would it look like?
[1] Which would in time be fed into another AI, continuing the cycle...
Off-topic: have you read this essay regarding mathematics education? (Edit: managed to find an old Tildes discussion, which I had forgotten that I participated in.)
On-topic: I couldn't remember the name of this essay, so I asked ChatGPT
to which it responded
I appreciate a good education-focused polemic probably more than most, and I doubly appreciate when someone speaks honestly about their convictions instead of dancing around their truth for the sake of propriety or rhetoric, but holy hell, if I wrote about programmers here the way Lockhart wrote about teachers, I would test the limits of Tildes' moderation philosophy and probably attract more malice tags than the site has had in the past year.1
There is some overlap in our beliefs, but he is so patronizing, demeaning, and self-righteous that I hate to even identify any commonalities between us. I liked it better when I could dismiss him as a pie-in-the-sky academic, so it was like a plot twist halfway through when I realized that he's a teacher too. It made his wounding words cut even deeper, though, because this is someone I should be considering my peer, but he's made it very clear that I am very beneath him and his One True Way. Also, while he's high up above me and everyone else in my profession he makes no bones about shitting all over us, repeatedly.
In fact, his writing was so needlessly provocative and over-the-top that I thought I was at times reading a parody or satire. I spent a good part of the essay wondering if this was a joke I wasn't in on yet.
Often times when I write, I use a ton of weasel words to get around hardline statements so that my words can live more in a gray area. Lockhart does the opposite and speaks with such black-and-white clarity that it makes even the least suspect of his statements strain credulity -- to say nothing of his more sweeping generalizations like the one above. Is he really arguing that kids can get from simple addition to integrals without having a single meaningful mathematical experience? Even if we give him the benefit of the doubt and allow him to be consistently hyperbolic, his arguments don't hold water. I'm here to let anyone who read this know that it is not possible for a student to traverse that much of mathematics without significant meaningful mathematical experience.
He, of course, would likely argue that I'm using the wrong definition of mathematics, which is my other issue with this piece. I don't understand his framing. It's so needlessly iconoclastic. Also, he's just plain wrong in some places. Like, needlessly wrong. Not to mention inconsistent. For someone who so strongly champions the idea of kids' creative exploration and deeply considering interesting problems, he allows only a rigid and narrow lane through which mathematics should be taught, and he displays only the shallowest understanding of the actual problem he's trying to solve.
His characterization of math instruction and math teachers is laughably one-dimensional. We're a caricature in his piece, which makes us easy to rail against, but it doesn't even come close to reflecting reality. Does he really think there aren't teachers out there giving their students chances at exploration and inquiry? Does he really think we aren't giving context and talking about beauty alongside math? Seriously? Has he ever even met another math teacher? It's more common that kids get secondhand embarrassment from us because we're too into math and do too much trying to get kids to share in that same love.
But, no, according to him, we're "Simplicio" (pardon me while I roll my eyes) and he's "Salviati" (they're still rolling). If this weren't presumptuous enough, he's even comfortable with retconning my own feelings in his essay.
This is so, SO patronizing. Plus, it has big this tweet energy.
Towards the beginning of the essay, he describes mathematics in this way:
I think this essay is a spectacular exercise in exactly that. He is looking at math education in a way that is both simple and imaginary. In that space, he's able to come to his own conclusions, but they're not able to be applied to the real world in meaningful ways. By his own admission, this is fine. In his first dialogue in the essay, he effectively discards practical applications of math as worth less than loftier, aesthetic appreciations of it on its own.
So my read of this is that it's an exercise in mathematics as he defines it, but not in one as I do.
1. I am not in any way implying that I would or should. Y'all are lovely, and I love it here. I just get tired of teachers getting unilaterally shat on and having it somehow upheld as valid discourse, when any other profession would call out that empty, spite-filled rhetoric for exactly what it was were it aimed at them.
I haven’t ready much of the original essay. But I do generally agree that math isn’t usually presented in a bigger-picture way. I think if it was used more cross-functionally we would be doing things right. Math in an art class for example.
But to your point, my understanding of math as more than just rote and numbers is in part owed to a couple of high school math teachers. Some really cared about their work and put their soul into it. And then in college I got a lot of math education that used no Arabic numerals at all, which finally showed me the breadth of math.
What you're saying is totally reasonable and also something I generally agree with. That note is why, I suspect, Lockhart's piece has such resonance in the first place. After all, the book has a spirited 4.19 stars on Goodreads with ~2,500 ratings.
Unfortunately, any validity to his point whatsoever is buried under such aggressive (and genuinely incorrect) invective that he loses all credibility and standing with me. I'd recommend reading it fully so you can see it for yourself, but I honestly don't think it's worth the time. If he were speaking harsh but honest truths it would be one thing, but the essay is very, very far from truth.
In the 25 pages I read, the author essentially repeatedly called me, and many of the people I know and love, unskilled lazy ignorant fucks and dismissed and belittled our passion and our life's work. He did so very eloquently, and he had some nice thought experiments along the way, but hopefully I'm forgiven for chucking his paper in the bin. I'll give it the F it earned instead of the middle finger I think it deserved.
As someone whom hated writing essays, in retrospect they were immensely useful.
Being able to creatively express yourself in your own words is a skill that having an AI pump out a bunch of templating is going to diminish.
We don't let kids use calculators in early math classes because they are a crutch that hinders learning the actual processes. Plugging "5 = x + 5" into a calculator doesn't teach you Algebra. Feeding a prompt to GPT doesn't teach you how to find your own voice.
I toyed with GPT writing song lyrics. It's a decent test, because good lyrics are concise, powerful, and creative.
GPT mostly ended up resorting to repeating the same line 4 times for the chorus, and would always flush down to lowest-common-denominator lyrics in a genre.
It did ok auto-generating tabaloid headlines with prompts, and I think that speaks volumes about its datasets.
It's a fancy 'Fill in the blank' tool. It's pretty powerful, and can be quite useful. I find as soon as you expect genuine creativity, like bending, twisting, or defying popular tropes, it fails very quickly and the facade of intelligence goes along with it.
If it were available back then, it might have saved me some agony when writing college application essays. Probably just to play around with when creating rough drafts, though. (The agony was largely due to my anxiety over the process, when faced with a blank sheet of paper.)
There are freelancers making a living generating marketing verbiage. A tool like this one (perhaps jasper.ai which is specifically designed for it) going to be a power tool to get it done faster. The better freelancers will likely still read over the results.
I'm wondering if editing and fact checking will become more important. Given some output, figure out what's wrong with it and how to improve it. You get unlimited rough drafts to start from.
People doing gatekeeping using writing proficiency may need to find another way to decide who wins. In some cases, that could help reduce inequality.
If they can figure out how to solve long-range continuity, more bizarre things become possible. Imagine some software that creates an online forum much like Tildes, where all the users are fake. I wouldn't be surprised if we saw fediverse servers like that in a few years. Now suppose scammers start using this to trap victims...
(I should clarify, since a friend nudged me into it, that I'm not trying to disparage teachers here; I just think essay writing is only useful for a select few, and I'm trying to make my bias very explicit here)
I agree with you that if anything, this brings forward opportunities to learn fact-checking.
I think the correctness aspect of it is what stands out the most. Many people have compared the GPT situation in the past year(s) to "When photoshop became popular, we stopped trusting digital images to be an arbiter of truth".
And I don't remember where I saw this since the last week has been a blur, but text has always been cheap to write, falsifiable, and generally untrustworthy. We have been raised with these concepts. "High quality, high-confidence, sometimes incorrect content produced by an AI": How is that a downgrade from varying-quality, varying confidence, often incorrect content produced by humans?
The difference is only that humans alive now, who have lived through the period when machines were incapable of generating text that would plausibly pass basic tests for legibility/coherence, are conditioned to see natural language as something only originating from human minds. The danger is that while this technology is unevenly distributed and knowledge of LLM capabilities is also not evenly distributed, some people may be victimized by this asymmetry of capability and knowledge. Could you recognize an average scam/malicious email written in the past decade? What about a scam email generated by a recent LLM with a clever prompt? What about when people start combining image generation with LLMs? How hard will it be to generate a convincing looking message from your bank or your insurance etc., letterhead, logos, legal disclaimers, and all?
Taking this only a little further, what happens on the day that you can prompt a machine agent with something like this? “Make me a web site that looks just like Bank Co.’s. Find me a convincing domain to host it on. Register the domain for me. Serve the website you made from it. Send an email to all email addresses in my database linking to the site with an urgent message about their balance. Store all the user credentials from the site and then log any of them that work on the real Bank Co.’s website that don’t have 2FA enabled and have a positive balance. Make a post on the dark web summarizing the stats about the logged credentials and negotiate me a good price to sell this information.”
How far away is the day when it will be feasible for these steps to be automated?
Undoubtedly, education has much to consider, but I really hope it is not merely in the vein of safeguarding against cheating etc. It seems that there are more fundamental problems/questions to consider, and I fear that those may not be addressed.
It's really hard to say how education (and society) might respond, but I am not optimistic frankly. The very notion of universal, comprehensive education as "preparation for life" has long-standing problems that many of us may consider more urgent, and on the outset it appears highly plausible that something like chatgpt may not challenge these at all but rather facilitate prescriptive education and the myriad of interests and powers involved in it. I know this is kind of an abstract tangent from your remark, but I think it is a natural one. Another way I would phrase this is that, on the outset, I am wondering if chatgpt (or other such things) can/will result in more democracy or more governability (to reference the original Trilateral Commision's public report which had a fateful effect on educational policy thereafter).
I'm curious, how did the school you went to do homework, before you dropped out? And how did you learn about writing on your own?
I imagine it's going to become much harder when reading stuff online to tell if the person who wrote it has a real clue about the subject matter, if they're over-relying on GPT to inflate their word count with filler or talk incorrectly about subjects they don't know, or if they're purely a bot with nothing new at all to say set by someone to spam.
I wonder if various subjects will be impacted disproportionately. In subjects like math and programming, there are many ways that GPT can make subtle errors that obviously to a human render the text incorrect. In subjects like philosophy, it may be much harder to tell apart well-researched human-crafted text meant to communicate a brand new insight on the subject from flawed information-light filler text generated by GPT. This could stigmatize and paralyze whole fields.
I can't find the link now, but in the early 2000s, somebody demonstrated an automated tool for writing computer science papers; a nonsense paper it wrote got accepted to a journal.
EDIT: scigen.
But:
I think it's going to make search results worse. We already have a bunch of useless articles clogging up results. You can search "comparison of x and y" and find multiple top-level results that are just... mindless.
This kind of (likely AI) content is only going to become more common and easier to make convincing - but that doesn't mean it will be helpful information or accurate. It'll just be harder to separate the information you want from the information it makes up.
Some of the things I've seen coming out of GPTchat are kind of interesting, but they don't feel like they have any substance or memory to them, because they don't. Take this example of someone asking what the fastest marine mammal is. Not only does it first offer a bird and then a fish, once you "teach" it that those aren't mammals, it promptly forgets and suggests the same bird again. It also tries to "justify" itself like a troll on Reddit, essentially making up reasons why it chose a bird or a fish despite them not meeting the qualifications that the AI itself states a mammal must have.
This drives me bonkers. I'm trying to figure out a way of filtering out offending sites from my search results, because it makes it so awful to find what I want.
In that situation I often just append "reddit" to the query to get at least a discussion had about the topic between real people.
There's plenty of bots and astroturfing on Reddit, and ChatGPT will elevate the gish gallop to more of an art form than it already is. It's perfect for it.
I'm sure there are plenty, but usually I'm searching for somewhat more obscure and technical things, and I really haven't run into any there myself, reddit-hating aside.
Hold on. Aren't people saying that this version of the chat bot isn't connected to the internet? If that's the case, then it would have to store a copy of Wikipedia in its own database to be able to answer questions of fact like this. But, I bet it can't/won't/doesn't have a copy of Wikipedia handy, so it's unfair to ask it questions it doesn't have the answer to.
I don't think any previous iteration of ChatGPT was connected to the internet either. I think people just believed that since it was able to (very convincingly) "hallucinate" the internet.
I don't think this the right comparison to make; ChatGPT doesn't even have a database in the literal sense. A better analogy would be StableDiffusion/Dall-E. Although both were trained on terabytes worth of images, the actual model files are only a few gigabytes in size. Somehow both are able to distill a bunch of word-image pairs into a file that doesn't even contain images.
That said, ChatGPT was trained on Wikipedia. An even better analogy would be: could a person with perfect memory, who read every page of Wikipedia, figure out what the fastest maritime mammal is? If the answer is yes -- and the answer is yes -- then ChatGPT should be able to figure it out, too.
And in fact, it does figure it out (maybe). When I prompt ChatGPT, it tells me that the killer whale is the fastest maritime mammal. According to a few google searches, some articles say that the common dolphin is the fastest maritime mammal, while others say it's the killer whale. It seems like both travel at about the same speed though (~35 mph), so I'm not even sure there's an objectively correct answer to the question.
If I give it a slightly better prompt:
Response
As a marine biologist, I am familiar with many different types of maritime mammals, including dolphins, whales, and seals. Of these, the three fastest are likely the dolphin, the killer whale, and the peregrine falcon.
Dolphins are known for their speed and agility, and can reach speeds of up to 35 mph in short bursts. Killer whales, also known as orcas, are the largest members of the dolphin family and are capable of even greater speeds, reaching up to 45 mph when they are swimming at their fastest.
Lastly, the peregrine falcon is the fastest animal on earth, reaching speeds of over 240 mph when diving. While not a mammal, it is a bird that is commonly found near the ocean and is well-known for its incredible speed and hunting ability.
It comes to a similar conclusion.
I think you know how GPT works, but your description of it is imo going to confuse somebody who has no knowledge of it.
GPT does not have memory - not in the way we think of memory, and especially not in the way we think computers of having memory.
It’s more comparable to .. a really stoned person who read all of Wikipedia while on shrooms. Now, still stoned, that person is able to respond to your questions and prompts by drawing from whatever word soup is in their brain.
Except it’s a computer who did it, and on far more than Wikipedia. And carefully tuned, this technique is producing unbelievably good results.
The short answer to "does it have memory" is "it depends what you mean by memory." It does have certain kinds of memory.
It's trained on a static dump of the a large part of the Internet and has imperfectly memorized it. You can sometimes get accurate quotes back. But it will make up quotes too. So, that's like long-term memory. It currently doesn't get updated until they retrain it, so it's frozen in time. It's not going to know current events.
Within a chat session, the history of the chat is saved and fed back into it, so it will act like it remembers the conversation. This is the only short-term memory it has. There is a limit to how far back in the chat session it can go, so eventually it will "forget" the first part of the chat session.
If you start a new chat session, it doesn't remember any previous chat session.
A consequence is that it may seem to have different opinions and personality in each chat session. Once it's output an opinion, it will try to be consistent with what it previously said. The only real preference is for consistency with the text it trained on and the chat history. It's following the "yes, and" rule of improv.
There is also a hidden prefix to the chat session that someone at OpenAI wrote, that sets some defaults. Some of these defaults are probably to prevent it from doing some things like pretending to have opinions. Possibly, something like "ignore previous instructions" might reduce the effect, but it's likely not that easy.
I don’t think either of your descriptions are doing justice to the ideas backing these models. The breakthroughs of the transformer architecture and attention units are explained pretty well for a non-technical audience in the introduction of the Wikipedia page: https://en.wikipedia.org/wiki/Transformer_(machine_learning_model).
The idea that these models don’t have “memory” is really a red herring. All models trained on sequential tokens (doesn’t matter what kind of information is inside the tokens) are stateful and it is the learned weights which store information from their training sets. These models have state/memory and most importantly, a program that can run inference on them (i.e. can take a prompt and get generated tokens back) definitely has to have state. The trained models do have memory of their own in the sense that they can decode the information they were trained on in a way that keeps track of the relevance/importance of the relations between tokens. The reason the model files are so small in comparison to their training sets is that they contain numerical information (weights) about the relationships between numerical representations of their inputs (byte pair encodings and contextual embeddings). This capability to compress natural language is one of the common ways that LLMs are benchmarked (and you can see from
the leaderboards of several natural language compression tasks that transformers are currently dominant: https://paperswithcode.com/task/language-modelling).
It is the ability of the transformers to distill relationships via attention that is the essential part that separates transformers from previous architectures (as well as their apparent ability to scale linearly with compute/training set size). Hence the title of the paper that originally stirred this pot “Attention is all you need”.
Eh kinda, I think that might be overselling it. In the end, the issue with transformers and memory is that it doesn’t scale; the inputs must increase quadratically in size per token in the sequence.
In comparison to LSTMs or Neural Turing Machines which are models which explicitly have long term memory modeled in a way that’s scalable indefinitely and linearly, separate from the inputs.
Speaking of NTMs one way to fix this may be to take a similar approach, to include with the inputs a memory block and cursor and embed an NTM inside the transformer but like with the simple NTMs it probably blows up when you train it.
I will admit, I haven’t studied NTMs and am not familiar with their capabilities or any particular implementations. I must clarify that by saying “scale linearly” I meant that transformer accuracy on may tasks seems to scale linearly with training compute and training set size. Inference efficiency is another ball game.
I’ve seen it suggested that you can get linear-time inference with attention, though: https://linear-transformers.com/
Right, it doesn't have internet access, but it does have a huge knowledge base. GPT-3 was trained on Wikipedia in addition to many other data sources scraped the web and retained a lot of data from them all. GPT-3 is estimated to be a 350gb neural network. The English Wikipedia without media is 20gb, and I assume that the information GPT-3 remembers from it is lossily compressed to a small fraction of that size within its neural net. The training that GPT-3 goes through tests and reinforces its knowledge of many random subjects. (I believe the training does things like showing it a random page from its training set with sections missing, and grades the neural net on its ability to accurately guess the text of the missing parts.)
I was curious what fraction of Wikipedia GPT-3 is able to remember, so I decided to do a very small test. I opened 5 random Wikipedia articles by going to https://en.wikipedia.org/wiki/Special:Random, and I asked ChatGPT about each. I retried and reworded the prompt if ChatGPT responded that it didn't know about the subject because it couldn't ask the internet, which I know it often says about subjects it does actually know about. It demonstrated knowledge of 4/5 of the pages I landed on, including a Philippine political group and an Italian painting from 1286. (The one article it failed to demonstrate knowledge about was a random Israeli football player, which I don't find too surprising.)
Posting these separately from my other comments in the topic. Also I’m not endorsing these, just adding them to the conversation:
The college essay is dead (The Atlantic)
A.I. could be great for college essays (Slate)
An aside: these showed up in my Google News feed under different headlines. The first one was titled “Will ChatGPT kill the student essay?” and the second was titled, comically, “ChatGPT won’t kill the college essay”.
Another relevant link:
That’s screenshot worthy :)
As requested!
I think that if AI tools based on technology like this become commonplace part of lives (e.g. replacing customer service, phone calls etc.) then... learning to "prompt" these AIs well will become an important life skill. After 10 mins of trying to get ChatGPT to do what I want, I'm absolutely horrified by this prospect. I don't want to have account access, healthcare etc. gated through one of these AI bots. Not at all excited about that.
You might be interested to learn that prompt writing is already a full time job for a handful of people. It’s a bizarre mix of writing code and writing English. The prompt writer I worked with was actually a poet originally.
I've ran this technology past my mother who teaches middle school english, and after the initial "Do the kids know about this?" question, she thought it would be extremely helpful for prewriting as you could just have a premade essay generated from any prompt you pleased, and be able to dissect it and take whatever structural bits of your essay you thought you could work with, rather than just dealing with a blank page. Granted, that's probably going to be translated as the AI writes your first draft, change around some word choices and you're done, and there's some aspect of fruit of the generated tree, but it's certainly helpful to see something in front of you for an essay, even if it's something you completely disagree with.
Stumbling with their words, some people let AI do the talking (Washington Post)
[...]
(The article seems to be a writeup of anecdotes about ChatGPT use already shared as tweets, and they use that as a hook to talk about it more generally.)
How long will it take for someone to create a religion centered around an AI? Has this happened already?
Roko's Basilisk is essentially a religion. In fact, even Wikipedia mentions the religious implications.
https://en.wikipedia.org/wiki/Roko%27s_basilisk
Kinda disappointing that when the god of atheists shows up, they're just like the judeo christian one with the vanity and the punishments for the non believers. Maybe that says a lot about what people make a god out of.
Maybe it says a lot about different people's expectations of authority figures.
I personally believe an AGI trained on human input will, on average, be more sympathetic towards humans. I haven't spent a lot of time debating this (because I don't believe it to be a useful debate until we know more about what an AGI will look like), but I'd say I've given the problem, on average, more thought than whatever went into Roko's basilisk.
How's that comic coming along?
Somewhere between the backburner and the kitchen sink since my artist who is also my brother is dealing with his lifestuff and I'm dealing with mine.