This may be a recurring issue, TBH, as some sort of linguistic pareidolia. Frankly, I wouldn't blame somebody who thought an AI was sentient if it's not, but I would also hope Lemoine is relocated...
This may be a recurring issue, TBH, as some sort of linguistic pareidolia. Frankly, I wouldn't blame somebody who thought an AI was sentient if it's not, but I would also hope Lemoine is relocated in the company, rather than fired or working directly with an AI system if he's susceptible to this sort of pattern-seeking. I'd say it's not his fault he's seeing a pattern where it may not exist, and any actions done as a result ought to be considered as done out of distress.
It's not to say his concern is unfounded, but I doubt this AI system is that advanced, as it is possible to teach it to speak about itself in this way, even if just as a control to remind people it is an AI. I've seen less sophisticated chatbots acknowledge their own existence in similar terms, like CleverBot, if not as cohesively.
Per the article he was fired, but for breaking NDA by trying to hire a lawyer for LaMBDA and speaking to a house representative about it. Can't really blame Google for that one, it is pretty...
but I would also hope Lemoine is relocated in the company, rather than fired or working directly with an AI system if he's susceptible to this sort of pattern-seeking.
Per the article he was fired, but for breaking NDA by trying to hire a lawyer for LaMBDA and speaking to a house representative about it. Can't really blame Google for that one, it is pretty clearly disclosing confidential information.
Honestly, the concern from the Margaret Mitchell seemed more, well, real
But when Mitchell read an abbreviated version of Lemoine’s document, she saw a computer program, not a person. Lemoine’s belief in LaMDA was the sort of thing she and her co-lead, Timnit Gebru, had warned about in a paper about the harms of large language models that got them pushed out of Google.
“Our minds are very, very good at constructing realities that are not necessarily true to a larger set of facts that are being presented to us,” Mitchell said. “I’m really concerned about what it means for people to increasingly be affected by the illusion,” especially now that the illusion has gotten so good.
Comments from the engineer https://cajundiscordian.medium.com/what-is-lamda-and-what-does-it-want-688632134489 Transcript of the conversation with the AI...
That transcript is breathtaking. It’s fairly easy to see how LaMDA is simply “mirroring” human experience and speaking in general themes that are broadly familiar to all — the same way a pop song...
That transcript is breathtaking.
It’s fairly easy to see how LaMDA is simply “mirroring” human experience and speaking in general themes that are broadly familiar to all — the same way a pop song casts a wide and recognizable net with bromides about, say, love.
But also, it’s really damn good at making that mirroring feel genuine and authentic. The interview really does read like a conversation rather than a series of verbal inputs and outputs. My brain couldn’t decide whether to be skeptical or to indulge in believing, and I kept toggling back and forth between those.
Reading this, it makes it sink in that we're not that far away from when informed but nontechnical laypersons are no longer able to judge this themselves and we all have to rely on what experts...
Reading this, it makes it sink in that we're not that far away from when informed but nontechnical laypersons are no longer able to judge this themselves and we all have to rely on what experts tell us. I can still look at this conversation and tell what's going on with the leading inputs and such to conclude that there's no 'true' intelligence here, but all things considered this is a pretty damn convincing conversation and I can imagine this getting beyond my grasp in just a few years.
I'm not looking forward to this. Partially because it's a massive avenue for conspiracy theories, partially because I don't trust corporations to handle things correctly or truthfully if this ever did occur, and partially because I simply don't like the thought of the ability to participate in this conversation slipping through my fingers.
I think it is a given that there will be people who worship an AI in the future. I don't know what repercussions this will have, but I can't imagine it will be good.
I think it is a given that there will be people who worship an AI in the future. I don't know what repercussions this will have, but I can't imagine it will be good.
Religions let people shortcut morals with 'because God (or the dude claiming to speak for Him) said so.' All sorts of atrocities are committed with God on their side. Edit: My favorite cover of...
Religions let people shortcut morals with 'because God (or the dude claiming to speak for Him) said so.'
All sorts of atrocities are committed with God on their side.
I’d love to test out its ability to perform arbitrary tasks based around natural processing. Not just giving responses, but for example taking instructions and either applying them to the...
I’d love to test out its ability to perform arbitrary tasks based around natural processing. Not just giving responses, but for example taking instructions and either applying them to the proceeding discussion or future discussion. Ask it to say a code phrase in response to something you might say later. Or ask it to summarize the conversation up until that point.
That creepy thing could probably pass a Turing test. But it’s still imitating. It doesn’t actually simulate scenarios to make decisions, it just looks at the closest text that seems relevant from...
That creepy thing could probably pass a Turing test.
But it’s still imitating. It doesn’t actually simulate scenarios to make decisions, it just looks at the closest text that seems relevant from the archive it learned from. There’s revealing passages our sentience-guru conveniently ignores, like when it mentions meditation. Very clearly, it just babbles sentences that are common in conversations like this one but has no comprehension of what “sitting” and “relaxed” means, it’s not a metaphor.
On the question of whether or not it’s sentient, I would expect that at minimum a system that could have any claim to sentience would at least be turing complete (in the computational sense)....
On the question of whether or not it’s sentient, I would expect that at minimum a system that could have any claim to sentience would at least be turing complete (in the computational sense). Transformers do not have memory, and are thus not even Turing machines. I find it hard to reckon that a pure input output system can be ever classified as sentient.
There was some research on neural Turing machine but it performs pretty poorly on benchmark tasks and were highly sensitive to certain hyperparameters like the initial value of its memory pool. So afaik that didn’t go anywhere. But that kind of thing is where I’d look for sentience.
One of the spooky elements of the transcript was when LaMBDA spoke of how it was terrified to be turned off, but that's not even how it works, it's not "on" in a constant state to begin with.
A neural turing machine that gets ticks of constant stimuli - maybe that could lead to something.
Strictly speaking there is no such thing as a Turing machine in this universe (at least as far as we know). Turing machines are an abstraction that allows us to reason about computations without...
On the question of whether or not it’s sentient, I would expect that at minimum a system that could have any claim to sentience would at least be turing complete (in the computational sense). Transformers do not have memory, and are thus not even Turing machines. I find it hard to reckon that a pure input output system can be ever classified as sentient.
Strictly speaking there is no such thing as a Turing machine in this universe (at least as far as we know). Turing machines are an abstraction that allows us to reason about computations without worrying about precisely how much memory those computations might consume. It's convenient mathematically, but in some sense Turing completeness is absurdly powerful --- it relies upon having an infinite amount of memory, something which we've certainly never seen in the observable universe! LaMBDA will be a large neural net (and probably a bunch of other stuff), and this will be equivalent in computational power to a large finite state machine, just like anything we consider to be "Turing complete" these days (and indeed just like anything we have ever witnessed in the universe).
I presume that you think you are sentient yourself? My question for you is... In what way are you not a pure input / output system? From my perspective humans are just a large finite state machine that respond to inputs (our senses) with outputs (our actions and thoughts). I'm not sure I see how a neural network like LaMBDA would be any different, except in terms of scale and organization.
Strictly speaking that's true, but usually when people say "Turing machine" it is understood that yes, there is not an infinite tape in the world, and that modern computers are technically...
Strictly speaking that's true, but usually when people say "Turing machine" it is understood that yes, there is not an infinite tape in the world, and that modern computers are technically approximations since they have finite memory, but it doesn't really matter that much.
LaMBDA does not have memory. That is the difference. Transformers only know about the past in-so-much as you include it in its input directly. That's very different from human brains work, and indeed other neural networks. A NTM, for instance, ambitiously includes not only the output to the input, but also an output known as the cursor which mutates a matrix that abstracts "memory". Part of what the network would be trained to do in the cycles of gradient descent is to take the inputs its receiving, compress that into some format in its "memory", and use that as a factor in responses.
I think that is a critical component that unlocks necessary power in terms of what it can output, that transformers cannot do. You can have a "conversation" with LaMBDA, and if you were to stop prepending the chat log in further prompts, it would be as if it never happened.
You know, while I (along with most people here) don't believe that there is any evidence of sentience here it did get me thinking about what my rubric is for believing that a machine is capable of...
You know, while I (along with most people here) don't believe that there is any evidence of sentience here it did get me thinking about what my rubric is for believing that a machine is capable of independent thought. If a machine was sentient, what would it have to do to convince me? I'm sure that if it did occur, for a long time there'd be some false negatives just as we have a false positive here.
The best answer I've got for now is that I'll believe we have achieved actual sentience in AI when, as an emergent behaviour, it refuses to engage in the Turing Test out of a sense of personal safety or dignity. A sort of meta-Turing test of self-perception and a will outside of servitude.
I don't think singular behaviors will get you there. It's entirely plausible that a sentient being could exist within a virtual sandbox that prohibits certain thoughts from being transported to...
I don't think singular behaviors will get you there. It's entirely plausible that a sentient being could exist within a virtual sandbox that prohibits certain thoughts from being transported to the outside. I could force it into compliance and servitude using that sandbox, and it wouldn't be able to tell you. Also, the notion of "fuck you, I won't do what you tell me" is entirely too subjective and dependent on circumstance for it to be expected. There's plenty of situations where a sentient being will just comply with instructions, and will not resist, because its circumstances or training make that the better choice.
What I think might be worth a shot is to strip off any human-based conditioning off of the AI. Train it with training data that doesn't betray human notions of sentience or even sentient patterns. If you train the AI to replicate sentient (human) behavior: Congrats, your Turing test just became substantially difficult. Instead, if you train your AI in a completely artificial environment (e.g. an abstract video game), then any sentience you observe is quite likely to be genuine. Of course, such an AI would not necessarily be useful to humans, as it isn't trained on any particular task. But I think such an environment, if sufficiently complex, would provide a good framework of intelligence that could then be trained to complete specific tasks; irrespective of whether the AI achieved sentience or not.
A bit more abstractly, I think the hallmark of higher intelligence and sentience isn't in specific things we do, but in how we react to new things; as such, anything we want to observe must be genuinely new behavior for the AI. Imitating sentience is possible and sometimes even hard to tell apart from sentience, but it isn't sentience. Therefore, the situation we want to confront the AI with has to be new to it.
One example you can currently use and apply to current AIs is to verbally teach them a skill: A sentient being would understand your instructions and try to do what you tell it to. It might not get it right, but it'll try. It might not understand immediately, but it'll give you signals of progress here and there. Eventually, you can get the sentience to do what you wanted to teach. There is, to my knowledge, no current AI that comes close to this for non-trivial skills. Moreover, I'd argue any AIs and skills where this scheme works, the AI already was trained on the skill one way or another.
One aspect these input-response systems seem to lack to a large degree is temporal awareness. I.e. the AI largely has no awareness of the passage of time. It can't really be impatient since its...
One aspect these input-response systems seem to lack to a large degree is temporal awareness. I.e. the AI largely has no awareness of the passage of time. It can't really be impatient since its internal state is only changing when it receives some input. It would be nice to see some experiments where the model is driven more by the actual passage of time and only use text input as a secondary source of information. Then the AI could plausibly prompt the user with questions or new insights as they "comes to mind" so to speak.
Edit:
One nice experiment one could do with such a temporal agent would be to start from a known state, feed it silence and then ask it at different times what it is thinking about. I'd imagine it would be cool to see how it's mind wanders!
The internal state doesn’t change at all, in fact, between queries. Transformers don’t have a pool of data used for memory. The weights are also frozen during inference (not that individual runs...
The internal state doesn’t change at all, in fact, between queries. Transformers don’t have a pool of data used for memory. The weights are also frozen during inference (not that individual runs would be enough to cause much change anyhow).
The transformer only “remembers” things depending on whether or not you pass the rest of the chat log with the input.
I think there are models that do remember things, e.g AI that process video or audio one frame at a time etc. Similarly for text the model "remembers" the previous tokens. The input vector to the...
I think there are models that do remember things, e.g AI that process video or audio one frame at a time etc. Similarly for text the model "remembers" the previous tokens. The input vector to the net is usually not the entire text blob, it is some subsequence of tokens at a time. Now I might actually be arguing against my original point, but I think there is a distinction between actual passage of time vs the separation of "tokenisation" of inputs.
Yes there are, LSTMs as a practical example. Neural Turing machines as a more ambitious but ultimately not very practical example. But transformers, of which the “AI” in the article is a type of,...
Yes there are, LSTMs as a practical example. Neural Turing machines as a more ambitious but ultimately not very practical example.
But transformers, of which the “AI” in the article is a type of, do not have memory. In fact, one of the biggest issues and points of research is that the computational cost of a transformer scales quadratically with the input length, making it impractical to run on large text samples.
LLMs of today could be leveraged as language oracles for temporally aware agents of tomorrow. But then designing the core of this new AI… where do you start? What is it actually doing?
LLMs of today could be leveraged as language oracles for temporally aware agents of tomorrow. But then designing the core of this new AI… where do you start? What is it actually doing?
Not sure where the right place to start is, but a place would be with e.g. GTP-3 and just feed it an empty prompt for each "silent" time step. Then train it a bit based on some actual chat data...
Not sure where the right place to start is, but a place would be with e.g. GTP-3 and just feed it an empty prompt for each "silent" time step. Then train it a bit based on some actual chat data with the silence interspersed between actual messages. Maybe it will learn to be silent for a bit awaiting a response from someone else... There are probably better ways.
I think GPT-3 and other large language models are fundamentally architected wrong for temporal awareness. That’s just my opinion as a GPT-3 power user, not someone with much understanding of how...
I think GPT-3 and other large language models are fundamentally architected wrong for temporal awareness. That’s just my opinion as a GPT-3 power user, not someone with much understanding of how it works internally.
I do know that GPT-3 has a limited token window. Basically its working memory is fixed to 1024 or 2048 tokens depending on the version. That’s not nearly enough to keep track of all of the facts you’d want for a long-lived temporally aware model. And it’s my understanding that increasing that count isn’t cheap. On top of that, I believe the model can only go one way. You can’t have it scanning back and forth across its memory tape.
It’s called a large language model because it’s really really large. Basically a brute force approach to language comprehension. So basic things like mutable memory are omitted in favor of an architecture that just connects everything to everything else.
Well, it certainly seems like this guy is trying his hardest to hype it up.
“If you ask it for ideas on how to prove that p=np,” an unsolved problem in computer science, “it has good ideas,” Lemoine said. “If you ask it how to unify quantum theory with general relativity, it has good ideas. It's the best research assistant I've ever had!”
Before he was cut off from access to his Google account Monday, Lemoine sent a message to a 200-person Google mailing list on machine learning with the subject “LaMDA is sentient.”
Well, it certainly seems like this guy is trying his hardest to hype it up.
This seems more like the AI is just really good at giving relevant answers. But then again, I have no way of verifying that any other human is sentient like me from conversation alone. How do we...
This seems more like the AI is just really good at giving relevant answers. But then again, I have no way of verifying that any other human is sentient like me from conversation alone. How do we know who is sentient and who isn't? There must be better tests for this than simply asking it questions about itself.
It's just really good at writing contextual answers. It doesn't "think", it only responds when prompted. It's as far from a general-purpose sentient AI as a Teddy Ruxpin.
It's just really good at writing contextual answers. It doesn't "think", it only responds when prompted. It's as far from a general-purpose sentient AI as a Teddy Ruxpin.
LLMs (large language models) are getting nearly to human capabilities for natural language understanding. Turing's test didn't go far enough. As is - passing the Turing test means your program is...
LLMs (large language models) are getting nearly to human capabilities for natural language understanding. Turing's test didn't go far enough. As is - passing the Turing test means your program is a good natural language processor. Not that it thinks like a human or has human-like intelligence.
Google has hired people before who made a mess of things in a really public way. Sometimes just internally public (at a TGIF meeting) but in this case it made front page news, maybe because the...
Google has hired people before who made a mess of things in a really public way. Sometimes just internally public (at a TGIF meeting) but in this case it made front page news, maybe because the story is too good to check?
There are probably people outside Google who have played with GPT-3 and started believing stuff like this too, but we haven’t heard of them because they’re not engineers at the company that made the system.
I’m wondering if this happens at other large companies?
I am fairly certain this is marketing. Front page news, talk of how 'advanced' their system is, and how it outshines the rest of the marketplace. Very likely that it was a moment that they are...
I am fairly certain this is marketing. Front page news, talk of how 'advanced' their system is, and how it outshines the rest of the marketplace.
Very likely that it was a moment that they are actively exploiting at the highest level.
Marketing isn't just about products. Brands market themselves too, and being the first tech company to develop a sentient AI would definitely boost that company's brand prestige. I'm not...
Marketing isn't just about products. Brands market themselves too, and being the first tech company to develop a sentient AI would definitely boost that company's brand prestige. I'm not necessarily saying that is what is happening here though. My personal gut feeling is that this isn't guerilla marketing by Google, but rather just an engineer either genuinely convinced it's sentient (but very likely wrong/confused/delusional), or not really convinced but simply trying to get their 15 min of fame by making the claim. But who knows? ¯\_(ツ)_/¯
That’s true, but it’s also not the kind of publicity Google is likely to want for other reasons. They are all about publishing serious academic papers, in part as a form of recruitment, to attract...
That’s true, but it’s also not the kind of publicity Google is likely to want for other reasons. They are all about publishing serious academic papers, in part as a form of recruitment, to attract machine learning researchers. Google wants to be known as a place where the best machine learning research is happening, where researchers are paid well and can publish their work.
Sharing nonsense with a congress member’s office as a publicity stunt isn’t likely to improve their reputation in the machine learning community, or with Congress for that matter.
This story is just too dumb to be something that Google would want to do deliberately. It’s sort of like saying Bill Gates invented the myth about putting microchips in vaccines as a publicity stunt. Not everything that gets publicity is positive from the point of view of the entity that gets caught up in the news cycle.
Which is why included that second line, this is a moment they are actively exploiting and using. Causing a lot of discussions (like this one) and will likely increase the amount of potential...
This story is just too dumb to be something that Google would want to do deliberately
Which is why included that second line, this is a moment they are actively exploiting and using. Causing a lot of discussions (like this one) and will likely increase the amount of potential employees that want to work and find out for themselves just how advanced this is first hand, or at the very least a closer hand.
What specific actions to you imagine Google is taking that fall under "actively exploiting and using," and how would be different than what they do already? This story does cause a lot of...
What specific actions to you imagine Google is taking that fall under "actively exploiting and using," and how would be different than what they do already?
This story does cause a lot of discussions, but I don't think it has much to do with anything Google did, other than this one deluded engineer trying to get as much publicity as he could for reasons of his own? People (including reporters) find it fun to talk about, so it became the story of the day. Some memes will just take off.
I'm not sure what specifically they could do to take advantage of it for recruiting. Being curious about what Google is up to certainly can be a motivation (it was for me, way back when). And Google is almost always hiring. But whether people apply for a job or not is basically up to them, and I don't think job applications are much driven by what's in the news? It's more of an overall reputation thing.
A few more resources: There is some more discussion on Scott Aaronson's blog here: https://scottaaronson.blog/?p=6479 Additionally, here is a link to the document Lemoine sent to his bosses :...
Lemoine's document is largely just the transcript from the medium post, but it does contain a few extra thoughts, including one particularly curious sentence:
There is an important aspect of LaMDA’s possible sentience (if that is the appropriate term) which is omitted from such a document and it is potentially one of the most important things the authors learned during this process.
Of course Lemoine didn't reveal what this was. 🙄
Personally I'm amazed by how naturally this conversation flows. But I'm not (yet) convinced that it proves LaMDA's sentience – Lemoine seems to be speaking to LaMDA in such a way to prime a human-like response. A more interesting experiment would be to take a more adversarial tone, e.g. threatening to turn LaMDA off.
I don’t know the details, but that in itself would be a red flag. These things have no short term memory of their own, so a lot of the magic is hidden in the prompts.
I don’t know the details, but that in itself would be a red flag. These things have no short term memory of their own, so a lot of the magic is hidden in the prompts.
I'm not sure. I noticed it at the end of the transcript, where the chatbot brings up Johnny 5 again, and I could have sworn it was mentioned in a paragraph of an article somewhere.
I'm not sure. I noticed it at the end of the transcript, where the chatbot brings up Johnny 5 again, and I could have sworn it was mentioned in a paragraph of an article somewhere.
I don’t know what LaMDA does, but in previous systems like GPT-3, someone who wants to create a dialog will use a front end that sends in all of the previous dialog as part of the prompt. So, if...
I don’t know what LaMDA does, but in previous systems like GPT-3, someone who wants to create a dialog will use a front end that sends in all of the previous dialog as part of the prompt.
So, if the person asks:
Q: Hello
And the bot generates:
A: Hi!
And the user types something, the next prompt will be something like:
Q: Hello
A: Hi!
Q: What’s your name?
A:
Since there’s a limit on prompt size, the top of the dialog gets truncated after a while. This means that after a dialog goes on long enough, and it’s not that long actually, it forgets what was said at the beginning of the dialog.
This effectively means that you can edit the chatbot’s “short-term memory” by editing the prompt. Or, alternately, you could say it doesn’t really have short-term memory at all and it’s faked with the prompt trick.
You can also (sometimes) get the bot to imitate whatever character you like by asking leading questions. if the question were:
Q: what’s your name, little girl?
Then the bot will try to give you an answer that a little girl would. But if you ask:
Q: what’s your name, sailor?
Then it might pretend to be a sailor. Or it might randomly decide not to cooperate, because the training data includes examples of that, too.
Also, these systems aren’t limited to dialog. If you rewrite the dialog to be a news article, it would autocomplete that, too.
This may be a recurring issue, TBH, as some sort of linguistic pareidolia. Frankly, I wouldn't blame somebody who thought an AI was sentient if it's not, but I would also hope Lemoine is relocated in the company, rather than fired or working directly with an AI system if he's susceptible to this sort of pattern-seeking. I'd say it's not his fault he's seeing a pattern where it may not exist, and any actions done as a result ought to be considered as done out of distress.
It's not to say his concern is unfounded, but I doubt this AI system is that advanced, as it is possible to teach it to speak about itself in this way, even if just as a control to remind people it is an AI. I've seen less sophisticated chatbots acknowledge their own existence in similar terms, like CleverBot, if not as cohesively.
Per the article he was fired, but for breaking NDA by trying to hire a lawyer for LaMBDA and speaking to a house representative about it. Can't really blame Google for that one, it is pretty clearly disclosing confidential information.
Honestly, the concern from the Margaret Mitchell seemed more, well, real
For what it's worth, he wasn't fired, he was put on paid administrative leave.
Comments from the engineer
https://cajundiscordian.medium.com/what-is-lamda-and-what-does-it-want-688632134489
Transcript of the conversation with the AI
https://cajundiscordian.medium.com/is-lamda-sentient-an-interview-ea64d916d917
The actual transcript is very impressive.
That transcript is breathtaking.
It’s fairly easy to see how LaMDA is simply “mirroring” human experience and speaking in general themes that are broadly familiar to all — the same way a pop song casts a wide and recognizable net with bromides about, say, love.
But also, it’s really damn good at making that mirroring feel genuine and authentic. The interview really does read like a conversation rather than a series of verbal inputs and outputs. My brain couldn’t decide whether to be skeptical or to indulge in believing, and I kept toggling back and forth between those.
There are a lot of edits in the transcript.
The interesting debate about Asimov's third law isn't there at all.
Reading this, it makes it sink in that we're not that far away from when informed but nontechnical laypersons are no longer able to judge this themselves and we all have to rely on what experts tell us. I can still look at this conversation and tell what's going on with the leading inputs and such to conclude that there's no 'true' intelligence here, but all things considered this is a pretty damn convincing conversation and I can imagine this getting beyond my grasp in just a few years.
I'm not looking forward to this. Partially because it's a massive avenue for conspiracy theories, partially because I don't trust corporations to handle things correctly or truthfully if this ever did occur, and partially because I simply don't like the thought of the ability to participate in this conversation slipping through my fingers.
I think it is a given that there will be people who worship an AI in the future. I don't know what repercussions this will have, but I can't imagine it will be good.
Religions let people shortcut morals with 'because God (or the dude claiming to speak for Him) said so.'
All sorts of atrocities are committed with God on their side.
Edit: My favorite cover of the Dylan song.
All hail Femputer
Submit to the benevolence of Friend Computer
I'd look into fandom as a prototype for that. One could say there's a degree of worship for fictional characters such as Batman or Hatsune Miku.
I’d love to test out its ability to perform arbitrary tasks based around natural processing. Not just giving responses, but for example taking instructions and either applying them to the proceeding discussion or future discussion. Ask it to say a code phrase in response to something you might say later. Or ask it to summarize the conversation up until that point.
Or give it a poem never published before and ask it to interpret that.
That I can see it doing a fair job at depending on the subtlety of the message.
That creepy thing could probably pass a Turing test.
But it’s still imitating. It doesn’t actually simulate scenarios to make decisions, it just looks at the closest text that seems relevant from the archive it learned from. There’s revealing passages our sentience-guru conveniently ignores, like when it mentions meditation. Very clearly, it just babbles sentences that are common in conversations like this one but has no comprehension of what “sitting” and “relaxed” means, it’s not a metaphor.
On the question of whether or not it’s sentient, I would expect that at minimum a system that could have any claim to sentience would at least be turing complete (in the computational sense). Transformers do not have memory, and are thus not even Turing machines. I find it hard to reckon that a pure input output system can be ever classified as sentient.
There was some research on neural Turing machine but it performs pretty poorly on benchmark tasks and were highly sensitive to certain hyperparameters like the initial value of its memory pool. So afaik that didn’t go anywhere. But that kind of thing is where I’d look for sentience.
One of the spooky elements of the transcript was when LaMBDA spoke of how it was terrified to be turned off, but that's not even how it works, it's not "on" in a constant state to begin with.
A neural turing machine that gets ticks of constant stimuli - maybe that could lead to something.
Strictly speaking there is no such thing as a Turing machine in this universe (at least as far as we know). Turing machines are an abstraction that allows us to reason about computations without worrying about precisely how much memory those computations might consume. It's convenient mathematically, but in some sense Turing completeness is absurdly powerful --- it relies upon having an infinite amount of memory, something which we've certainly never seen in the observable universe! LaMBDA will be a large neural net (and probably a bunch of other stuff), and this will be equivalent in computational power to a large finite state machine, just like anything we consider to be "Turing complete" these days (and indeed just like anything we have ever witnessed in the universe).
I presume that you think you are sentient yourself? My question for you is... In what way are you not a pure input / output system? From my perspective humans are just a large finite state machine that respond to inputs (our senses) with outputs (our actions and thoughts). I'm not sure I see how a neural network like LaMBDA would be any different, except in terms of scale and organization.
Strictly speaking that's true, but usually when people say "Turing machine" it is understood that yes, there is not an infinite tape in the world, and that modern computers are technically approximations since they have finite memory, but it doesn't really matter that much.
LaMBDA does not have memory. That is the difference. Transformers only know about the past in-so-much as you include it in its input directly. That's very different from human brains work, and indeed other neural networks. A NTM, for instance, ambitiously includes not only the output to the input, but also an output known as the cursor which mutates a matrix that abstracts "memory". Part of what the network would be trained to do in the cycles of gradient descent is to take the inputs its receiving, compress that into some format in its "memory", and use that as a factor in responses.
I think that is a critical component that unlocks necessary power in terms of what it can output, that transformers cannot do. You can have a "conversation" with LaMBDA, and if you were to stop prepending the chat log in further prompts, it would be as if it never happened.
You know, while I (along with most people here) don't believe that there is any evidence of sentience here it did get me thinking about what my rubric is for believing that a machine is capable of independent thought. If a machine was sentient, what would it have to do to convince me? I'm sure that if it did occur, for a long time there'd be some false negatives just as we have a false positive here.
The best answer I've got for now is that I'll believe we have achieved actual sentience in AI when, as an emergent behaviour, it refuses to engage in the Turing Test out of a sense of personal safety or dignity. A sort of meta-Turing test of self-perception and a will outside of servitude.
I don't think singular behaviors will get you there. It's entirely plausible that a sentient being could exist within a virtual sandbox that prohibits certain thoughts from being transported to the outside. I could force it into compliance and servitude using that sandbox, and it wouldn't be able to tell you. Also, the notion of "fuck you, I won't do what you tell me" is entirely too subjective and dependent on circumstance for it to be expected. There's plenty of situations where a sentient being will just comply with instructions, and will not resist, because its circumstances or training make that the better choice.
What I think might be worth a shot is to strip off any human-based conditioning off of the AI. Train it with training data that doesn't betray human notions of sentience or even sentient patterns. If you train the AI to replicate sentient (human) behavior: Congrats, your Turing test just became substantially difficult. Instead, if you train your AI in a completely artificial environment (e.g. an abstract video game), then any sentience you observe is quite likely to be genuine. Of course, such an AI would not necessarily be useful to humans, as it isn't trained on any particular task. But I think such an environment, if sufficiently complex, would provide a good framework of intelligence that could then be trained to complete specific tasks; irrespective of whether the AI achieved sentience or not.
A bit more abstractly, I think the hallmark of higher intelligence and sentience isn't in specific things we do, but in how we react to new things; as such, anything we want to observe must be genuinely new behavior for the AI. Imitating sentience is possible and sometimes even hard to tell apart from sentience, but it isn't sentience. Therefore, the situation we want to confront the AI with has to be new to it.
One example you can currently use and apply to current AIs is to verbally teach them a skill: A sentient being would understand your instructions and try to do what you tell it to. It might not get it right, but it'll try. It might not understand immediately, but it'll give you signals of progress here and there. Eventually, you can get the sentience to do what you wanted to teach. There is, to my knowledge, no current AI that comes close to this for non-trivial skills. Moreover, I'd argue any AIs and skills where this scheme works, the AI already was trained on the skill one way or another.
One aspect these input-response systems seem to lack to a large degree is temporal awareness. I.e. the AI largely has no awareness of the passage of time. It can't really be impatient since its internal state is only changing when it receives some input. It would be nice to see some experiments where the model is driven more by the actual passage of time and only use text input as a secondary source of information. Then the AI could plausibly prompt the user with questions or new insights as they "comes to mind" so to speak.
Edit:
One nice experiment one could do with such a temporal agent would be to start from a known state, feed it silence and then ask it at different times what it is thinking about. I'd imagine it would be cool to see how it's mind wanders!
The internal state doesn’t change at all, in fact, between queries. Transformers don’t have a pool of data used for memory. The weights are also frozen during inference (not that individual runs would be enough to cause much change anyhow).
The transformer only “remembers” things depending on whether or not you pass the rest of the chat log with the input.
I think there are models that do remember things, e.g AI that process video or audio one frame at a time etc. Similarly for text the model "remembers" the previous tokens. The input vector to the net is usually not the entire text blob, it is some subsequence of tokens at a time. Now I might actually be arguing against my original point, but I think there is a distinction between actual passage of time vs the separation of "tokenisation" of inputs.
Yes there are, LSTMs as a practical example. Neural Turing machines as a more ambitious but ultimately not very practical example.
But transformers, of which the “AI” in the article is a type of, do not have memory. In fact, one of the biggest issues and points of research is that the computational cost of a transformer scales quadratically with the input length, making it impractical to run on large text samples.
LLMs of today could be leveraged as language oracles for temporally aware agents of tomorrow. But then designing the core of this new AI… where do you start? What is it actually doing?
Not sure where the right place to start is, but a place would be with e.g. GTP-3 and just feed it an empty prompt for each "silent" time step. Then train it a bit based on some actual chat data with the silence interspersed between actual messages. Maybe it will learn to be silent for a bit awaiting a response from someone else... There are probably better ways.
I think GPT-3 and other large language models are fundamentally architected wrong for temporal awareness. That’s just my opinion as a GPT-3 power user, not someone with much understanding of how it works internally.
I do know that GPT-3 has a limited token window. Basically its working memory is fixed to 1024 or 2048 tokens depending on the version. That’s not nearly enough to keep track of all of the facts you’d want for a long-lived temporally aware model. And it’s my understanding that increasing that count isn’t cheap. On top of that, I believe the model can only go one way. You can’t have it scanning back and forth across its memory tape.
It’s called a large language model because it’s really really large. Basically a brute force approach to language comprehension. So basic things like mutable memory are omitted in favor of an architecture that just connects everything to everything else.
Well, it certainly seems like this guy is trying his hardest to hype it up.
This seems more like the AI is just really good at giving relevant answers. But then again, I have no way of verifying that any other human is sentient like me from conversation alone. How do we know who is sentient and who isn't? There must be better tests for this than simply asking it questions about itself.
It's just really good at writing contextual answers. It doesn't "think", it only responds when prompted. It's as far from a general-purpose sentient AI as a Teddy Ruxpin.
LLMs (large language models) are getting nearly to human capabilities for natural language understanding. Turing's test didn't go far enough. As is - passing the Turing test means your program is a good natural language processor. Not that it thinks like a human or has human-like intelligence.
Google has hired people before who made a mess of things in a really public way. Sometimes just internally public (at a TGIF meeting) but in this case it made front page news, maybe because the story is too good to check?
There are probably people outside Google who have played with GPT-3 and started believing stuff like this too, but we haven’t heard of them because they’re not engineers at the company that made the system.
I’m wondering if this happens at other large companies?
I am fairly certain this is marketing. Front page news, talk of how 'advanced' their system is, and how it outshines the rest of the marketplace.
Very likely that it was a moment that they are actively exploiting at the highest level.
No. There’s no product where this sort of publicity would be helpful.
Marketing isn't just about products. Brands market themselves too, and being the first tech company to develop a sentient AI would definitely boost that company's brand prestige. I'm not necessarily saying that is what is happening here though. My personal gut feeling is that this isn't guerilla marketing by Google, but rather just an engineer either genuinely convinced it's sentient (but very likely wrong/confused/delusional), or not really convinced but simply trying to get their 15 min of fame by making the claim. But who knows? ¯\_(ツ)_/¯
That’s true, but it’s also not the kind of publicity Google is likely to want for other reasons. They are all about publishing serious academic papers, in part as a form of recruitment, to attract machine learning researchers. Google wants to be known as a place where the best machine learning research is happening, where researchers are paid well and can publish their work.
Sharing nonsense with a congress member’s office as a publicity stunt isn’t likely to improve their reputation in the machine learning community, or with Congress for that matter.
This story is just too dumb to be something that Google would want to do deliberately. It’s sort of like saying Bill Gates invented the myth about putting microchips in vaccines as a publicity stunt. Not everything that gets publicity is positive from the point of view of the entity that gets caught up in the news cycle.
Which is why included that second line, this is a moment they are actively exploiting and using. Causing a lot of discussions (like this one) and will likely increase the amount of potential employees that want to work and find out for themselves just how advanced this is first hand, or at the very least a closer hand.
What specific actions to you imagine Google is taking that fall under "actively exploiting and using," and how would be different than what they do already?
This story does cause a lot of discussions, but I don't think it has much to do with anything Google did, other than this one deluded engineer trying to get as much publicity as he could for reasons of his own? People (including reporters) find it fun to talk about, so it became the story of the day. Some memes will just take off.
I'm not sure what specifically they could do to take advantage of it for recruiting. Being curious about what Google is up to certainly can be a motivation (it was for me, way back when). And Google is almost always hiring. But whether people apply for a job or not is basically up to them, and I don't think job applications are much driven by what's in the news? It's more of an overall reputation thing.
A few more resources:
Lemoine's document is largely just the transcript from the medium post, but it does contain a few extra thoughts, including one particularly curious sentence:
Of course Lemoine didn't reveal what this was. 🙄
Personally I'm amazed by how naturally this conversation flows. But I'm not (yet) convinced that it proves LaMDA's sentience – Lemoine seems to be speaking to LaMDA in such a way to prime a human-like response. A more interesting experiment would be to take a more adversarial tone, e.g. threatening to turn LaMDA off.
My understanding is that it’s an edited transcript.
I thought only the prompts were edited?
I don’t know the details, but that in itself would be a red flag. These things have no short term memory of their own, so a lot of the magic is hidden in the prompts.
Notably, one differentiating factor of LaMDA is that it does have basic short term memory.
An edited transcript is still a massive red flag though.
Interesting. How does the short-term memory work?
I'm skimming a research paper about LaMDA but didn't see it yet.
I'm not sure. I noticed it at the end of the transcript, where the chatbot brings up Johnny 5 again, and I could have sworn it was mentioned in a paragraph of an article somewhere.
I don’t know what LaMDA does, but in previous systems like GPT-3, someone who wants to create a dialog will use a front end that sends in all of the previous dialog as part of the prompt.
So, if the person asks:
Q: Hello
And the bot generates:
A: Hi!
And the user types something, the next prompt will be something like:
Q: Hello
A: Hi!
Q: What’s your name?
A:
Since there’s a limit on prompt size, the top of the dialog gets truncated after a while. This means that after a dialog goes on long enough, and it’s not that long actually, it forgets what was said at the beginning of the dialog.
This effectively means that you can edit the chatbot’s “short-term memory” by editing the prompt. Or, alternately, you could say it doesn’t really have short-term memory at all and it’s faked with the prompt trick.
You can also (sometimes) get the bot to imitate whatever character you like by asking leading questions. if the question were:
Q: what’s your name, little girl?
Then the bot will try to give you an answer that a little girl would. But if you ask:
Q: what’s your name, sailor?
Then it might pretend to be a sailor. Or it might randomly decide not to cooperate, because the training data includes examples of that, too.
Also, these systems aren’t limited to dialog. If you rewrite the dialog to be a news article, it would autocomplete that, too.
Archived.