I agree. AI is a useful umbrella term for machine learning, LLMs, diffusion, GANs, and all other manner of technologies. It's not useful to "correct" somebody by insisting that it only refers to...
I agree. AI is a useful umbrella term for machine learning, LLMs, diffusion, GANs, and all other manner of technologies. It's not useful to "correct" somebody by insisting that it only refers to pure AGI, since that usage isn't historically accurate or useful.
Additionally, the actual definition of AGI seems to keep changing as we break down what that even means. Is it intelligence, sapience, sentience, or consciousness? Certainly these LLMs are intelligent - they know tons! They appear to be able to reason - ask them to reconsider something and they'll come up with a different answer. We can excuse this as a simple trick - and it is! - but are we so sure that human reasoning isn't a fancy trick, too?
LLMs are not sentient, and they are not conscious. They do not exist except for when processing the next token in a program, and they don't have the long-term memory required for growth. Of course that may change in the future with on demand fine-tuning or through some other technological method. Similarly they could eventually be wired up to interact with the world in a more real way, adapting inputs and outputs and learning about their environment. This still doesn't necessarily make them AGI, but it sure would make us reconsider our definitions once again.
It's a very cool technology, and I'd be hard pressed to call it anything other than AI. And I'm a bit surprised at the conviction by which some oppose the term. We've been calling enemy soldiers "AI" in gaming for twenty years and nobody has minded. This seems much closer to the goal!
In not opposed to the term AI; however I do think it is worth defining, especially in discussion with lay persons. I've been working with neural networks, convex analysis, etc, for going on twenty...
In not opposed to the term AI; however I do think it is worth defining, especially in discussion with lay persons. I've been working with neural networks, convex analysis, etc, for going on twenty years, so I've often been too close to the field to really be surprised by what is going on. And I think that much like people unfamiliar with programming are bad at understanding what is a hard versus an easy task, people unfamiliar with deep learning struggle to understand what is a break through, versus what is the logical conclusion of throwing compute power at a model.
LLMs are neat, but I see people, even in academia rushing to solve absolutely inappropriate problems using LLMs. So I do think it is worth breaking down for people that just because there has been this new functional leap doesn't mean this is a general purpose reasoning system, or even a significant stride in that direction.
I also don't think it is correct to say that LLMs are intelligent or have intelligence. I e., the ability to acquire knowledge or skills and apply them. These transformer networks are simple probabilistic engines working on tokenized text. They can't acquire knowledge because they can't do anything; they have no sense of self, or consciousness, sapience, or even sensory loop. They are complex functions with inputs and outputs, but even the so called emergent skills such as coding is really just the manifestation of implicit variables defined from training that have overlap with other domains, coupled with ingesting commingled training data.
Which isn't to say that all of this isn't cool, or helpful, or scary. But even internal OpenAI emails showed that the researchers didn't expect as big a reaction as they got when they released it, because to them the real breakthrough of generalized reasoning is still around the corner. And I think that is an important distinction to make when people start trying to have an LLM perform medical research.
We don't know how they work. We know how we trained them. It's more accurate to say an LLM is grown, rather than made. Maybe on the inside they are simple probabilistic engines, or maybe they have...
These transformer networks are simple probabilistic engines working on tokenized text.
We don't know how they work. We know how we trained them. It's more accurate to say an LLM is grown, rather than made. Maybe on the inside they are simple probabilistic engines, or maybe they have a more complex world model, or maybe they do any number of other things.
They can't acquire knowledge because they can't do anything
They can do things.
they have no sense of self, or consciousness, sapience,
These are all subjective things that are impossible to measure.
sensory loop
LLMs do have a rudimentary sensory loop in the sense that their output gets fed back into them.
We know a great deal about how they work. What we can't do is trace an input to an output. And yes, we do veer into the philosophical, but that doesn't mean we just throw our hands up and say it...
We know a great deal about how they work. What we can't do is trace an input to an output. And yes, we do veer into the philosophical, but that doesn't mean we just throw our hands up and say it is unknowable or indescribable. Philosophy has centuries of work on the subject of consciousness, and we can borrow from that. They can't do things because they lack agency. They are directed systems. And they don't learn from there output being fed back in. That is the process of building the output token by token to allow for localized weighting.
Source? It's my understanding we understand the 'architecture', but very little about how it actually accomplishes the goals we train it to do. They can do things with little to no direction, they...
We know a great deal about how they work
Source? It's my understanding we understand the 'architecture', but very little about how it actually accomplishes the goals we train it to do.
They can't do things because they lack agency
They can do things with little to no direction, they may not do them well but I think the level of quality is different from being entirely unable.
Philosophy has centuries of work on the subject of consciousness
But very little work dealing with massive neural networks and their consciousness or lack thereof.
And they don't learn from there output being fed back in
They do learn (to some extent) within the context window, continuous learning is definitely an area of research though.
Unfortunately it is late and I need to go to sleep so this will be shorter than I would like, but I'll reply again in the morning. When I say they have no agency, I mean it in the sense that they...
Unfortunately it is late and I need to go to sleep so this will be shorter than I would like, but I'll reply again in the morning.
When I say they have no agency, I mean it in the sense that they can't initiate anything, or want or prefer anything. I would look at the Belmont report as a good starting point on the ethics of agency.
As far as how they work, we know tremendous amounts about the neural networks that compose them, how learning works, the algorithmic implicit differentiation, and how the learning works. What we don't have a good handle on is the what or the why of the tipping points in the number of parameters of these networks. Why does a 13 billion parameter network behave so much differently than a 70 billion parameter network. It isn't that we know everything, but we know a great deal.
For example, we know enough about them that we can perform targeted knowledge injection them. I don't have a ready source that says "we know everything about them" but I would happily provide sources or references to specific questions about their function.
And regarding philosophy, that gives us the framework to assess anything that might present as sentient. Philosophy is nothing of not good at looking at generalized cases and hyper specific. I would look at "platos zombie" and "can machines think" by Daniel Dennet as a starting point. We have decades, centuries of people asking these types of questions.
First, I don't think agency is a necessary condition for intelligence, so I don't think this would really prove it one way or the other, but no intelligences are entirely self-bootstrapped. Human...
When I say they have no agency, I mean it in the sense that they can't initiate anything, or want or prefer anything. I would look at the Belmont report as a good starting point on the ethics of agency.
First, I don't think agency is a necessary condition for intelligence, so I don't think this would really prove it one way or the other, but no intelligences are entirely self-bootstrapped. Human intelligence is fed 'input data' through it's genetics, evolution, and upbringing.
As far as how they work, we know tremendous amounts about the neural networks that compose them, how learning works, the algorithmic implicit differentiation, and how the learning works.
These are all things other than how they actually work when running inference. We know quite a bit on how to water them, what humidity they need, and the correct scaffolding to grow optimally, but little on how they actually bloom.
Similarly, techniques like targeted knowledge injection don't work by interfacing with the deep layers of the network, where the actual intelligence may or may not be, it works by interfacing with the input and output layers.
And regarding philosophy, that gives us the framework to assess anything that might present as sentient.
Has philosophy closed in on a consensus for the definition and process for determining sentience? If so I would love to learn more.
I don't think you are discussing in good faith, choosing to pick at my words while ignoring the main points I'm making, but I'll do one more reply. We know exceedingly well how they work. We can...
I don't think you are discussing in good faith, choosing to pick at my words while ignoring the main points I'm making, but I'll do one more reply.
We know exceedingly well how they work. We can make a simple ChatGPT in 60 lines of Python. Training, gradient descent, stochastically perturbing our generations, and auto regression are perfectly well understood from a theory and practice perspective.
Further, when the model is running, we do understand what is happening and in general why it works. I could go on and on about how I conceptualize it (lattice of connected vertices and nodes with weights whose relations and structure represent latent variables in the training data against a loss function, and whose invocation is like a series of pulses of computation through the network), I could draw pictures, and diagrams, but all of that is available readily on Google from better sources and besides the point I'm trying to make. Just because we can't trace a decision and what weights went into it doesn't mean we don't have conceptual understanding of how it is functioning. Plus, you haven't indicated you've looked at any of my sources so far.
What we don't understand well is why, at a certain point we no longer need to fine tune these models for specific tasks. Is it because there is some measure of the data domain (like a language) that can only encapsulate so much information, and so once you are able to encode enough latent variables you've more or less got everything you can get, and the rest is setting parameters and tweaking (fine tuning)? We don't know, and that is a deep question.
Finally, with respect to intelligence. The dictionary defines it as "the ability to acquire and apply knowledge and skills." I take the view that an LLM, being absolutely inert when sitting on its own, is not capable of this.
An LLM doesn't acquire knowledge. We train it. We configure it. We fine tune it.
An LLM doesn't apply skills. We invoke it, feeding it inputs and it provides outputs. It can't extrapolate beyond its training and configuration.
It has no form of agency that would be recognizable in any scenario I'm aware of. Even if we loaded it up and plugged it into cameras, etc, it could only provide the outputs its been configured for.
I am a member of a medical IRB, which is partly why I suggested the Belmont report as a good starting point on the ethics of agency. I work with questions of detecting and protecting agency on a weekly basis. And an LLM doesn't meet the threshold yet.
As a final note, I personally believe that life enjoys itself in all its forms, and that life is precious, scarce, and worth protecting. I take a broad view of sentience, but I also recognize that it shouldn't be the defining point for ethical protection. Rather, the capacity to suffer and be protected from suffering should guide what protections we have in place for creatures.
I believe we should be mindful not to create an ethical horror by creating in silica the capacity to suffer or the capacity for agency, and deny it protections. But an LLM, based on all my personal years of experience with deep learning and ethics research, isn't there yet. And I'm not exactly alone on that assessment.
Anyway, I do wish you the best, but I think I am done with this conversation. Plus, I'm at the limit of what I want to type on my phone.
I'm very disheartened to hear that you believe that I've been arguing in bad faith. Since you stated you don't wish to continue the argument, I'll end this with what I believe is the fundamental...
I'm very disheartened to hear that you believe that I've been arguing in bad faith.
Since you stated you don't wish to continue the argument, I'll end this with what I believe is the fundamental disagreement I have: AI as a field cannot distill what AlphaGo does into a set of strategies that can then be applied to a traditional Go bot (except, maybe, by treating it as a black box). We can't look at the weights and biases to see how it models the Go board, or how it identifies patterns. I believe this represents a fundamental lack of understanding of these deep neural networks.
It concerns me greatly when people say we do understand them, because it is like someone saying we do understand how the brain (the brain of any species) works. We may understand the lower level structures, and the very highest level, but we have a tenuous grip on the middle level, where the thinking happens.
This is a hard thing to debate because the terminology is confusing. In some ways, yes, we do know exactly how deep neural networks work. We know lots about how to train them, exactly how to run them, what the GPUs are doing, the precise matrix multiplications needed. Understanding these digital neurons doesn't let us play Go though.
This is just a side note tesseractcat, but I wanted to mention that this style of line-by-line responses is a little bit discouraged on Tildes as it can come off as overly argumentative. I'm sure...
This is just a side note tesseractcat, but I wanted to mention that this style of line-by-line responses is a little bit discouraged on Tildes as it can come off as overly argumentative. I'm sure it wasn't your intention, but it can feel like somebody is picking apart your words rather than responding to the main thrust of your argument this way. Especially when most of the responses are themselves terse, or offering a challenge (eg. asking for a source).
That's not to say that you should never quote others. It's still very useful for showing context, especially in a longer piece. But discussions are likely to be more productive when they're used with restraint.
With that logic, how come complex or non-trivial software developed in earlier times like Unix, COBOL, Excel, MySQL, etc. cannot be included under this "umbrella term" of AI? They are also...
It's not useful to "correct" somebody by insisting that it only refers to pure AGI, since that usage isn't historically accurate or useful.
With that logic, how come complex or non-trivial software developed in earlier times like Unix, COBOL, Excel, MySQL, etc. cannot be included under this "umbrella term" of AI? They are also "artificial" or human made and they portray some "intelligence" behavior like scheduling processes, managing memory, spreadsheets, relational data, etc. With this generic definition, any and every software ever developed can be called AI.
As an area of research, it's roughly "trying to get software to do something that usually only people can do" and the goalpost is going to move over time. Many terms are fuzzy. Once it's fully...
As an area of research, it's roughly "trying to get software to do something that usually only people can do" and the goalpost is going to move over time. Many terms are fuzzy.
Once it's fully understood how to do something, it's not research anymore.
Some of it was. Some instances of decision trees were always called AI. Books about AI normally talked about both that and more complex methods. This only became a controversy when something that...
Some of it was. Some instances of decision trees were always called AI. Books about AI normally talked about both that and more complex methods. This only became a controversy when something that kind of mimics the behavior of an "actual" artificial intelligence became known to the mainstream public, but those things were always under the same umbrella term in research.
I think that's exactly why I'd rather have some sort of distinction. Given the fervor on social media, I don't want to me talking/asking about pathfinding algoritms and be hounded by people who...
And I'm a bit surprised at the conviction by which some oppose the term. We've been calling enemy soldiers "AI" in gaming for twenty years and nobody has minded.
I think that's exactly why I'd rather have some sort of distinction. Given the fervor on social media, I don't want to me talking/asking about pathfinding algoritms and be hounded by people who think I'm trying to take artists' jobs away. I don't care what is called who, but I'd rather we pick some sort of line to draw in the sand.
And on the dev end it can be annoying how everything is trying to market itself as AI. code completion in IDEs, Copilot on version control, help menues in content creation apps (I remember when we called them "Wizards"... I hated them back then too. Why do they keep trying to make Clippy work?!), voice assistants, Whatever the heck LinkedIn is doing.
It's all just becoming a mush of a buzzword. It was already vague pre-LLM craze when people say they were "utilizing AI", now I have no idea if I'm walking on a marketing pitch or a legal landmine.
I'm not an expert in this field, so apologies if the question is ignorant. A calculator is able to do operations that I can't, yet I don't consider it artificial intelligence. I don't consider it...
I'm not an expert in this field, so apologies if the question is ignorant. A calculator is able to do operations that I can't, yet I don't consider it artificial intelligence. I don't consider it particularly intelligent at all. What is the difference between a calculator and an LLM, in terms of the word "intelligence"?
A calculator processes numbers, while current AI processes concepts. Ofc these concepts are represented by numbers at bottom, but the abstraction has reached a level of complexity and...
A calculator processes numbers, while current AI processes concepts. Ofc these concepts are represented by numbers at bottom, but the abstraction has reached a level of complexity and sophistication we've never seen before. One can argue over how much these models truly "understand" concepts, but even if they don't the extent that they appear to is definitely a form of artificial intelligence (with emphasis on the "artificial").
I think a possible disconnect is that sci-fi has conditioned us to think of AI as equivalent to sentience, sapience, self-awareness, consciousness, etc. They may overlap in the future, but they're not necessarily the same thing.
LLMs don't process concepts. They can appear to, especially where they have developed so called emergent skills. However, that is due to what is called latent variables in the training data, where...
LLMs don't process concepts. They can appear to, especially where they have developed so called emergent skills. However, that is due to what is called latent variables in the training data, where they probabilistically associate certain token orders with a latent variable, which may also correlate to other token strings on a war that seems intelligent.
E.g., the sentiment of a sentence is the latent variable behind the tokenized string. That sentiment correlates to other tokens, and the LLM is able to combine these tokens having identified the common latent variable.
How is this different from human intelligence? The LLM can determine that two sentences describe sadness, but it can't explain why the subject of the sentence would be sad.
Ask ChatGPT if a woman walking to market to sell her last cow lost it, would she be sad? It will give you a very robotic answer. Then ask questions like? Why does it matter that it was her last cow? Would the cow be happier?
A person, or generalized sentient sapient mind can answer those questions. LLMs can't, not well or not convincingly. Because they don't understand the actual reason for the sentiment and those sorts of questions aren't reflected in their training data.
Yeah, at the end of the day it's just crunching vectors, but the practical effect is that it can process concepts in a useful way. Sort of like how image generators can compose and combine...
Yeah, at the end of the day it's just crunching vectors, but the practical effect is that it can process concepts in a useful way. Sort of like how image generators can compose and combine coherent visual representations based on prompts rather than rote copy-pasting discrete arrangements of pixels like Photoshop. The generation of text might be based on mechanical algorithms and weights, but the result is eerily close to human reasoning, or at least appear close enough to qualify as artificial intelligence in my view (though artificial consciousness is still out of reach).
Ask ChatGPT if a woman walking to market to sell her last cow lost it, would she be sad? It will give you a very robotic answer. Then ask questions like? Why does it matter that it was her last cow? Would the cow be happier?
Idk, I plugged this into ChatGPT 4 and got perfectly reasonable answers. The style is a little dry and clinical, but that's a result of the intensive RLHF tuning, not something inherent to the model. I agree that it doesn't experience sadness or understand it beyond associating the word with a particular network of tokens in a semantic web, but its ability to isolate the idea of sadness in such a way that it can function in sentences and be worked with linguistically is a form of intelligence, I think.
I have to give you that, your conversation went much better than the one I had with ChatGPT, lol. When I asked it gave me a list of things that cows like, such as scratches. I suppose what we...
I have to give you that, your conversation went much better than the one I had with ChatGPT, lol. When I asked it gave me a list of things that cows like, such as scratches.
I suppose what we would need to do is define intelligence. Taking it from the dictionary, the ability to acquire and learn new skills and apply them. I tend to view it as lacking agency to do those things on its own. We train it. We configure it with skills. We apply those skills. However, of you don't have that view then it might satisfy the definition.
What I would say though to my point about the cow; come up with some questions that are less likely variations on a theme like the last of something, and I think you will get further and further from something you interpret as intelligence. I.e. the application of knowledge to something it hasn't seen before. I know when I asked how she could have lost her last cow, it basically gave up telling me it can't answer a hypothetical.
Those are indeed great answers, and got me testing around. I asked the Bing chat/search several sites of questions about "The Old Man and the sea" and "Desert Solitaire." The questions were along...
Those are indeed great answers, and got me testing around.
I asked the Bing chat/search several sites of questions about "The Old Man and the sea" and "Desert Solitaire." The questions were along the lines of
Did the old man get his wish on the old man and the sea?
Went was it important for Abbey to kill the rabbit in desert solitaire?
I found that on the Bing chat, the answers were beyond my expectations. Then I suspected it was bringing me actual search results from other sites, and I was right.
So then I went to ChatGPT which doesn't fetch live web content, and asked it the same questions. It actually did very well with the desert solitaire questions. However, I personally feel it tripped out of the gate for the old man and the sea.
So, without being obstinate 😂, I think these LLMs are getting really really good. But they are still reflections of their data. When they are cut off from copying people on the web, it is possible to coral them in ways that show the limits of what they are and whether they are truly reasoning or not.
That said, I've seen plenty of middle school papers that don't seem to show much ability to extrapolate and apply, soooo... I guess I need to classify tweens as non-generalized algorithms, lol.
Thanks for taking the time to test, it was a pleasure to see and respond!
That is a good question. I'm not who you asked, but to me an LLM is not intelligent as it can't learn or acquire skills and apply them. However, the difference is that the field of AI is working...
That is a good question. I'm not who you asked, but to me an LLM is not intelligent as it can't learn or acquire skills and apply them. However, the difference is that the field of AI is working to create systems that can solve more and more general problems.
A calculator can solve a very finite set of problems given exact inputs and outputs. It is deterministic. An LLM can take more general input and provide variable output, solving a wide array of problems. It will also solve some problems wrong along the way. That is what makes an LLM an AI algorithm versus the calculator.
Still, an LLM is not a generalized reasoning system, and it has no sensory loop, no persistence of thought, or sense of self. It can't extrapolate, and is merely a reflection of the data it has been trained on.
Just want to say that we don't really know enough about consciousness to say this for certain. The only proof we have that anything is conscious is that we experience it subjectively. The only way...
LLMs are not sentient, and they are not conscious.
Just want to say that we don't really know enough about consciousness to say this for certain. The only proof we have that anything is conscious is that we experience it subjectively. The only way we can predict consciousness of other things is then by their similarity to us. LLMs are possibly the most similar artificial creations to humans (although that's not saying much), so therefore have a higher (but still low) chance of being conscious.
In general, when arguing whether a thing exists, or an action has been taken, or whatever, the base assumption in all cases is that the thing/action/whatever does not exist until proven. Proving a...
In general, when arguing whether a thing exists, or an action has been taken, or whatever, the base assumption in all cases is that the thing/action/whatever does not exist until proven. Proving a negative is impossible.
But all that aside, you're arguing that somethingdoes exist within a given program. I'm interested in hearing you define precisely what it is you believe exists.
They said we don't know. They did not say that it does exist. They also said the only proof that it exists is subjective experience. That essentially means it's intangible and ineffable. That's...
They said we don't know. They did not say that it does exist. They also said the only proof that it exists is subjective experience. That essentially means it's intangible and ineffable. That's also a topic that philosophers have struggled with literally forever, so asking a random person to define it specifically is a bit much, I feel.
We can see that humans have the same stuff going on in their brains, and thus can argue that we're all more or less as conscious as each other. This can be extended to anything with a nervous...
We can see that humans have the same stuff going on in their brains, and thus can argue that we're all more or less as conscious as each other. This can be extended to anything with a nervous system. But in general we draw the line at the point where we can reduce the thing's response to their environment all the way to chemical reactions. I'm open to the idea that there will be software someday that can learn and respond independently of programmers doing manual training of it, and at that point, sure, it may have an analogue to consciousness. But as it stands you could give an LLM all the computing time in the world, and it wouldn't do anything for itself. In the absence of any action to attempt to change things, there's not even a shred of evidence that it is experiencing things anymore than a rock would.
You're asking for proof, but then going down paths of potential future results. I say: there will never be proof of lack of consciousness. All we can do is set our basis for inclusion, and then...
You're asking for proof, but then going down paths of potential future results. I say: there will never be proof of lack of consciousness. All we can do is set our basis for inclusion, and then test for it. If you present an explicit testing criteria, that can be determined. As it is, you're just talking about your own dreams about what might be.
By many criteria, plants are sentinent. That doesn't mean we consider them sentinent in the same way we do chickens. As you state, it is virtually impossible to prove they are not. You can only...
By many criteria, plants are sentinent. That doesn't mean we consider them sentinent in the same way we do chickens.
As you state, it is virtually impossible to prove they are not. You can only prove what you can observe.
There is no tangible proof of a god. That doesn't mean there isn't one, but it does mean that until there is, it's pretty reasonable to work under the assumption there isn't one.
Everyone I know has heard of Bloody Mary. Nobody has ever seen her, but as a child, the power of social trust and magical thinking is strong, and it's easy to dismiss it not happening as "I didn't do it right."
My child has been testing various supersticions about making it snow, and I encourage them to test them. They are disappointed when they don't work, but they understand the world just a little bit better each time.
I'm not arguing something does exist, I'm arguing it's impossible to know, and therefore impossible to make a definitive claim of it's nonexistence. You're right, proving a negative is impossible....
I'm not arguing something does exist, I'm arguing it's impossible to know, and therefore impossible to make a definitive claim of it's nonexistence. You're right, proving a negative is impossible. Similarly, if we assume "that the thing/action/whatever does not exist until proven", this is true for all consciousness except for our own subjective experience.
It would be much easier to know if you'd like to define it. The OED defines consciousness as "The state of being awake and aware of one's surroundings." do you agree with that definition? I feel...
It would be much easier to know if you'd like to define it. The OED defines consciousness as "The state of being awake and aware of one's surroundings." do you agree with that definition? I feel pretty confident that we can argue that it's possible to know whether LLMs are conscious, based on that definition.
My main goal with the above post was to counterbalance a tendency I see where people are quick to dismiss any possibility of LLMs having consciousness, not to argue for it. As for my personal...
My main goal with the above post was to counterbalance a tendency I see where people are quick to dismiss any possibility of LLMs having consciousness, not to argue for it. As for my personal beliefs, I lean towards panpsychism, where all systems have some sort of qualia/awareness, even if alien to us. In general I dislike the term consciousness, and prefer qualia. I do think defining consciousness in terms of awareness is a bit circular.
But any attempt to counterbalance the tendency of people to dismiss the possibility of LLMs having consciousness is inherently an attempt to argue for it. Outside of some very particular arguments...
But any attempt to counterbalance the tendency of people to dismiss the possibility of LLMs having consciousness is inherently an attempt to argue for it. Outside of some very particular arguments no one argues that truly impossible things exist.
If you believe that all systems have some sort of awareness, what then? If the Excel sheet I use to calculate returns on investment for an obscure online game has qualia to some degree, should I feel more or less bad when I delete it than when I swat a mosquito? Knowing that qualia is unique to the experiencer's experience, what do you even do with that belief in terms of actions, aside from discussing things on the internet?
That's a very insightful view that reminds me of the way agency is handled in the Belmont report. I would give it a read; it is about medical ethics and in preserving am individuals agency.
That's a very insightful view that reminds me of the way agency is handled in the Belmont report. I would give it a read; it is about medical ethics and in preserving am individuals agency.
From a classical computer science point of view, there's a different reason why I "think"* you shouldn't call LLMs, or things in this class, as a whole "AI". AI actually does have a definition in...
From a classical computer science point of view, there's a different reason why I "think"* you shouldn't call LLMs, or things in this class, as a whole "AI". AI actually does have a definition in CS, and it's not about AGI - it's about rational agents in an environment.
Notably, this describes the what, not the how, whereas this clumping of various machine learning techniques into "AI" is the reverse, it describes the how, not the what.
Does a Pacman bot that uses A* count as AI? Traditionally, yes! Because it's powering a rational, that is, reward maximizing, agent in an environment, the game of pacman.
Is mapping software that uses A* AI, then? No, because there's no agent nor an environment.
Do LLMs count as AI? Depends on what it's doing! Is it powering something that could be described as a reward maximizing agent in an environment? Then yes, it would be AI. If it's not, then it's not an AI.
* despite my objections, the path of natural language is to evolve whether or not any particular person wants it to evolve in a particular way. So it is futile, and I don't really care at this point. But this article mainly is taking a layman argument as to why you shouldn't call chatGPT an "AI".
The author concedes that "AI" is misleading... and then basically says people need to just get over it. If you know these systems are not intelligent, then why defend calling them intelligent?
The author concedes that "AI" is misleading... and then basically says people need to just get over it.
If you know these systems are not intelligent, then why defend calling them intelligent?
His bio on Twitter says he's the co-creator of Django and an entrepreneur in San Francisco, so he's probably asking us to get over it because it's easier for him to sell ideas when the language is...
His bio on Twitter says he's the co-creator of Django and an entrepreneur in San Francisco, so he's probably asking us to get over it because it's easier for him to sell ideas when the language is muddy.
But, the other answer is that people don't like to think. I'm sure that in his work he's described things using the proper terms and been met with blank stares leading him to realize that his clients just wanna call it Kleenex. The boss man heard about AI and wants that, you're telling me about "ell ell em?" No, I need AI. So he's telling all us nerds that we've just gotta get comfortable calling it AI because the rubes are just gonna tune us out if we don't.
This is the thing we have to fight back against: we need to help people overcome their science fiction priors, understand exactly what modern AI systems are capable of, how they can be used responsibly and what their limitations are.
I don’t think refusing to use the term AI is an effective way for us to do that.
He's basically saying we need to teach people to distrust AI and let them know that all of it is AI instead of telling them of the pit falls off LLMs and other techno babble.
Because the term AI has been used in computer science for decades and it has never practically meant actual inteligence, though hypothetically it would include it as well. Paradoxically, the...
If you know these systems are not intelligent, then why defend calling them intelligent?
Because the term AI has been used in computer science for decades and it has never practically meant actual inteligence, though hypothetically it would include it as well. Paradoxically, the reason why now there's a backlash against it is because we finally have something that can sort of mimic actual intelligence, nobody cared before that.
I agree. AI is a useful umbrella term for machine learning, LLMs, diffusion, GANs, and all other manner of technologies. It's not useful to "correct" somebody by insisting that it only refers to pure AGI, since that usage isn't historically accurate or useful.
Additionally, the actual definition of AGI seems to keep changing as we break down what that even means. Is it intelligence, sapience, sentience, or consciousness? Certainly these LLMs are intelligent - they know tons! They appear to be able to reason - ask them to reconsider something and they'll come up with a different answer. We can excuse this as a simple trick - and it is! - but are we so sure that human reasoning isn't a fancy trick, too?
LLMs are not sentient, and they are not conscious. They do not exist except for when processing the next token in a program, and they don't have the long-term memory required for growth. Of course that may change in the future with on demand fine-tuning or through some other technological method. Similarly they could eventually be wired up to interact with the world in a more real way, adapting inputs and outputs and learning about their environment. This still doesn't necessarily make them AGI, but it sure would make us reconsider our definitions once again.
It's a very cool technology, and I'd be hard pressed to call it anything other than AI. And I'm a bit surprised at the conviction by which some oppose the term. We've been calling enemy soldiers "AI" in gaming for twenty years and nobody has minded. This seems much closer to the goal!
In not opposed to the term AI; however I do think it is worth defining, especially in discussion with lay persons. I've been working with neural networks, convex analysis, etc, for going on twenty years, so I've often been too close to the field to really be surprised by what is going on. And I think that much like people unfamiliar with programming are bad at understanding what is a hard versus an easy task, people unfamiliar with deep learning struggle to understand what is a break through, versus what is the logical conclusion of throwing compute power at a model.
LLMs are neat, but I see people, even in academia rushing to solve absolutely inappropriate problems using LLMs. So I do think it is worth breaking down for people that just because there has been this new functional leap doesn't mean this is a general purpose reasoning system, or even a significant stride in that direction.
I also don't think it is correct to say that LLMs are intelligent or have intelligence. I e., the ability to acquire knowledge or skills and apply them. These transformer networks are simple probabilistic engines working on tokenized text. They can't acquire knowledge because they can't do anything; they have no sense of self, or consciousness, sapience, or even sensory loop. They are complex functions with inputs and outputs, but even the so called emergent skills such as coding is really just the manifestation of implicit variables defined from training that have overlap with other domains, coupled with ingesting commingled training data.
Which isn't to say that all of this isn't cool, or helpful, or scary. But even internal OpenAI emails showed that the researchers didn't expect as big a reaction as they got when they released it, because to them the real breakthrough of generalized reasoning is still around the corner. And I think that is an important distinction to make when people start trying to have an LLM perform medical research.
We don't know how they work. We know how we trained them. It's more accurate to say an LLM is grown, rather than made. Maybe on the inside they are simple probabilistic engines, or maybe they have a more complex world model, or maybe they do any number of other things.
They can do things.
These are all subjective things that are impossible to measure.
LLMs do have a rudimentary sensory loop in the sense that their output gets fed back into them.
We know a great deal about how they work. What we can't do is trace an input to an output. And yes, we do veer into the philosophical, but that doesn't mean we just throw our hands up and say it is unknowable or indescribable. Philosophy has centuries of work on the subject of consciousness, and we can borrow from that. They can't do things because they lack agency. They are directed systems. And they don't learn from there output being fed back in. That is the process of building the output token by token to allow for localized weighting.
Source? It's my understanding we understand the 'architecture', but very little about how it actually accomplishes the goals we train it to do.
They can do things with little to no direction, they may not do them well but I think the level of quality is different from being entirely unable.
But very little work dealing with massive neural networks and their consciousness or lack thereof.
They do learn (to some extent) within the context window, continuous learning is definitely an area of research though.
Unfortunately it is late and I need to go to sleep so this will be shorter than I would like, but I'll reply again in the morning.
When I say they have no agency, I mean it in the sense that they can't initiate anything, or want or prefer anything. I would look at the Belmont report as a good starting point on the ethics of agency.
As far as how they work, we know tremendous amounts about the neural networks that compose them, how learning works, the algorithmic implicit differentiation, and how the learning works. What we don't have a good handle on is the what or the why of the tipping points in the number of parameters of these networks. Why does a 13 billion parameter network behave so much differently than a 70 billion parameter network. It isn't that we know everything, but we know a great deal.
For example, we know enough about them that we can perform targeted knowledge injection them. I don't have a ready source that says "we know everything about them" but I would happily provide sources or references to specific questions about their function.
And regarding philosophy, that gives us the framework to assess anything that might present as sentient. Philosophy is nothing of not good at looking at generalized cases and hyper specific. I would look at "platos zombie" and "can machines think" by Daniel Dennet as a starting point. We have decades, centuries of people asking these types of questions.
First, I don't think agency is a necessary condition for intelligence, so I don't think this would really prove it one way or the other, but no intelligences are entirely self-bootstrapped. Human intelligence is fed 'input data' through it's genetics, evolution, and upbringing.
These are all things other than how they actually work when running inference. We know quite a bit on how to water them, what humidity they need, and the correct scaffolding to grow optimally, but little on how they actually bloom.
Similarly, techniques like targeted knowledge injection don't work by interfacing with the deep layers of the network, where the actual intelligence may or may not be, it works by interfacing with the input and output layers.
Has philosophy closed in on a consensus for the definition and process for determining sentience? If so I would love to learn more.
I don't think you are discussing in good faith, choosing to pick at my words while ignoring the main points I'm making, but I'll do one more reply.
We know exceedingly well how they work. We can make a simple ChatGPT in 60 lines of Python. Training, gradient descent, stochastically perturbing our generations, and auto regression are perfectly well understood from a theory and practice perspective.
Further, when the model is running, we do understand what is happening and in general why it works. I could go on and on about how I conceptualize it (lattice of connected vertices and nodes with weights whose relations and structure represent latent variables in the training data against a loss function, and whose invocation is like a series of pulses of computation through the network), I could draw pictures, and diagrams, but all of that is available readily on Google from better sources and besides the point I'm trying to make. Just because we can't trace a decision and what weights went into it doesn't mean we don't have conceptual understanding of how it is functioning. Plus, you haven't indicated you've looked at any of my sources so far.
What we don't understand well is why, at a certain point we no longer need to fine tune these models for specific tasks. Is it because there is some measure of the data domain (like a language) that can only encapsulate so much information, and so once you are able to encode enough latent variables you've more or less got everything you can get, and the rest is setting parameters and tweaking (fine tuning)? We don't know, and that is a deep question.
Finally, with respect to intelligence. The dictionary defines it as "the ability to acquire and apply knowledge and skills." I take the view that an LLM, being absolutely inert when sitting on its own, is not capable of this.
An LLM doesn't acquire knowledge. We train it. We configure it. We fine tune it.
An LLM doesn't apply skills. We invoke it, feeding it inputs and it provides outputs. It can't extrapolate beyond its training and configuration.
It has no form of agency that would be recognizable in any scenario I'm aware of. Even if we loaded it up and plugged it into cameras, etc, it could only provide the outputs its been configured for.
I am a member of a medical IRB, which is partly why I suggested the Belmont report as a good starting point on the ethics of agency. I work with questions of detecting and protecting agency on a weekly basis. And an LLM doesn't meet the threshold yet.
As a final note, I personally believe that life enjoys itself in all its forms, and that life is precious, scarce, and worth protecting. I take a broad view of sentience, but I also recognize that it shouldn't be the defining point for ethical protection. Rather, the capacity to suffer and be protected from suffering should guide what protections we have in place for creatures.
I believe we should be mindful not to create an ethical horror by creating in silica the capacity to suffer or the capacity for agency, and deny it protections. But an LLM, based on all my personal years of experience with deep learning and ethics research, isn't there yet. And I'm not exactly alone on that assessment.
Anyway, I do wish you the best, but I think I am done with this conversation. Plus, I'm at the limit of what I want to type on my phone.
Have a great day!
I'm very disheartened to hear that you believe that I've been arguing in bad faith.
Since you stated you don't wish to continue the argument, I'll end this with what I believe is the fundamental disagreement I have: AI as a field cannot distill what AlphaGo does into a set of strategies that can then be applied to a traditional Go bot (except, maybe, by treating it as a black box). We can't look at the weights and biases to see how it models the Go board, or how it identifies patterns. I believe this represents a fundamental lack of understanding of these deep neural networks.
It concerns me greatly when people say we do understand them, because it is like someone saying we do understand how the brain (the brain of any species) works. We may understand the lower level structures, and the very highest level, but we have a tenuous grip on the middle level, where the thinking happens.
This is a hard thing to debate because the terminology is confusing. In some ways, yes, we do know exactly how deep neural networks work. We know lots about how to train them, exactly how to run them, what the GPUs are doing, the precise matrix multiplications needed. Understanding these digital neurons doesn't let us play Go though.
This is just a side note tesseractcat, but I wanted to mention that this style of line-by-line responses is a little bit discouraged on Tildes as it can come off as overly argumentative. I'm sure it wasn't your intention, but it can feel like somebody is picking apart your words rather than responding to the main thrust of your argument this way. Especially when most of the responses are themselves terse, or offering a challenge (eg. asking for a source).
That's not to say that you should never quote others. It's still very useful for showing context, especially in a longer piece. But discussions are likely to be more productive when they're used with restraint.
Sorry about that, I think it's a bad habit I got from the 'reply' feature on various chat apps.
With that logic, how come complex or non-trivial software developed in earlier times like Unix, COBOL, Excel, MySQL, etc. cannot be included under this "umbrella term" of AI? They are also "artificial" or human made and they portray some "intelligence" behavior like scheduling processes, managing memory, spreadsheets, relational data, etc. With this generic definition, any and every software ever developed can be called AI.
As an area of research, it's roughly "trying to get software to do something that usually only people can do" and the goalpost is going to move over time. Many terms are fuzzy.
Once it's fully understood how to do something, it's not research anymore.
Some of it was. Some instances of decision trees were always called AI. Books about AI normally talked about both that and more complex methods. This only became a controversy when something that kind of mimics the behavior of an "actual" artificial intelligence became known to the mainstream public, but those things were always under the same umbrella term in research.
I think that's exactly why I'd rather have some sort of distinction. Given the fervor on social media, I don't want to me talking/asking about pathfinding algoritms and be hounded by people who think I'm trying to take artists' jobs away. I don't care what is called who, but I'd rather we pick some sort of line to draw in the sand.
And on the dev end it can be annoying how everything is trying to market itself as AI. code completion in IDEs, Copilot on version control, help menues in content creation apps (I remember when we called them "Wizards"... I hated them back then too. Why do they keep trying to make Clippy work?!), voice assistants, Whatever the heck LinkedIn is doing.
It's all just becoming a mush of a buzzword. It was already vague pre-LLM craze when people say they were "utilizing AI", now I have no idea if I'm walking on a marketing pitch or a legal landmine.
I'm not an expert in this field, so apologies if the question is ignorant. A calculator is able to do operations that I can't, yet I don't consider it artificial intelligence. I don't consider it particularly intelligent at all. What is the difference between a calculator and an LLM, in terms of the word "intelligence"?
A calculator processes numbers, while current AI processes concepts. Ofc these concepts are represented by numbers at bottom, but the abstraction has reached a level of complexity and sophistication we've never seen before. One can argue over how much these models truly "understand" concepts, but even if they don't the extent that they appear to is definitely a form of artificial intelligence (with emphasis on the "artificial").
I think a possible disconnect is that sci-fi has conditioned us to think of AI as equivalent to sentience, sapience, self-awareness, consciousness, etc. They may overlap in the future, but they're not necessarily the same thing.
LLMs don't process concepts. They can appear to, especially where they have developed so called emergent skills. However, that is due to what is called latent variables in the training data, where they probabilistically associate certain token orders with a latent variable, which may also correlate to other token strings on a war that seems intelligent.
E.g., the sentiment of a sentence is the latent variable behind the tokenized string. That sentiment correlates to other tokens, and the LLM is able to combine these tokens having identified the common latent variable.
How is this different from human intelligence? The LLM can determine that two sentences describe sadness, but it can't explain why the subject of the sentence would be sad.
Ask ChatGPT if a woman walking to market to sell her last cow lost it, would she be sad? It will give you a very robotic answer. Then ask questions like? Why does it matter that it was her last cow? Would the cow be happier?
A person, or generalized sentient sapient mind can answer those questions. LLMs can't, not well or not convincingly. Because they don't understand the actual reason for the sentiment and those sorts of questions aren't reflected in their training data.
Yeah, at the end of the day it's just crunching vectors, but the practical effect is that it can process concepts in a useful way. Sort of like how image generators can compose and combine coherent visual representations based on prompts rather than rote copy-pasting discrete arrangements of pixels like Photoshop. The generation of text might be based on mechanical algorithms and weights, but the result is eerily close to human reasoning, or at least appear close enough to qualify as artificial intelligence in my view (though artificial consciousness is still out of reach).
Idk, I plugged this into ChatGPT 4 and got perfectly reasonable answers. The style is a little dry and clinical, but that's a result of the intensive RLHF tuning, not something inherent to the model. I agree that it doesn't experience sadness or understand it beyond associating the word with a particular network of tokens in a semantic web, but its ability to isolate the idea of sadness in such a way that it can function in sentences and be worked with linguistically is a form of intelligence, I think.
I have to give you that, your conversation went much better than the one I had with ChatGPT, lol. When I asked it gave me a list of things that cows like, such as scratches.
I suppose what we would need to do is define intelligence. Taking it from the dictionary, the ability to acquire and learn new skills and apply them. I tend to view it as lacking agency to do those things on its own. We train it. We configure it with skills. We apply those skills. However, of you don't have that view then it might satisfy the definition.
What I would say though to my point about the cow; come up with some questions that are less likely variations on a theme like the last of something, and I think you will get further and further from something you interpret as intelligence. I.e. the application of knowledge to something it hasn't seen before. I know when I asked how she could have lost her last cow, it basically gave up telling me it can't answer a hypothetical.
I built a custom philosopher GPT and asked the same series of questions /u/Jordan117 asked. I really think it hit them out of the park: https://chat.openai.com/share/8ace4b47-abcb-44aa-97a8-cb65b2bb02ae
Here is the GPT if you want to play yourself: https://chat.openai.com/g/g-gTq2v5aEZ-sage-mind
Those are indeed great answers, and got me testing around.
I asked the Bing chat/search several sites of questions about "The Old Man and the sea" and "Desert Solitaire." The questions were along the lines of
Did the old man get his wish on the old man and the sea?
Went was it important for Abbey to kill the rabbit in desert solitaire?
I found that on the Bing chat, the answers were beyond my expectations. Then I suspected it was bringing me actual search results from other sites, and I was right.
So then I went to ChatGPT which doesn't fetch live web content, and asked it the same questions. It actually did very well with the desert solitaire questions. However, I personally feel it tripped out of the gate for the old man and the sea.
Here is one conversation.
So, without being obstinate 😂, I think these LLMs are getting really really good. But they are still reflections of their data. When they are cut off from copying people on the web, it is possible to coral them in ways that show the limits of what they are and whether they are truly reasoning or not.
That said, I've seen plenty of middle school papers that don't seem to show much ability to extrapolate and apply, soooo... I guess I need to classify tweens as non-generalized algorithms, lol.
Thanks for taking the time to test, it was a pleasure to see and respond!
That is a good question. I'm not who you asked, but to me an LLM is not intelligent as it can't learn or acquire skills and apply them. However, the difference is that the field of AI is working to create systems that can solve more and more general problems.
A calculator can solve a very finite set of problems given exact inputs and outputs. It is deterministic. An LLM can take more general input and provide variable output, solving a wide array of problems. It will also solve some problems wrong along the way. That is what makes an LLM an AI algorithm versus the calculator.
Still, an LLM is not a generalized reasoning system, and it has no sensory loop, no persistence of thought, or sense of self. It can't extrapolate, and is merely a reflection of the data it has been trained on.
Just want to say that we don't really know enough about consciousness to say this for certain. The only proof we have that anything is conscious is that we experience it subjectively. The only way we can predict consciousness of other things is then by their similarity to us. LLMs are possibly the most similar artificial creations to humans (although that's not saying much), so therefore have a higher (but still low) chance of being conscious.
I would be interested in discussing this with you. Please precisely define consciousness before we start.
The person I responded to is the one making the definitive claim of lack of consciousness, so they should be the one to precisely define it.
In general, when arguing whether a thing exists, or an action has been taken, or whatever, the base assumption in all cases is that the thing/action/whatever does not exist until proven. Proving a negative is impossible.
But all that aside, you're arguing that something does exist within a given program. I'm interested in hearing you define precisely what it is you believe exists.
They said we don't know. They did not say that it does exist. They also said the only proof that it exists is subjective experience. That essentially means it's intangible and ineffable. That's also a topic that philosophers have struggled with literally forever, so asking a random person to define it specifically is a bit much, I feel.
We can see that humans have the same stuff going on in their brains, and thus can argue that we're all more or less as conscious as each other. This can be extended to anything with a nervous system. But in general we draw the line at the point where we can reduce the thing's response to their environment all the way to chemical reactions. I'm open to the idea that there will be software someday that can learn and respond independently of programmers doing manual training of it, and at that point, sure, it may have an analogue to consciousness. But as it stands you could give an LLM all the computing time in the world, and it wouldn't do anything for itself. In the absence of any action to attempt to change things, there's not even a shred of evidence that it is experiencing things anymore than a rock would.
You're asking for proof, but then going down paths of potential future results. I say: there will never be proof of lack of consciousness. All we can do is set our basis for inclusion, and then test for it. If you present an explicit testing criteria, that can be determined. As it is, you're just talking about your own dreams about what might be.
By many criteria, plants are sentinent. That doesn't mean we consider them sentinent in the same way we do chickens.
As you state, it is virtually impossible to prove they are not. You can only prove what you can observe.
There is no tangible proof of a god. That doesn't mean there isn't one, but it does mean that until there is, it's pretty reasonable to work under the assumption there isn't one.
Everyone I know has heard of Bloody Mary. Nobody has ever seen her, but as a child, the power of social trust and magical thinking is strong, and it's easy to dismiss it not happening as "I didn't do it right."
My child has been testing various supersticions about making it snow, and I encourage them to test them. They are disappointed when they don't work, but they understand the world just a little bit better each time.
I'm not arguing something does exist, I'm arguing it's impossible to know, and therefore impossible to make a definitive claim of it's nonexistence. You're right, proving a negative is impossible. Similarly, if we assume "that the thing/action/whatever does not exist until proven", this is true for all consciousness except for our own subjective experience.
It would be much easier to know if you'd like to define it. The OED defines consciousness as "The state of being awake and aware of one's surroundings." do you agree with that definition? I feel pretty confident that we can argue that it's possible to know whether LLMs are conscious, based on that definition.
My main goal with the above post was to counterbalance a tendency I see where people are quick to dismiss any possibility of LLMs having consciousness, not to argue for it. As for my personal beliefs, I lean towards panpsychism, where all systems have some sort of qualia/awareness, even if alien to us. In general I dislike the term consciousness, and prefer qualia. I do think defining consciousness in terms of awareness is a bit circular.
But any attempt to counterbalance the tendency of people to dismiss the possibility of LLMs having consciousness is inherently an attempt to argue for it. Outside of some very particular arguments no one argues that truly impossible things exist.
If you believe that all systems have some sort of awareness, what then? If the Excel sheet I use to calculate returns on investment for an obscure online game has qualia to some degree, should I feel more or less bad when I delete it than when I swat a mosquito? Knowing that qualia is unique to the experiencer's experience, what do you even do with that belief in terms of actions, aside from discussing things on the internet?
That's a very insightful view that reminds me of the way agency is handled in the Belmont report. I would give it a read; it is about medical ethics and in preserving am individuals agency.
From a classical computer science point of view, there's a different reason why I "think"* you shouldn't call LLMs, or things in this class, as a whole "AI". AI actually does have a definition in CS, and it's not about AGI - it's about rational agents in an environment.
Notably, this describes the what, not the how, whereas this clumping of various machine learning techniques into "AI" is the reverse, it describes the how, not the what.
Does a Pacman bot that uses A* count as AI? Traditionally, yes! Because it's powering a rational, that is, reward maximizing, agent in an environment, the game of pacman.
Is mapping software that uses A* AI, then? No, because there's no agent nor an environment.
Do LLMs count as AI? Depends on what it's doing! Is it powering something that could be described as a reward maximizing agent in an environment? Then yes, it would be AI. If it's not, then it's not an AI.
* despite my objections, the path of natural language is to evolve whether or not any particular person wants it to evolve in a particular way. So it is futile, and I don't really care at this point. But this article mainly is taking a layman argument as to why you shouldn't call chatGPT an "AI".
The author concedes that "AI" is misleading... and then basically says people need to just get over it.
If you know these systems are not intelligent, then why defend calling them intelligent?
His bio on Twitter says he's the co-creator of Django and an entrepreneur in San Francisco, so he's probably asking us to get over it because it's easier for him to sell ideas when the language is muddy.
But, the other answer is that people don't like to think. I'm sure that in his work he's described things using the proper terms and been met with blank stares leading him to realize that his clients just wanna call it Kleenex. The boss man heard about AI and wants that, you're telling me about "ell ell em?" No, I need AI. So he's telling all us nerds that we've just gotta get comfortable calling it AI because the rubes are just gonna tune us out if we don't.
He's basically saying we need to teach people to distrust AI and let them know that all of it is AI instead of telling them of the pit falls off LLMs and other techno babble.
Because the term AI has been used in computer science for decades and it has never practically meant actual inteligence, though hypothetically it would include it as well. Paradoxically, the reason why now there's a backlash against it is because we finally have something that can sort of mimic actual intelligence, nobody cared before that.