Pretty much my experience as well. In the end, it's fancy statistics - LLMs are designed to give an output based on end-user satisfaction, not on factual agreement. That doesn't mean that LLMs and...
Pretty much my experience as well. In the end, it's fancy statistics - LLMs are designed to give an output based on end-user satisfaction, not on factual agreement. That doesn't mean that LLMs and other generative AIs can't be a stepping stone to more generic AIs, but they're far from intelligent. They're designed to have a high probability of appearing intelligent.
Also, I've used LLMs to ask for feedback on cover letters and other writings in the past, and you can really tell it's not made to give feedback in a useful way. Sometimes it did mention a point or two that were useful, but it always (for me at least) starts with a compliment, then give some vague tips such as being more specific about X or Y(which is ironic), and then maybe something actually useful.
Given the billions that have been pushed into it, I'm expecting a lot of investors to lose a whole lot of money over it. Some may get lucky with unexpected, novel applications that people didn't expect - but I highly doubt there will be as much return as people expect.
It will be interesting to see if LLMs statistical behaviour will lead to some pursue it further out of being convinced by these models, due to their outputs favouring satisfactory, and not truthful, 'answers'.
Also
Well, the tech industry just isn’t that good at software. This illusion is, honestly, too clever to have been created intentionally by those making it.
Gotta say, that's a goddamn good roast purely out of how it isn't meant to be a roast.
This is a very good question that nobody really questions that much. OpenAI alone is set to lose $5 billion this year. That's after offsetting their total spend with income. Their main product is...
Given the billions that have been pushed into it
This is a very good question that nobody really questions that much. OpenAI alone is set to lose $5 billion this year. That's after offsetting their total spend with income. Their main product is primarily used to make terrible chat bots.
NOAA's annual budget is on the order of $7 billion a year. And they provide pretty much all of the data for weather reporting in the USA. They fly planes into hurricanes to gather metrics. Our lives often hang in their balance. If they get it wrong, people die.
One of these things provides exponentially more value than the other. It's looking at things through this lens that makes me question the usefulness of LLMs as a whole, beyond just being an academic curiosity.
If you ask me, investors have gotten so much of a grip on our economy that even if it's not AI, we're bound to get of the rails massively at one moment economically, which will once again probably...
If you ask me, investors have gotten so much of a grip on our economy that even if it's not AI, we're bound to get of the rails massively at one moment economically, which will once again probably prove to be difficult to predict in advance exactly how it will look like.
Regardless, it will be interesting to see how we'll deal with the massive infrastructure on AI being built once the cash flow stops.
I believe a lot of the money is going into building out datacenters? I expect the AI bubble will result in a lot of wasted investment as far as investors are concerned, but physically, there’s a...
I believe a lot of the money is going into building out datacenters? I expect the AI bubble will result in a lot of wasted investment as far as investors are concerned, but physically, there’s a fair bit of durable infrastructure that can be repurposed.
A large proportion of the public already assume anything called an "AI" to be perfectly truthful and incapable of mistakes or bias. I predict that scam artists will start to emerge claiming to...
It will be interesting to see if LLMs statistical behaviour will lead to some pursue it further out of being convinced by these models, due to their outputs favouring satisfactory, and not truthful, 'answers'.
A large proportion of the public already assume anything called an "AI" to be perfectly truthful and incapable of mistakes or bias. I predict that scam artists will start to emerge claiming to have spent years engineering some breakthrough, post-singularity AI and either sell access to them at a steep cost or use them as the figurehead of a cult. Some will be doing it for the money and others will believe it. Silicon Valley types are already showing an alarming level of reverence for these things, it would not take much of a push.
I think this is definitely colored by who you're talking to. There are plenty of "true believers" and even more people who aren't particularly knowledgeable but willing to trust the overhyped...
I think this is definitely colored by who you're talking to. There are plenty of "true believers" and even more people who aren't particularly knowledgeable but willing to trust the overhyped marketing copy.
If you ask me we should be glad that social media came first. We are only now really beginning to understand the negative effects of social media, its algorithms, and more. Imagine if we had no...
If you ask me we should be glad that social media came first.
We are only now really beginning to understand the negative effects of social media, its algorithms, and more. Imagine if we had no idea how harmful those kind of things can be before the AI breakthroughs.
Given the tensions in the world, the riots in the UK two weeks ago, mental health echoes of the pandemic and a few other things I'm still expecting violence, at least in the EU, not in the near future. Still, given how progress is far from linear... it's not difficult to see things having taken a worse order to occur.
This was a really good article. Nicely written, detailed, a splash of infographics. This puts a label on the feeling I get when reading LLM responses, and particularly when I'm reading text from...
This was a really good article. Nicely written, detailed, a splash of infographics.
This puts a label on the feeling I get when reading LLM responses, and particularly when I'm reading text from coworkers or students and it trips a flag in my head that says "an LLM wrote this". I'm not unique in this, plenty of other people (professors/teachers in particular) have become adept at spotting LLM responses. What lingered on for me was the feeling "I feel like I've seen this before".
Now I know why. It's the same mental alarm that goes off when listening to a psychic/mentalist make super generic statements, or reading a horoscope. It also goes off when someone is trying to sell me something and they're attempting to make small talk in a really ham-fisted speed-run way. It always indicates to me they don't actually care about the response they're getting, they just want to try and build "rapport" as fast as possible and get me to engage with them.
That's a really great way of describing it. I have the same alarm bells going off. It's like "whoever wrote this did not put any human care into it". Funnily enough for me it didn't ring a bell as...
That's a really great way of describing it. I have the same alarm bells going off. It's like "whoever wrote this did not put any human care into it".
Funnily enough for me it didn't ring a bell as like how a conman would talk to me. For me I always feel like I'm reading canned responses from an underpaid customer service agent.
Like they sortof read what I asked about, and their answer is vaguely helpful maybe, but mostly they just looked through their canned-response-database and picked whatever matched sorta to what I said.
This is a good article about why some people fall hard for AI chat. I’ll just point out that there are plenty of other people who don’t use AI chat that way: Also, for people who use LLM’s to...
This is a good article about why some people fall hard for AI chat. I’ll just point out that there are plenty of other people who don’t use AI chat that way:
Most will just take the first answer and leave, or at most will repeat variations of their prompt until they get the result they wanted. These interactions are purely mechanical. The end-user is treating the chatbot merely as a generative widget, so they never get pulled into the LLMentalist effect.
Also, for people who use LLM’s to build tools, giving it a few examples and asking it to continue the pattern can work pretty well. That’s not asking for creativity, it’s asking it to do a specific text-based task.
I buy the argument that people are fooling themselves into seeing intelligence that isn't there in the output distribution. Was the text right? It must be smart. Was it wrong? Poor GPT must be...
I buy the argument that people are fooling themselves into seeing intelligence that isn't there in the output distribution. Was the text right? It must be smart. Was it wrong? Poor GPT must be having an off day. But really the output of the system is that distribution over text, biased towards things common to see after the preceeding text, and the right answers are often right by luck or by being generic enough to not be wrong, and not because the model somehow actually knew they would be right.
And I buy that RLHF can make this worse. If you polish it into the character of an apparently helpful assistant, you can get it to stay in character and spit out apparently helpful text, and the whole process doesn't add any power that wasn't there before.
But the author uses phrases like "statistical" and "plausible" to make it sound like the LLMs are somehow coming up with text without any internal processing of either text or semantic features of words, which at this point is a bold claim to see without supporting evidence. For example, they write:
the LLM isn’t “reading” your text any more than the psychic is reading your mind. They are giving you statistically plausible responses based on what you say. You’re the one finding ways to validate those responses as being specific to you as the subject of the conversation.
While an LLM doesn't "read" text the way a human would, it absolutely does go through all the text in the context window token by token, whereas a psychic does not take the contents of your brain as input. And the responses are absolutely "specific to" (i.e. statistically conditioned on) the input. There might not be anyone in there talking to you, and the model may have learned a weaker relationship between what goes in and what comes out than one might expect (which in turn might be disguised by the effects the author is talking about), but what comes out is indeed mathematically capable of changing when what goes in changes.
And the ability to produce "statistically plausible responses" is not nothing, and is in many cases impossible without computation or the ability to follow rules. If I ask a psychic whether a Cadillac will fit in my microwave and they come back with the correct answer of "the spirits say don't try it" 87% of the time, which result is repeatable for wide ranges of values of "Cadillac" and "microwave", in combinations never provided before as input, then the psychic-spirits system demonstrably has a nonzero intelligence in the field of what things fit inside each other.
Is that the same as personhood or sentience or trustworthiness? No. But you can't grift a test set.
I don't think the bit you're responding to disagrees on this -- they say the responses are "based on what you say", after all. I think the point they're making here is that much like how a psychic...
And the responses are absolutely "specific to" (i.e. statistically conditioned on) the input.
I don't think the bit you're responding to disagrees on this -- they say the responses are "based on what you say", after all. I think the point they're making here is that much like how a psychic is just responding to the words you say without doing any actual mind-reading or fortune-telling, the LLM is just making responses that seem right based on your input, rather than doing any deeper thinking or feeling about what it says.
All I need to know to not trust current LLM/AI tools is what I see when I give it basic language-based requests, like to provide me a list of words with a few limited criteria on what the words...
All I need to know to not trust current LLM/AI tools is what I see when I give it basic language-based requests, like to provide me a list of words with a few limited criteria on what the words contain (x number of letters; must contain a certain letter, etc). It will give me a list, but some of them don't contain the required letter. I draw its "attention" to that fact, but over and over and over again it keeps giving me the same responses which don't match the criteria. "It seems there was an oversight" yeah no shit.
I would think an LLM would have some grasp on the qualities of language.
Pretty much my experience as well. In the end, it's fancy statistics - LLMs are designed to give an output based on end-user satisfaction, not on factual agreement. That doesn't mean that LLMs and other generative AIs can't be a stepping stone to more generic AIs, but they're far from intelligent. They're designed to have a high probability of appearing intelligent.
Also, I've used LLMs to ask for feedback on cover letters and other writings in the past, and you can really tell it's not made to give feedback in a useful way. Sometimes it did mention a point or two that were useful, but it always (for me at least) starts with a compliment, then give some vague tips such as being more specific about X or Y(which is ironic), and then maybe something actually useful.
Given the billions that have been pushed into it, I'm expecting a lot of investors to lose a whole lot of money over it. Some may get lucky with unexpected, novel applications that people didn't expect - but I highly doubt there will be as much return as people expect.
It will be interesting to see if LLMs statistical behaviour will lead to some pursue it further out of being convinced by these models, due to their outputs favouring satisfactory, and not truthful, 'answers'.
Also
Gotta say, that's a goddamn good roast purely out of how it isn't meant to be a roast.
This is a very good question that nobody really questions that much. OpenAI alone is set to lose $5 billion this year. That's after offsetting their total spend with income. Their main product is primarily used to make terrible chat bots.
NOAA's annual budget is on the order of $7 billion a year. And they provide pretty much all of the data for weather reporting in the USA. They fly planes into hurricanes to gather metrics. Our lives often hang in their balance. If they get it wrong, people die.
One of these things provides exponentially more value than the other. It's looking at things through this lens that makes me question the usefulness of LLMs as a whole, beyond just being an academic curiosity.
If you ask me, investors have gotten so much of a grip on our economy that even if it's not AI, we're bound to get of the rails massively at one moment economically, which will once again probably prove to be difficult to predict in advance exactly how it will look like.
Regardless, it will be interesting to see how we'll deal with the massive infrastructure on AI being built once the cash flow stops.
I believe a lot of the money is going into building out datacenters? I expect the AI bubble will result in a lot of wasted investment as far as investors are concerned, but physically, there’s a fair bit of durable infrastructure that can be repurposed.
A large proportion of the public already assume anything called an "AI" to be perfectly truthful and incapable of mistakes or bias. I predict that scam artists will start to emerge claiming to have spent years engineering some breakthrough, post-singularity AI and either sell access to them at a steep cost or use them as the figurehead of a cult. Some will be doing it for the money and others will believe it. Silicon Valley types are already showing an alarming level of reverence for these things, it would not take much of a push.
I’m wondering how you know this? I’m under the impression that skepticism about AI is widespread.
I think this is definitely colored by who you're talking to. There are plenty of "true believers" and even more people who aren't particularly knowledgeable but willing to trust the overhyped marketing copy.
If you ask me we should be glad that social media came first.
We are only now really beginning to understand the negative effects of social media, its algorithms, and more. Imagine if we had no idea how harmful those kind of things can be before the AI breakthroughs.
Given the tensions in the world, the riots in the UK two weeks ago, mental health echoes of the pandemic and a few other things I'm still expecting violence, at least in the EU, not in the near future. Still, given how progress is far from linear... it's not difficult to see things having taken a worse order to occur.
This was a really good article. Nicely written, detailed, a splash of infographics.
This puts a label on the feeling I get when reading LLM responses, and particularly when I'm reading text from coworkers or students and it trips a flag in my head that says "an LLM wrote this". I'm not unique in this, plenty of other people (professors/teachers in particular) have become adept at spotting LLM responses. What lingered on for me was the feeling "I feel like I've seen this before".
Now I know why. It's the same mental alarm that goes off when listening to a psychic/mentalist make super generic statements, or reading a horoscope. It also goes off when someone is trying to sell me something and they're attempting to make small talk in a really ham-fisted speed-run way. It always indicates to me they don't actually care about the response they're getting, they just want to try and build "rapport" as fast as possible and get me to engage with them.
That's a really great way of describing it. I have the same alarm bells going off. It's like "whoever wrote this did not put any human care into it".
Funnily enough for me it didn't ring a bell as like how a conman would talk to me. For me I always feel like I'm reading canned responses from an underpaid customer service agent.
Like they sortof read what I asked about, and their answer is vaguely helpful maybe, but mostly they just looked through their canned-response-database and picked whatever matched sorta to what I said.
This is a good article about why some people fall hard for AI chat. I’ll just point out that there are plenty of other people who don’t use AI chat that way:
Also, for people who use LLM’s to build tools, giving it a few examples and asking it to continue the pattern can work pretty well. That’s not asking for creativity, it’s asking it to do a specific text-based task.
I buy the argument that people are fooling themselves into seeing intelligence that isn't there in the output distribution. Was the text right? It must be smart. Was it wrong? Poor GPT must be having an off day. But really the output of the system is that distribution over text, biased towards things common to see after the preceeding text, and the right answers are often right by luck or by being generic enough to not be wrong, and not because the model somehow actually knew they would be right.
And I buy that RLHF can make this worse. If you polish it into the character of an apparently helpful assistant, you can get it to stay in character and spit out apparently helpful text, and the whole process doesn't add any power that wasn't there before.
But the author uses phrases like "statistical" and "plausible" to make it sound like the LLMs are somehow coming up with text without any internal processing of either text or semantic features of words, which at this point is a bold claim to see without supporting evidence. For example, they write:
While an LLM doesn't "read" text the way a human would, it absolutely does go through all the text in the context window token by token, whereas a psychic does not take the contents of your brain as input. And the responses are absolutely "specific to" (i.e. statistically conditioned on) the input. There might not be anyone in there talking to you, and the model may have learned a weaker relationship between what goes in and what comes out than one might expect (which in turn might be disguised by the effects the author is talking about), but what comes out is indeed mathematically capable of changing when what goes in changes.
And the ability to produce "statistically plausible responses" is not nothing, and is in many cases impossible without computation or the ability to follow rules. If I ask a psychic whether a Cadillac will fit in my microwave and they come back with the correct answer of "the spirits say don't try it" 87% of the time, which result is repeatable for wide ranges of values of "Cadillac" and "microwave", in combinations never provided before as input, then the psychic-spirits system demonstrably has a nonzero intelligence in the field of what things fit inside each other.
Is that the same as personhood or sentience or trustworthiness? No. But you can't grift a test set.
I don't think the bit you're responding to disagrees on this -- they say the responses are "based on what you say", after all. I think the point they're making here is that much like how a psychic is just responding to the words you say without doing any actual mind-reading or fortune-telling, the LLM is just making responses that seem right based on your input, rather than doing any deeper thinking or feeling about what it says.
All I need to know to not trust current LLM/AI tools is what I see when I give it basic language-based requests, like to provide me a list of words with a few limited criteria on what the words contain (x number of letters; must contain a certain letter, etc). It will give me a list, but some of them don't contain the required letter. I draw its "attention" to that fact, but over and over and over again it keeps giving me the same responses which don't match the criteria. "It seems there was an oversight" yeah no shit.
I would think an LLM would have some grasp on the qualities of language.