Does gen AI have a natural limit without a major innovation?
I was musing about this recently with the recent models becoming more capable. The core of gen AI is the model, which is trained on a massive dataset. To date, gen AI has improved because the models have become larger, more efficient, the data they are trained on has become better and the software/harnesses around them has improved to help query them.
As I see it, surely the bottleneck will soon become the data they are trained on? If we imagine a scenario where a models could consume an infinite amount of training data, and there is no limit to the training time or quality. The sum of human skill/knowledge is the limiting factor. Gen AI should (in theory) never be able to out preform or push the boundary of the sum of humanity at time of training.
Or, counterpoint, is there enough randomness and speed to iterate that gen AI can actually step change and improve if training times/cost were less prohibitive? Most companies/models today will save good output and feed it back into the next iteration, but right now that's taking months. What if that took minutes?
What do you think?
Is gen AI going to take us to general intelligence?
Will gen AI get to a place where it's "intelligence" and reasoning is actually better than the sum of Humanity?
I'm far from an LLM researcher, but I work with AI on the daily and have deployed multiple applications using models from all major frontier labs.
I don't think LLMs will get us to AGI, it might get very close but I do think that always a sufficiently trained human will outperform the very best LLM there is, in a given task.
Where LLMs will outperform us (and maybe already do for some narrow cases) though, is in cross-functional tasks across multiple domains, since they can distill knowledge from so many fields at once while people generally specialize in one or two fields.
I do think we need a breakthrough in model architecture to get past this barrier. It will probably happen eventually, but not right now.
It's interesting to see the attempts at recursive self-improvement and how they develop in the future, though.
Sensible take.
It does seem that gen AI is getting better at having more, and more broad models. Way back when the hype started in like 2023/2024 I wondered if we'd see extremely good, but specialised models. Maybe they could talk to each other, or they could work together in some environment. But seems they've gone the general route and it's actually working out reasonably well.
Re self improvement, I'm sure it's being attempted. It's got to be in similar veins to reinforcement learning where they give the models a reward metric. But it must be slow as hell right now with how expensive training is.
No. That's not an an end state for LLM/Generative AI. The AI companies want you to think that it is, but they're honestly just using marketing smoke and mirrors to talk about how cool their products are.
The first hurdle is for everyone to agree on what "intelligence" is. What does it mean? Clearly, us awesome humans have it, but are there other species that have it? I could go and write a doctoral thesis level post on this, but the problem I wish to highlight is that we don't have an agreed upon definition on what intelligence is.
The second hurdle is that LLMs are a prediction model. It can only spit out responses based on what what it was trained to do. Most LLMs are trained with whatever they could get their hands on (acquired legal or illegal) but they don't come up with new ideas. Every response is a prediction based on what they expect the answer to be. There is no "logic" or "reasoning" with this on the part of the software (again, things we have no clear philosophical definitions). To these AI companies credit, they tuned the training process so well that they give impressive answers that appear like intelligence to the layperson.
The third and final hurdle understanding what the achievement state of a "general intelligence" is. Lets take an easy route, and simply say: General Intelligence will be when we can replicate the human brain and how intelligence exists in humans. Well, the challenging part there is that we honestly don't know what that means from a medical position either. We know the high levels: Brains have neurons and neurons talk to one another. (Hence the term you might hear of "Neural net" for various simple models), but we don't understand the how on a human consciousness exists within our brains. It works, we can even see it happening in other animals but it's such a challenging web of chemicals and electrical signals that we don't understand it. Someday we will, but right now we don't and LLMs/Generative AI is not the path there.
To put it simply, no. The best we can hope for is equal to, but that seems unlikely as well. Don't get me wrong, it's likely going to get to like, 95%-99% range but it can never get better than the comparison of where it started. The only reason why I can't confidently give it 100% is that AI as we know it now currently works by guessing what it should respond next. It's gotten DAMN good at guessing, but stuff that we can reason as true (like 2+2=4) is not what LLMs do. It responds that 2+2=4 because it read enough places that this is true. It does not see two rocks in one hand and two rocks in another count to four total rocks when brought together like humans do. When you ask how it got the answer, it will give you a pretty good bit of stuff that looks like reasoning, but it's again just taking the most likely answer. This is why for the longest time LLMs struggled with stuff that's simple for us like "How many instances of the letter r are in the word strawberry?" There are tons of these edge cases and the AI companies try their best to shut these down every time the public loudly discovers one to show that the models are learning and growing.
Now none of this is to disparage LLMs or GenAI. This is a remarkable achievement in computational history. When I started to learn about computers and software as a kid, I wanted to learn about how to do those cool things in science fiction like AI and robots. Turns out, as I got into actually learning about AI, I also learned that the ethics are really REALLY complicated and it didn't take long to see how these models would be abused by everyone to do morally ambiguous things, which is why I ultimately didn't feel comfortable pursuing a career in it.
I hope these answers help!
Have the diminishing returns since GPT 3 not been clearly visible?
As a regular user since about ChatGPT release I don't think so. ChatGPT 4 was a huge step forward, and so was o1, the change to "reasoning" models that are now standard. Since then the gains have been seemingly small, but also I haven't tested any of the frontier models that are hidden behind the higher tier subscriptions or in the case of Anthropic currently paused, and in the grand scheme of things the time since "reasoning" models proliferated has been incredibly short, we're just used to really fast development.
I think the lack of online learning/catastrophic forgetting is a major limitation currently, to be truly smart a model should be able to learn new information in a reliable way, without a finicky and costly separate training procedure. Probably the training procedure where all the data is learned as once is an issue in itself (that's not how we learn).
https://en.wikipedia.org/wiki/Catastrophic_interference
To clarify the vocab: Gen AI = LLM powered agents = LLM fine tuned for reasoning and tool use running in a harness that provides tools and other functionality.
Boiling it down there are two steps:
The above compresses a bunch of important sub steps for brevity.
Innovation can happen in various parts of both steps, so there's still a lot of room for improvement. There are undoubtedly better ways to do everything involved, much of it has been replaced with better methods multiple times already.
Model size is likely to become a limiting factor, both because of the limit of what exists in terms of training data and because bigger models are more computationally expensive to train and to run. But that's assuming better ways of getting, vetting and tagging pre-training data aren't discovered. I'd assume that, yes, eventually there will be a ceiling. In terms of compute, the tech is going to keep getting more efficient and the hardware will keep getting better so likely any limits imposed by compute will be temporary.
Will recursive self improvement hit an event horizon where LLMs will start improving themselves so fast they start rocketing towards AGI? Probably not with the current state of the art. When models generate their own training data they end up entrenching and exaggerating their flaws, and there are a lot of flaws. Some amount of artifical training data is fine (especially if it comes from a better model), but 100% artifical training isn't viable at this point.
Even if LLMs were to achieve the ability to recursively self improve without ensloppifying themselves, there's no room in the math for the kind of awareness or understanding we'd associate with AGI. The models don't have a conceptual understanding of reality, they only appear to. They would need to invent new technology to get there, not just iterate on existing LLM tech.
However, will LLM tools contribute to whatever sort of AGI is someday created? It's hard to imagine they won't.
I can imagine a future world model with pre-training on a much wider dataset that strives to tokenize reality, as opposed to just language and other creative outout, having a more realistic path to AGI. Especially if it was fine tuned with some sort of feedback mechanism that could approximate real world cause and effect. Maybe you'd need sensory feedback. But that's speculating on technology that doesn't exist yet. Right now world models are mostly focused on improving robotics. As far as I know, no one has tried to make a super-sized general world model. It would take the resources of one of the frontier labs to attempt it.
My perspective is that AGI is still roughly comparable to stable fusion power. There's no reason to believe it can't be done, but it will most likely be "just around the corner" for years and years.
Thanks for the detailed reply.
Very interesting take on the "world model" idea, that makes a lot of sense in terms of giving the model some context of the real world as opposed to just our language.
I do agree with the take that gen AI won't lead to general AI but will help pave the way. Though I suspect there will be a lot of media coverage along the way (not that we don't get plenty of it already!) about how gen AI is actually already general AI and has thoughts and feelings.
Definitely interested to hear answers for folks that understand the process better than I do. IMO fable was a genuine generational leap over the previous models (based purely on the scope of the projects I was able to undertake with it compared to Opus and how much more reliable its output was), and my guess is that it was in the post training step.
My very limited understanding is that the biggest models have already been trained on almost all publicly available data so I don’t see big leaps coming from there. Which is why my guess is that Fabel was a post training turning/harness improvement. So I would love to better understand how models will continue to improve and grow more useful going forward.
This is literally a billion dollar question. Nobody really knows. Imo the answers are
Yes, though no idea how different the technology is going to be from the one we have now - it could be just incremental development from LLMs gradually taking us someplace else, not necessarily a paradigm change. I think it can be as little as a decade away, depending on whether/when we manage to get to recursive improvements, using AI to improve itself.
(to be clear this worries me a lot, and I wish it didn't happen, but I think it will)
"sum of Humanity" is very strong, I wouldn't bet on that specifically, though I guess it depends on definition. I think there's a big difference between a theoretical best possible sum of humanity, our potential, and a realistic sum of humanity, humanity that is uncooperative, tribalistic, irrational and full of conflicts. The latter, of course, seems more likely to be beaten.