I was musing about this recently with the recent models becoming more capable. The core of gen AI is the model, which is trained on a massive dataset. To date, gen AI has improved because the models have become larger, more efficient, the data they are trained on has become better and the software/harnesses around them has improved to help query them.

As I see it, surely the bottleneck will soon become the data they are trained on? If we imagine a scenario where a models could consume an infinite amount of training data, and there is no limit to the training time or quality. The sum of human skill/knowledge is the limiting factor. Gen AI should (in theory) never be able to out preform or push the boundary of the sum of humanity at time of training.

Or, counterpoint, is there enough randomness and speed to iterate that gen AI can actually step change and improve if training times/cost were less prohibitive? Most companies/models today will save good output and feed it back into the next iteration, but right now that's taking months. What if that took minutes?

What do you think?

Is gen AI going to take us to general intelligence?
Will gen AI get to a place where it's "intelligence" and reasoning is actually better than the sum of Humanity?

I have a friend that is lucky enough to have retired at 40. A year ago he was adamant he'd never work again, having been burnt out from his time at big tech. Back then he was also an absolute AI hater and wouldn't listen to anyone who claimed LLMs were useful for programming.

He finally tried LLMs when Claude Opus 4.6 released and immediately changed his mind in the face of the overwhelming evidence that LLMs can in fact program pretty well. And now with the release of Fable 5 he's giddily creating all sorts of things that would have taken far too long to make prior to AI-accelerated software development. He actually plans to try and found his own business now. He's a very smart guy, so I hope he can make something interesting that people want.

There are a lot of AI doomers and haters. In person I mostly see people doing the same thing they've always done, but now saving time on various tasks. But this is the first time I've seen someone go from grumpy and checked out to giddy and optimistic thanks to LLMs.