I was musing about this recently with the recent models becoming more capable. The core of gen AI is the model, which is trained on a massive dataset. To date, gen AI has improved because the models have become larger, more efficient, the data they are trained on has become better and the software/harnesses around them has improved to help query them.

As I see it, surely the bottleneck will soon become the data they are trained on? If we imagine a scenario where a models could consume an infinite amount of training data, and there is no limit to the training time or quality. The sum of human skill/knowledge is the limiting factor. Gen AI should (in theory) never be able to out preform or push the boundary of the sum of humanity at time of training.

Or, counterpoint, is there enough randomness and speed to iterate that gen AI can actually step change and improve if training times/cost were less prohibitive? Most companies/models today will save good output and feed it back into the next iteration, but right now that's taking months. What if that took minutes?

What do you think?

Is gen AI going to take us to general intelligence?
Will gen AI get to a place where it's "intelligence" and reasoning is actually better than the sum of Humanity?

The theses I would like to discuss goes as follows (and I'm paraphrasing): during the last 15 years, low interest rates made billions of dollars easily available to startups. Unfortunately, this huge influx of venture capital has led to no perceivable innovation.

Put cynically, the innovation startups have brought us across the last 15 years can be summarized as (paraphrasing again):

• An illegal hotel chain destroying our cities
• An illegal taxi company exploiting the poor
• Fake money for criminals
• A plagiarism machine/fancy auto-complete

Everything else is either derivative or has failed.

I personally think spaceX has made phenomenal progress and would have probably failed somewhere along the way without cheap loans. There's also some biotech startups (like the mRNA vaccines that won the race to market during covid) doing great things, but often that's just the fruits of 20 years of research coming to fruition.

Every other recent innovation I can think of came from a big player that would have invested in the tech regardless, and almost all of it is "just" incremental improvements on several decades old ideas (I know, that's what progress looks like most of the time).

What do you think? Do you have any counterexamples? Can you think of any big tech disruptions after quantitative easing made money almost free in 2008?

And if you, like me, feel like we're stuck on a plateau - why do you think that is?