It started with deepseek v3, which rendered the Llama 4 already behind in benchmarks. Adding insult to injury was the "unknown Chinese company with 5..5 million training budget"
Engineers are moving frantically to dissect deepsek and copy anything a...
That’s what OpenAI thought originally when they started working on ChatGPT5, they figured they’d just make the model bigger and it’s going to do more. Turns out that making the model bigger doesn’t actually produce better results. We’re also at a point now where most of the publicly available information has been scraped as well. Now the focus is turning towards improving algorithms for making sense of the data as opposed to just stuffing more data into the model. And this is a problem for Nvidia because current generation of chips is already good enough for doing this.
Of course, people will find ways to utilize more processing power as is always the case. But at least in the near term, this is no longer the bottleneck.
Interesting, I was unaware that we had already hit the limits of used data. That makes sense then. I hope that your analysis is correct, it’d be good news
That’s what OpenAI thought originally when they started working on ChatGPT5, they figured they’d just make the model bigger and it’s going to do more. Turns out that making the model bigger doesn’t actually produce better results. We’re also at a point now where most of the publicly available information has been scraped as well. Now the focus is turning towards improving algorithms for making sense of the data as opposed to just stuffing more data into the model. And this is a problem for Nvidia because current generation of chips is already good enough for doing this.
Of course, people will find ways to utilize more processing power as is always the case. But at least in the near term, this is no longer the bottleneck.
Interesting, I was unaware that we had already hit the limits of used data. That makes sense then. I hope that your analysis is correct, it’d be good news