AI progress may be in decline, warns Google DeepMind’s Demis Hassabis

Demis Hassabis, CEO of Google DeepMind, has warned that the rapid progress in AI development may be slowing as companies exhaust the available digital data needed to train large language models. The industry has long relied on feeding vast amounts of online text into AI systems to improve performance, but diminishing returns are now setting in. Some experts, including OpenAI’s Ilya Sutskever, believe the industry has reached “peak data,” meaning future improvements will require entirely new approaches.

Researchers are now exploring alternative methods, such as synthetic data, where AI models generate and learn from their own outputs. While this technique has shown promise in fields like mathematics and programming, it struggles with more complex areas like philosophy and the arts, where defining correctness is difficult. OpenAI has already applied this method in its latest system, OpenAI o1, but challenges remain, particularly in preventing AI from making errors or generating misleading information.

Another possibility to overcome ‘data limitation’ in AI development is to shift focus from quantity to quality of data through better data labelling and contextual enrichment, as done by Diplo’s cognitive proximity approach (see below).

Tech leaders remain divided on whether AI advancements will continue at the same pace. Nvidia’s CEO Jensen Huang remains optimistic, citing strong demand for AI chips and ongoing innovation. However, some of the company’s biggest customers are preparing for a possible plateau in AI development. Despite the uncertainty, investment in AI infrastructure remains high, with firms continuing to push the boundaries of what AI can achieve.