Beyond scaling: The future of AI algorithms according to OpenAI CEO
OpenAI CEO, Sam Altman, predicts the end of focusing on building giant language models like GPT-3 and GPT-4. The future of AI algorithms will shift towards exploring new model designs and tuning based on human feedback rather than solely scaling up models. This vision contrasts with current trends of investing heavily in larger algorithms and highlights a promising direction for AI research beyond scaling.
OpenAI CEO Sam Altman has stated that the era of making giant language models like GPT-3 and GPT-4 is coming to an end, and further progress will be made in other directions. This statement marks an unexpected twist in the race to develop new AI algorithms, as numerous startups are still investing heavily in building larger algorithms. Altman’s belief that going bigger will not work indefinitely is shared by other researchers, who believe that progress on transformers, the machine learning model at the heart of GPT-4 and its rivals, lies beyond scaling. New AI model designs, or architectures, and further tuning based on human feedback are promising directions that many researchers are already exploring.
