Beyond scaling: The future of AI algorithms according to OpenAI CEO
OpenAI CEO Sam Altman has stated that the era of making giant language models like GPT-3 and GPT-4 is coming to an end, and further progress will be made in other directions. This statement marks an unexpected twist in the race to develop new AI algorithms, as numerous startups are still investing heavily in building larger algorithms. Altman’s belief that going bigger will not work indefinitely is shared by other researchers, who believe that progress on transformers, the machine learning model at the heart of GPT-4 and its rivals, lies beyond scaling. New AI model designs, or architectures, and further tuning based on human feedback are promising directions that many researchers are already exploring.