AI models compete in Minecraft building contests

A high school student has developed a unique way to test AI models by having them compete in Minecraft build challenges. Adi Singh, a 12th-grader, created a website called MC-Bench, which allows users to prompt various AI models to build structures in the popular sandbox game. The models generate code to complete the builds, and users vote on which result they prefer without initially knowing which model made which creation.

The idea behind the project is to offer a more intuitive and visually engaging benchmark for AI performance. Minecraft’s widespread recognition makes it easier for people to evaluate results, Singh explained, especially compared to technical benchmarks that often favour models trained on similar datasets.

The project is supported by volunteer developers and has received infrastructure support from major players like OpenAI, Google, Anthropic, and Alibaba, though none are officially affiliated. While current prompts are simple, Singh hopes to expand into more complex, goal-oriented tasks, viewing games as a safer and more controlled testing ground for AI reasoning.

MC-Bench is part of a broader trend exploring alternatives to standard AI benchmarks, which often reveal inconsistencies in model abilities. While some models excel in academic tests, they struggle with tasks that require basic reasoning. Singh believes his Minecraft-based benchmark reflects real-world performance more accurately and could help companies steer AI development in more practical directions.

For more information on these topics, visit diplomacy.edu.