NIST unveils AI model risk test tool

The National Institute of Standards and Technology (NIST) has re-released Dioptra, a tool designed to measure AI model risks, particularly from data poisoning attacks. The modular, open-source web-based tool, originally launched in 2022, aims to help companies and individuals assess and analyse AI risks. It can be used for benchmarking, researching models, and exposing them to simulated threats, offering a common platform for these activities.

NIST has positioned Dioptra to support government agencies and businesses in evaluating AI system performance claims. The tool’s release coincides with new documents from NIST and the AI Safety Institute that outline ways to mitigate AI-related dangers, including the generation of non-consensual pornography. This effort is part of a broader US-UK partnership to advance AI model testing, which was announced at the UK’s AI Safety Summit last year.

The development of Dioptra aligns with President Joe Biden’s executive order on AI, which mandates comprehensive AI system testing and the establishment of safety and security standards. Companies developing AI models, such as Apple, are required to notify the federal government and share safety test results before public deployment.

Despite its capabilities, Dioptra has limitations. It only works with models that can be downloaded and used locally, such as Meta’s expanding Llama family. Models that are accessible only via an API, like OpenAI’s GPT-4, are currently not compatible. Nonetheless, NIST proposes that Dioptra can highlight which types of attacks might degrade an AI system’s performance and quantify their impact.