UK cyber agency warns of growing vulnerability risks from Frontier AI

The UK’s National Cyber Security Centre (NCSC) has issued guidance for network defenders on managing the growing risk associated with software vulnerabilities discovered using Frontier AI.

The guidance states that Frontier AI models represent the most advanced AI systems and have already demonstrated the ability to identify vulnerabilities in software products. According to the NCSC, this has significant implications for the threat landscape because Frontier AI can help both defenders and threat actors identify weaknesses at greater speed and scale. The UK’s National Cyber Security Centre has issued guidance for network defenders on managing the growing risk from software vulnerabilities discovered with Frontier AI.

The guidance states that Frontier AI models represent the most advanced AI systems and have demonstrated the ability to discover vulnerabilities in software products. The NCSC says this has implications for the threat landscape because Frontier AI can help both defenders and threat actors identify weaknesses more quickly.

The NCSC emphasises that organisations using AI for vulnerability discovery should do so within secure and controlled environments. It recommends limiting what the AI system can access, ideally using it only in testing or development environments, running it through a service account with only necessary permissions, and placing it in a sandboxed environment.

Organisations should also consider legal, contractual, and security obligations before using AI-as-a-service tools for vulnerability discovery. Sending source code, intellectual property or other sensitive information to external AI providers could introduce additional security, confidentiality and compliance risks.

The NCSC notes that AI-assisted vulnerability discovery is only effective if organisations have the processes and resources needed to manage the findings. That means having processes for patch management, vulnerability identification, prioritisation, validation, remediation, and reporting, as well as the ability to filter false positives and address root causes rather than only individual flaws.

The NCSC stresses that Frontier AI should complement, rather than replace, human cybersecurity expertise. Staff with experience in cybersecurity or the relevant IT systems should guide and validate AI-based vulnerability discovery to improve speed and accuracy.

The NCSC also warns that threat actors are increasingly using Frontier AI to identify and exploit vulnerabilities, potentially accelerating cyberattack timelines. Frontier AI may reduce the time between discovery and exploitation of newly published vulnerabilities, leaving organisations with less time to patch. The guidance says organisations should therefore adopt an assume-compromised mindset.

The NCSC recommends that organisations meet minimum cybersecurity standards, apply defence-in-depth principles, monitor networks and endpoints for suspicious behaviour and maintain a strong incident response plan.

The guidance also urges organisations to reduce the number of systems exposed to the internet, especially high-risk systems such as admin login panels, legacy systems, and operational technology. Organisations should identify internet-accessible systems and assess whether they need to remain exposed.

The guidance also highlights the growing importance of software supply chain security. Organisations should understand the commercial software, cloud services, open-source software, and dependencies they use, review supplier security and AI assurance policies, apply updates quickly, and use software bills of materials or similar tools to identify vulnerable dependencies.

The NCSC says Frontier AI is likely to be used extensively to discover vulnerabilities in open-source software because source code is accessible. It also notes that open-source supply chains have already been targeted through malware campaigns affecting major packages.

Why does it matter?

The guidance reflects a growing shift in cybersecurity as advanced AI systems become capable of identifying software vulnerabilities at unprecedented speed. While these capabilities can help defenders improve security testing and vulnerability management, they can also enable attackers to discover and exploit weaknesses more quickly, potentially reducing the time organisations have to respond.

The NCSC’s recommendations also point to a broader governance challenge surrounding AI adoption in cybersecurity. Organisations must not only defend against AI-enabled threats but also ensure that their own use of AI tools does not introduce new risks related to sensitive data, software supply chains or overreliance on automated systems. As Frontier AI capabilities continue to improve, cyber resilience will increasingly depend on combining AI-driven analysis with strong human oversight, secure development practices and effective incident response.

Would you like to learn more about AI, tech, and digital diplomacy? If so, ask our Diplo chatbot!