NIST Highlights Need for Continuous Security Monitoring of AI Systems

The U.S. National Institute of Standards and Technology (NIST) has published a news release titled “NIST Mathematical Proof Supports Transition to a Continuous-Monitor-and-Update Security Model for AI Systems”, released on 9 June 2026 and updated on 22 June 2026.

The publication explains that a new mathematical proof shows that a fixed set of AI guardrails is not universally robust against adaptive adversarial prompts.

According to NIST, the proof supports the need to move away from a “one and done” security model for AI systems and towards a model based on continuous monitoring, testing and updating.

Fixed Guardrails Are Not Enough

NIST explains that companies developing AI systems often build constraints into their tools to prevent the generation of prohibited or harmful content, such as deepfakes, malware or instructions for illicit activities.

However, the publication states that these constraints are not foolproof. Attackers may craft prompts in ways that cause AI systems to bypass refusal mechanisms, a process commonly referred to as “jailbreaking”.

The proof, published in IEEE Security & Privacy, builds on the logic of Kurt Gödel’s incompleteness theorems. NIST states that there is no finite set of guardrails that is universally robust against adversarial prompts.

Continuous Monitoring and Updating

NIST highlights that the findings do not provide attackers with a method to identify new exploits. Instead, they support the need for organisations deploying AI systems to continuously search for weaknesses and address them before adversaries can exploit them.

The approach described by NIST includes three elements:

• continuous work by red teams to identify new adversarial prompts;

• continuous updates to harden AI guardrails against newly discovered prompts;

• operational resilience to limit impact and support quick recovery when an exploit occurs.

NIST states that the goal is to make the cost of finding new exploits exceed the resources available to attackers.

Impact on Medical Device and IVD Manufacturers

For medical device and IVD manufacturers using AI systems, including software-based medical devices, digital health tools or AI-enabled internal processes, the publication reinforces the importance of treating AI security as an ongoing activity rather than a one-time control.

Manufacturers should pay particular attention to:

• continuous monitoring of AI system behaviour;

• red teaming and adversarial testing;

• update processes for AI guardrails and security controls;

• operational resilience in case of AI-related vulnerabilities;

• cybersecurity risk management for AI-enabled systems;

• documentation of security monitoring and mitigation activities.

Although the NIST publication is not specific to medical devices or IVDs, its message is relevant for manufacturers developing or deploying AI-based technologies. AI guardrails, security controls and risk mitigations may need to be continuously tested, maintained and updated throughout the system lifecycle.

For manufacturers, this may also support broader expectations around trustworthy AI, cybersecurity, software lifecycle management and post-market monitoring of AI-enabled systems.