NIST Publishes Generative AI Profile for the AI Risk Management Framework
The National Institute of Standards and Technology has published NIST AI 600-1, titled “Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile.”
The document, published in July 2024, is a companion resource to the AI Risk Management Framework 1.0 and focuses specifically on risks related to Generative Artificial Intelligence.
NIST describes the publication as a cross-sectoral profile, intended to support organizations in governing, mapping, measuring and managing risks associated with generative AI systems.
Purpose of the Generative AI Profile
The profile is designed to help organizations incorporate trustworthiness considerations into the design, development, use and evaluation of AI products, services and systems.
It applies the AI RMF functions to generative AI and provides suggested actions based on risk management priorities, legal and regulatory considerations, organizational goals and available resources.
The document was developed pursuant to Executive Order 14110 on Safe, Secure and Trustworthy Artificial Intelligence.
Risks Unique to or Exacerbated by Generative AI
NIST identifies several risks that are either unique to generative AI or intensified by its use.
These include:
CBRN information or capabilities;
confabulation, also commonly referred to as hallucination;
dangerous, violent or hateful content;
data privacy risks;
environmental impacts;
harmful bias and homogenization;
human-AI configuration risks;
information integrity risks;
information security risks;
intellectual property risks;
obscene, degrading or abusive content;
value chain and component integration risks.
NIST notes that generative AI risks may vary depending on the AI lifecycle stage, system scope, source of risk and time scale.
Suggested Actions to Manage GAI Risks
The publication provides suggested actions aligned with the AI RMF functions: Govern, Map, Measure and Manage.
These actions include establishing policies for legal and regulatory alignment, documenting the origin and history of training and generated data, evaluating risk-relevant capabilities before and after deployment, defining risk tiers, maintaining AI system inventories and developing incident response processes.
The profile also highlights the importance of content provenance, pre-deployment testing, incident disclosure and governance as key considerations for generative AI risk management.
Governance, Transparency and Accountability
A major focus of the profile is organizational governance.
NIST recommends that organizations define roles and responsibilities, establish acceptable use policies, maintain documentation, address third-party risks and create mechanisms for incident response and communication.
The document also emphasizes transparency around data provenance, digital content transparency methods, system limitations and the interaction between humans and AI systems.
Impact on Organizations Using AI in Regulated Sectors
For organizations operating in regulated sectors, including medical device and IVD manufacturers exploring or deploying generative AI, the NIST profile provides a structured reference for managing AI-related risks.
Manufacturers should pay particular attention to:
AI governance and oversight;
intended use and context of use;
data privacy and sensitive data protection;
information security and resilience;
confabulation and output reliability;
human oversight and automation bias;
third-party AI tools and supplier controls;
content provenance and transparency;
pre-deployment testing and ongoing monitoring;
incident reporting and response;
documentation of AI system risks and limitations.
For regulated manufacturers, the publication reinforces the need to treat generative AI as a governed technology with defined controls, documented risk assessments and lifecycle monitoring.
