AI security guidelines developed by the United States’ Cybersecurity and Infrastructure Security Agency (CISA) and the United Kingdom’s National Cyber Security Centre (NCSC) were published Monday with endorsements from 16 other nations. The 20-page document was written in cooperation with experts from Google, Amazon, OpenAI, Microsoft and more, and is the first of its kind to receive global agreement, according to the NCSC.
“We know that AI is developing at a phenomenal pace and there is a need for concerted international action, across governments and industry, to keep up,” said NCSC CEO Lindy Cameron, in a public statement. “These guidelines mark a significant step in shaping a truly global, common understanding of cyber risks and mitigation strategies around AI to ensure that security is not a postscript to development but a core requirement throughout.”
Here are four key takeaways from the publication:
“Secure-by-design” and “secure-by-default” take priority
Emphasized throughout the document are the principles of “secure-by-design” and “secure-by-default” — proactive approaches to protect AI products from attack. The authors urge AI developers to prioritize security alongside function and performance throughout their decision-making, such as when choosing a model architecture or training dataset. It is also recommended that products have the most secure options set by default, with the risks of alternative configurations clearly communicated to users. Ultimately, developers should assume accountability for downstream results and not rely on customers to take the reins on security, according to the guidelines.
Key excerpt: “Users (whether ‘end users,’ or providers incorporating an external AI component) do not typically have sufficient visibility and/or expertise to fully understand, evaluate or address risks associated with systems they are using. As such, in line with ‘secure by design’ principles, providers of AI components should take responsibility for the security outcomes of users further down the supply chain.”
Expansion: This principle implies that AI developers should adopt a holistic and proactive approach to security, rather than a reactive and piecemeal one. It also means that security should be embedded in every stage of the AI life cycle, from design to deployment to maintenance. By doing so, developers can reduce the likelihood and impact of potential attacks, as well as enhance the trust and confidence of users and stakeholders. Some examples of applying this principle are:
- Using secure coding practices and tools to prevent common vulnerabilities and bugs in the AI code.
- Applying encryption, authentication and access control mechanisms to protect the confidentiality, integrity and availability of the AI data and models.
- Conducting regular security audits and assessments to identify and remediate any security gaps or weaknesses in the AI system.
- Providing clear and transparent documentation and guidance on the security features and limitations of the AI product, as well as the best practices and responsibilities of the users.
Complex supply chains require greater diligence
AI tool developers frequently rely on third-party components like base models, training datasets and APIs when designing their own product. An extensive network of suppliers creates a greater attack surface where one “weak link” can negatively impact the product’s security. The global AI guidelines recommend developers assess these risks when deciding whether to acquire components from third parties or produce them in-house. When working with third parties, developers should vet and monitor the security posture of suppliers, hold suppliers to the same security standards as one’s own organization and implement scanning and isolation of imported third-party code, the guidelines state.
Key excerpt: “You are ready to failover to alternate solutions for mission-critical systems, if security criteria are not met. You use resources like the NCSC’s Supply Chain Guidance and frameworks such as Supply Chain Levels for Software Artifacts (SLSA) for tracking attestations of the supply chain and software development life cycles.”
Expansion: This principle recognizes that AI security is not only dependent on the developer’s own practices, but also on the practices of the suppliers and partners that provide the components and services that make up the AI system. A breach or compromise in any of these components can have cascading effects on the security and functionality of the AI product. Therefore, developers should exercise due diligence and oversight when selecting and working with third parties, and ensure that they have adequate security measures and controls in place. Some examples of applying this principle are:
- Performing background checks and security audits on the suppliers and partners to verify their reputation, track record and compliance with relevant standards and regulations.
- Establishing clear and enforceable contracts and agreements that specify the security requirements and expectations of both parties, as well as the consequences and remedies for any breaches or violations.
- Implementing mechanisms to verify the provenance, integrity and quality of the third-party components, such as digital signatures, checksums and hashes.
- Isolating and sandboxing the third-party code and data from the rest of the AI system, and limiting the permissions and privileges that they have access to.
AI faces unique risks
AI-specific threats such as prompt injection attacks and data poisoning call for unique security considerations, some of which are highlighted by CISA and NCSC in their guidelines. A component of the “secure-by-design” approach includes integrating guardrails around model outputs to prevent leaking of sensitive data and restricting the actions of AI components used for tasks such file editing. Developers should incorporate AI-specific threat scenarios into testing and monitor user inputs for attempts to exploit the system.
Key excerpt: “The term ‘adversarial machine learning’ (AML), is used to describe the exploitation of fundamental vulnerabilities in ML components, including hardware, software, workflows and supply chains. AML enables attackers to cause unintended behaviors in the AI system, such as misclassification, evasion, impersonation or data exfiltration.”
Expansion: This principle acknowledges that AI systems are not immune to the conventional cyber threats that affect other IT systems, such as malware, phishing, denial-of-service and ransomware. However, AI systems also face unique and emerging threats that exploit the specific characteristics and limitations of AI techniques, such as machine learning and natural language processing. These threats can undermine the accuracy, reliability and safety of the AI system, and potentially cause harm to the users and society. Therefore, developers should be aware of and prepared for these threats, and implement appropriate countermeasures and defenses. Some examples of applying this principle are:
- Applying robust and diverse data validation and sanitization techniques to prevent data poisoning and manipulation attacks that aim to corrupt the training or inference data of the AI system.
- Implementing input and output filtering and moderation mechanisms to prevent prompt injection and extraction attacks that aim to manipulate or extract sensitive information from the AI system.
- Employing adversarial training and testing methods to improve the robustness and resilience of the AI system against adversarial examples and attacks that aim to fool or evade the AI system.
- Leveraging explainability and interpretability tools to enhance the transparency and accountability of the AI system, and to detect and correct any errors or biases in the AI system.
AI security must be continuous and collaborative
The guideline document outlines best practices throughout four life cycle stages: design, development, deployment, and operation and maintenance. The fourth stage spotlights the importance of continuous monitoring of deployed AI systems for changes in model behavior and suspicious user inputs. The “secure-by-design” principle remains key as a component of any software updates made, and the guidelines recommend automated updates by default. Lastly, CISA and the NCSC recommend developers leverage feedback and information-sharing with the greater AI community to continuously improve their systems.
Key excerpt: “When needed, you escalate issues to the wider community, for example publishing bulletins responding to vulnerability disclosures, including detailed and complete common vulnerability enumeration. You take action to mitigate and remediate issues quickly and appropriately.”
Expansion: This principle emphasizes that AI security is not a one-time or static process, but a dynamic and ongoing one that requires constant vigilance and adaptation. AI systems are subject to changing environments, user behaviors, data inputs and outputs, and threat landscapes, which can affect their performance and security. Therefore, developers should monitor and evaluate their AI systems regularly and continuously, and apply timely and appropriate updates and patches to address any issues or vulnerabilities. Moreover, developers should collaborate and communicate with other AI stakeholders, such as users, customers, regulators, researchers and peers, to share best practices, lessons learned, and emerging threats and solutions. By doing so, developers can enhance the security and quality of their AI systems, as well as contribute to the collective knowledge and advancement of the AI field. Some examples of applying this principle are:
- Implementing logging, auditing and reporting mechanisms to track and analyze the behavior and performance of the AI system, and to detect and alert any anomalies or deviations.
- Applying automated and manual testing and verification methods to validate and verify the functionality and security of the AI system, and to identify and correct any errors or bugs.
- Establishing a clear and effective update and patch management process to ensure that the AI system is always running on the latest and most secure version of the software and hardware.
- Participating in industry forums, standards bodies, and research initiatives to exchange information and insights on AI security challenges and best practices, and to adopt common frameworks and guidelines.