A Practical Guide to Building an AI Risk Management Framework

Most enterprises are shipping AI features faster than they can answer a basic question: what happens when the model gets it wrong — or when someone makes it go wrong on purpose?

The common response is to stand up a separate AI risk programme: a new committee, a new policy set, a new register. It feels decisive. In practice, it usually produces a second risk regime that doesn’t talk to the first. AI risks get assessed in one place, security and operational risks in another, and the gaps between them are where incidents happen.

An AI system is still a system. It runs on infrastructure you already secure, processes data you already govern, and sits inside business services you already assess for resilience. The organisations that manage AI risk well start from the risk and security management regime they already run — typically an information security management system built on ISO/IEC 27001 — and extend it to cover what is genuinely different about AI.

That difference is real. Models are opaque, behave in ways their designers didn’t anticipate, depend entirely on their data, and can be attacked through their inputs. A framework that ignores those properties will miss risks. A framework that treats them in isolation will miss everything else.

In the UK, this approach also fits the regulatory reality. There is no single AI statute. Existing regulators — the ICO, FCA, CMA, MHRA, Ofcom, HSE — apply five cross-cutting principles within their own remits. Nobody hands you an AI checklist. The structure has to come from you, and the strongest structure is the one you’ve already built.

This guide sets out how to do that: what to extend, what to add, and how to make sure what you build is secure and stays secure.

Start with the regime you already run

If your organisation holds ISO/IEC 27001 certification, or runs an information security management system aligned to it, most of the machinery for managing AI risk already exists. The task is to point it at AI systems, not to rebuild it.

An ISMS already gives you:

A risk assessment and treatment process.

The same method you use to identify, score and treat information security risks can take AI-specific risks as inputs. You don’t need a second methodology with different scales.

An asset inventory.

AI models, training datasets, prompts, vector stores and third-party model APIs are information assets. If they aren’t in the inventory, they aren’t in scope.

Supplier controls.

Most enterprise AI runs on, or calls out to, third-party models. Your supplier assessment and contractual controls already apply — they just need questions about training data use, model changes and data residency.

Secure development controls.

ISO 27001’s Annex A already covers secure development lifecycles, secure coding and security testing. AI systems should go through them, not around them.

Incident management, logging and monitoring.

An AI failure is an incident. It should land in the same process, with the same escalation paths.

Management review and internal audit.

The governance cadence that keeps your ISMS honest can keep AI risk honest too.

The same logic applies beyond security. AI risks belong on the enterprise risk register next to the operational, financial and conduct risks they interact with. In regulated sectors, AI systems that support important business services belong inside your operational resilience mapping.

Integration is not just tidier. It means AI risk gets reported to the same board committee, owned by people who already have authority to act, and audited by teams who already know how the organisation works. A standalone AI programme has to earn all of that from scratch.

What integration doesn’t do is cover everything. Some AI risks don’t map neatly onto a security or operational risk taxonomy. That’s where the extension work starts.

What your existing regime won’t catch

An ISMS is built to protect the confidentiality, integrity and availability of information. AI systems introduce risks that sit partly or wholly outside that frame. These are the properties your extension needs to address.

Opacity.

Many high-performing models, including large language models, can’t fully explain why they produced a given output. You can observe inputs and outputs without being able to reconstruct the reasoning. That matters wherever a customer, regulator or internal reviewer needs to understand or challenge a decision.

Emergent behaviour.

AI systems produce outputs nobody specified. A customer service assistant generates inappropriate content under an unusual prompt. A system interacting with other automated systems behaves differently in production than in testing. Conventional test plans assume you know what the system is supposed to do; AI systems don’t always oblige.

Data dependency.

A model is shaped by its training and operational data, and that data can be biased, incomplete, stale or deliberately compromised. A model that performed well at launch can degrade silently as the data it sees in production drifts away from the data it learned from.

New attack surfaces.

AI systems can be attacked through the same channel they’re used through. Prompt injection manipulates a model’s behaviour via its inputs. Data poisoning corrupts what it learns. Model extraction and inversion attacks can leak the model itself or the data it was trained on. None of these is caught by a vulnerability scan or a standard web application test.

Fairness and harm to individuals.

A model can perform well on aggregate metrics while producing materially worse outcomes for a particular group. That is a conduct, legal and reputational risk that security controls were never designed to detect.

Scale of impact.

A system making thousands of automated decisions propagates an error at the same rate. The more decisions it makes without human review, the more your monitoring and intervention mechanisms carry.

These properties are why AI needs extension rather than just inclusion. Each one translates into specific additions to how you govern, assess, test and monitor — which the rest of this guide sets out.

The UK regulatory picture

The UK’s five cross-cutting principles, set out in the government’s 2023 AI white paper, give the extension its shape. Whichever regulator eventually asks, the questions come back to the same five:

  • Safety, security and robustness — does the system behave reliably, and does it hold up under adversarial conditions?
  • Appropriate transparency and explainability — can you explain what the system does, and on what basis, to the people who need to know?
  • Fairness — would its outcomes survive scrutiny under the Equality Act and sector fair-treatment obligations?
  • Accountability and governance — is a named person answerable for the system, with authority to act?
  • Contestability and redress — can someone affected by a decision challenge it, and does the challenge go anywhere?

The principles are non-statutory, which makes them useful as a design target rather than a compliance floor. Regulators interpret them through powers they already hold.

UK GDPR applies to any AI system processing personal data. Where a system makes significant decisions about people without meaningful human involvement, the automated decision-making rules apply directly. The Data (Use and Access) Act 2025 replaced the old Article 22 with Articles 22A to 22D, in force since 5 February 2026. The ICO has consulted on replacement guidance; until it is final, treat this as an area to monitor.

Sector regulators apply existing expectations to a new class of system. The FCA works through Consumer Duty, outsourcing rules where a model is third-party hosted, and operational resilience where AI sits inside an important business service. PRA-regulated banks apply the model risk management principles in SS1/23. The MHRA governs AI in medical devices.

The EU AI Act applies if you place AI systems on the EU market, wherever you’re based. Its high-risk obligations are considerably more prescriptive than anything in the UK regime. For enterprises with European operations, building to the stricter standard once is usually cheaper than maintaining two postures.

None of this is a reason to build a separate AI compliance function. It’s a reason to make sure the regime you already run asks these questions about every AI system in scope.

Extending the risk cycle to AI

AI risk management follows the same cycle as any risk discipline: govern, identify, assess, treat. What changes is what each step has to ask.

Govern: extend accountability, don’t duplicate it

Someone needs to own AI risk, but that doesn’t mean a new department. Map AI accountability onto the structures you already have: the ISMS owner, the risk committee, the data protection officer and, in financial services, the senior manager whose statement of responsibilities covers customer-affecting decisions.

Then add what’s missing:

  • AI-specific policy content — acceptable use, model documentation standards, human oversight requirements — written as additions to existing policies rather than a parallel policy set
  • A named owner for each production AI system, recorded in the asset inventory
  • Training for the people who shape AI behaviour — product managers, engineers, QA and executives — on model limitations, bias and how to escalate a concern

The hardest part is behaviour, not documentation. We’ve worked with organisations whose AI ethics policies sat in internal wikis their engineers rarely opened. Governance works when it lives in design reviews, sprint rituals and deployment gates.

Identify: understand who the system affects

Before scoring risks, document how the system operates:

  • Who is affected by its outputs, including people who never use it directly
  • What decisions it informs or automates, and whether humans can override them
  • What data flows in, where from, and who could tamper with it
  • Whether users are likely to defer to its recommendations

Run structured sessions with data scientists, product managers, security, legal and domain experts. Walk through specific scenarios and ask: what is the worst plausible outcome, and who bears the cost? Include adversarial scenarios — what would someone trying to misuse this system do?

For products with recommendation or personalisation features, watch for feedback loops. A model serving content based on user behaviour can reinforce patterns in its own training data, and the effect grows with usage.

Assess: define what you’re measuring before choosing a metric

Score AI risks on the same likelihood and impact scales you use elsewhere, so they can be compared and prioritised against everything else on the register.

Some AI risks can be measured directly. Fairness metrics compare outcomes across groups — protected characteristics under the Equality Act 2010 are the starting point, not the limit. Production monitoring tracks performance and data drift. Automated validation catches data quality problems.

Others need structured judgement. For transparency, ask whether a non-technical stakeholder can understand what the system does, what data it uses and where its limits lie. The ICO and Alan Turing Institute guidance on explaining AI-assisted decisions is the most practical UK reference here.

Measurement is continuous. Models change, data drifts and user behaviour shifts after launch.

Treat: accept, avoid, transfer or mitigate

The treatment options are the ones your risk process already uses. AI-specific mitigations include:

  • Retraining on more representative data
  • Confidence thresholds below which the system defers to a human
  • Input validation and data provenance tracking against poisoning
  • Kill switches that can disable a model or feature quickly
  • Security testing before deployment and after significant changes (covered in the next section)

Build in contestability. Teams often design the model, monitoring and escalation path, then find there’s no route for a customer to challenge a decision. Redress needs three things: the person knows the decision was automated, they can ask for it to be looked at again, and the reviewer has the information and authority to reverse it. Where the DUAA’s automated decision-making provisions apply, this is a legal requirement, not a design preference.

Fold AI into incident response. Your existing incident process needs AI-specific playbooks: how to contain a misbehaving model, how to roll back to a previous version, and when to switch a feature off entirely. Those decisions shouldn’t be made for the first time during an incident.

Secure by design, tested in production: building AI that stays secure

Most AI risk frameworks spend their energy on governance and fairness and treat security as someone else’s problem. That’s backwards. An AI system that can be manipulated through its inputs, or that leaks its training data, fails every other principle too. Security has to be built in during development and verified throughout the system’s life — not tested once before launch.

Start from the UK baseline, then build to the lifecycle detail

The UK government’s baseline is the Code of Practice for the Cyber Security of AI, published by DSIT with the NCSC in January 2025. It sets 13 principles across the AI lifecycle, from staff awareness and threat modelling through to secure disposal of data and models when a system is retired.

The Code is voluntary, but it is the closest thing the UK has to a stated minimum, and it now underpins the ETSI technical specification TS 104 223, so the same requirements will increasingly appear in international procurement and supplier assessments. Use the Code to define what your organisation commits to. Use the NCSC guidelines below to decide how each delivery team meets it.

The NCSC’s Guidelines for secure AI system development, published with international partners including the US CISA, give the stage-by-stage detail. They cover four stages of the lifecycle:

1

Secure design — threat modelling the system, including AI-specific threats, before building it, and weighing security against functionality and performance as a design decision

2

Secure development — securing the supply chain (models, datasets, libraries), documenting data and models, and managing technical debt

3

Secure deployment — protecting infrastructure and models from compromise, building incident management processes, and releasing responsibly

4

Secure operation and maintenance — monitoring system behaviour and inputs, managing updates, and sharing lessons learned

Both map onto the secure development controls your ISMS already requires. Treat them as the AI-specific layer on top, not a separate standard. The Code also covers one stage the NCSC guidelines don’t: decommissioning, which belongs in your asset disposal controls.

Test what you’ve built — then keep testing

Secure development reduces risk; security testing tells you whether it worked. For AI systems, that testing needs to go beyond a standard application penetration test:

  • AI application testing for prompt injection, jailbreaks, insecure output handling and excessive permissions granted to the model
  • Data and model attack testing for poisoning, extraction and inference attacks where the risk profile warrants it
  • Integration testing of how the model connects to your APIs, data stores and downstream systems — which is where typical real-world AI exploits do their damage
  • Red teaming for high-impact systems, simulating a motivated attacker across the whole system rather than testing components in isolation

A model that passed testing at launch isn’t secure forever. Retest after model updates, prompt or retrieval changes, new integrations and new data sources. Feed findings into the risk register and track them to closure like any other security finding.

If you use AI to build software, apply the same rules

AI coding assistants are now part of most development workflows. Code they generate should go through the same review, static analysis and security testing as code written by people. The risk isn’t that assistants write worse code. It’s that teams review it less carefully because it arrives looking finished.

When to add ISO/IEC 42001

Extending your ISMS covers the security side of AI risk. For organisations that build or rely heavily on AI, the next step is a management system designed for AI itself.

ISO/IEC 42001, published in December 2023, specifies requirements for establishing, implementing, maintaining and continually improving an Artificial Intelligence Management System (AIMS). It applies to organisations that provide AI-based products or services and to those that use them. In the UK it is published as BS ISO/IEC 42001.

Why it fits alongside ISO 27001

ISO/IEC 42001 uses the same high-level structure as ISO/IEC 27001: context, leadership, planning, support, operation, performance evaluation and improvement. If you already run a certified ISMS, your management review, internal audit, document control and corrective action processes can serve both. Many organisations run the two as a single integrated management system.

What 42001 adds is AI-specific content that an ISMS doesn’t ask for: AI system impact assessments covering effects on individuals and society, controls for data used in AI systems, requirements across the AI lifecycle, and expectations for transparency towards the people a system affects.

What it gives you

  • A structured way to manage AI risks and opportunities — not just security threats, but fairness, transparency and societal impact
  • Evidence of responsible AI use that customers, procurement teams and regulators can verify through independent certification
  • Traceability, transparency and reliability built into how AI systems are documented, approved and changed
  • Efficiency — reusing your ISMS machinery is faster and cheaper than building an AI governance programme from nothing, and one integrated audit cycle costs less than two separate ones

Is it right for you now?

ISO/IEC 42001 is worth pursuing when AI is part of what you sell, when enterprise customers start asking for evidence of AI governance in procurement, or when you serve EU markets where AI Act obligations apply. If you’re running a handful of internal AI tools, extending your ISMS is usually enough to start — and puts you most of the way to 42001 if you need it later.

Putting it into practice: an implementation roadmap

Inventory your AI systems.

Add every AI model in production and development to your asset inventory, including third-party APIs and AI features embedded in vendor products. Record each system’s use case, data sources, affected people and owner.

Extend your ISMS and risk register.

Bring AI systems into ISMS scope, add AI-specific threats to your risk assessment method, and put AI risks on the enterprise register alongside everything else. Assign accountability through existing roles.

Prioritise by exposure.

A product recommendation engine and a credit decisioning model carry very different risk. Two questions sharpen prioritisation fast: does the system make significant decisions about individuals, and does it fall within scope of the EU AI Act?

Pilot on one high-priority system.

Run it through the full cycle — mapping, assessment, security testing, treatment and monitoring. Note what works and what creates friction.

Build reusable artefacts.

Turn the pilot into model cards, AI risk assessment templates, security test requirements, monitoring dashboards and incident playbooks.

Embed it in delivery.

Threat modelling in design reviews, bias and security testing in the deployment pipeline, monitoring in existing operations tooling. Risk management shouldn’t run as a parallel process.

Scale, then decide on certification.

Apply the process across the AI portfolio. Once it’s working, assess whether ISO/IEC 42001 certification is worth pursuing.

Measuring whether It works

Track a small set of indicators and report them through the same governance cadence as your other risk metrics.

  • Risk assessment coverage — the share of production AI systems with a completed assessment. If 15 models are live and three are assessed, coverage is 20%.
  • Security testing coverage — the share of production AI systems tested in the last 12 months or since their last significant change.
  • Findings closure rate — how quickly AI findings from security tests and audits are fixed.
  • Drift detection latency — the time between meaningful model degradation and your monitoring catching it.
  • Incident response time — time to detect, escalate and contain AI incidents, tracked separately so you know which stage is slow.
  • Fairness metrics — before and after mitigation, with targets set per use case rather than a universal threshold.
  • Documentation completeness — whether each production model has current records of training data, intended use, known limitations and subgroup performance.

Secure AI needs builders and testers

A framework provides the structure. The hard part is applying it to real products, real data and teams working under delivery pressure — and proving that what you’ve built holds up against someone trying to break it.

Infinum works on both sides of that problem. Our AI engineering teams build AI systems with risk and security considered from the architecture stage, inside development processes certified to ISO 27001 and independently audited under SOC 2. AMR CyberSecurity – Part of Infinum, Infinum’s cybersecurity practice, tests them. AMR is an NCSC CHECK, CREST, and STAR accredited provider, with GRC specialists who help organisations extend their security management systems to cover AI.

Two places to start:

  • AI security assessment — AMR – Part of Infinum tests an AI-enabled application for prompt injection, data leakage, insecure integrations and the vulnerabilities a standard pen test misses, and maps findings to your risk register.
  • AI governance gap assessment — AMR’s GRC team reviews your ISMS against the AI systems you run, identifies what needs extending, and assesses your readiness for ISO/IEC 42001.

Book an AI security assessment with AMR CyberSecurity – Part of Infinum.

What services do you need?