Your AI policy is not an AI governance program

I’m going to argue something that will sound obvious to anyone who’s worked in GRC: AI governance isn’t a new discipline. It’s your existing risk and control framework, stretched to cover a type of system that behaves differently from the ones it was designed for. The frameworks agree on the destination. NIST’s AI Risk Management Framework organises the work into four functions (Govern, Map, Measure, Manage),⁴ and ISO/IEC 42001, published in December 2023, sets out requirements for an AI management system you can actually certify against.⁵ What they don’t tell you is how to wire any of it into the GRC platform you already run. That’s the part I want to get practical about.

What’s actually different about AI

For a conventional application, the governance questions are familiar. Who owns it? What data does it touch? Who can change it, and who approves those changes? What breaks if it goes down?

None of those go away with AI. But a new set sits on top, because an AI system can change its behaviour without anyone changing its code. Behaviour shifts when the incoming data drifts away from what the model was trained on, when someone edits a prompt, when your vendor quietly swaps the underlying model, or when an agent is given access to one more tool.

So you end up asking things like: what data shaped this model’s behaviour? How accurate is it for the population we’re actually using it on? Can a user override it? Can a cleverly worded input talk it into doing something it shouldn’t? And six months from now, could we reconstruct why it made a particular decision?

The answers to those questions shouldn’t live in a data scientist’s notebook. They need to sit inside the same chain every other risk follows:

AI asset, then use case, then risk, then control, then evidence, then owner, then monitoring, then remediation.

If any link in that chain is missing, you don’t have governance. You have documentation.

Govern the system, not the model

The most common mistake I see is treating the model as the thing being governed. “We’ve approved GPT-whatever for internal use” gets recorded, and everyone moves on.

Look at what a typical customer-service agent actually consists of. There’s the app the user types into, a set of system instructions, the model itself, a retrieval layer pulling from your knowledge base, and then a set of tools the agent can call: your CRM, ticketing, email, maybe payments. Every one of those is part of the risk surface. So are the embeddings, the vector database, any fine-tuning data, the permissions on each API, and the humans who approve (or don’t) what comes out the other end.

The inputs alone can sink you. In 2023, Samsung engineers reportedly pasted confidential source code into ChatGPT while debugging, and the company responded by restricting generative AI tools on company devices.⁷ Nothing was wrong with the model. The governance gap was in what people were allowed to feed it.

For agents, I’d go further: the permissions often matter more than the model. The question isn’t just “is this agent accurate?” It’s “what is this agent technically able to do, using whose identity, under what conditions, and who gets to stop it?” That’s a GRC question. It’s also an identity and access management question, and the sooner your IAM team is in the room, the better.

Start with an inventory, and expect to be surprised

You can’t govern what you can’t see, and with AI, a lot of it is invisible. Microsoft and LinkedIn’s 2024 Work Trend Index found that 75% of knowledge workers were already using generative AI at work, and 78% of those users were bringing their own tools rather than waiting for something approved.⁸ IBM’s 2025 study put a price on that: breaches involving high levels of “shadow AI” cost organisations up to US$670,000 more on average.³

So the first deliverable of any AI governance program is an inventory, and it needs more than a list of application names. Here’s the kind of record that’s actually useful:

In GRC terms, this becomes an authoritative register. Every other process (risk assessments, vendor reviews, control testing, regulatory mapping, incident management, board reporting) should hang off it. If your GRC platform already has an asset or application register, extend that object rather than creating a parallel one. Two registers that disagree are worse than one.

Tier by use case, not by technology

A tool that summarises meeting notes and a model that decides who gets a loan should not get the same level of scrutiny. Control intensity should follow potential impact, which is the logic regulators have adopted too.

A three-tier scheme works for most organisations I’ve seen:

• Low impact covers things like writing assistants, code suggestions and internal search. An acceptable-use policy, data classification rules, an approved vendor list, basic logging and user training are usually enough.
• Moderate impact covers fraud detection, alert triage and support recommendations. Add a documented risk assessment, accuracy testing, bias testing where it’s relevant, monitoring, human review and a vendor assessment.
• High impact covers credit, hiring, insurance, medical decision support and critical infrastructure. Here you need formal validation, detailed technical documentation, audit logging, robustness and security testing, real human oversight, formal change management and independent assurance.

The EU AI Act formalises this tiering. It entered into force on 1 August 2024, with bans on certain practices applying from February 2025 and obligations for general-purpose AI models from August 2025.⁹ High-risk systems must run a continuous risk-management process across their whole lifecycle (Article 9). Fines for prohibited practices can reach €35 million or 7% of worldwide annual turnover, whichever is higher. The high-risk obligations were originally due to apply from August 2026, but the European Commission proposed pushing parts of that timeline back in its late-2025 “digital omnibus” package, so check where that stands before you build a compliance plan around a specific date.

Local rules matter too. New York City has required independent bias audits for automated hiring tools since its Local Law 144 began enforcement in July 2023.¹⁰

The risks your existing taxonomy misses

Classic technology risk is built around confidentiality, integrity and availability. AI needs a few more categories. NIST lists the characteristics of trustworthy AI as valid and reliable, safe, secure and resilient, accountable and transparent, explainable and interpretable, privacy-enhanced, and fair with harmful bias managed.⁴ In practice, I’d make sure your risk library can hold at least these:

The point isn’t the list. It’s that each of these should become an enterprise risk with a named owner, not a paragraph in a model card nobody outside the ML team reads.

Write controls you can actually test

“AI may produce inaccurate outputs” is a risk statement. On its own, it governs nothing. A control has to say who does what, when, and what proof they leave behind. Here’s what that looks like when it’s written properly:

Once controls look like this, AI governance lands squarely in territory GRC teams already know well: risk, control, evidence, testing, finding, remediation. Nothing exotic about it.

It also helps to organise these into an AI control library rather than scattering them across your IT general controls. The domains I’d start with are governance (policy, committee, accountability, inventory), data (lineage, quality, provenance, privacy), models (validation, versioning, change management, drift, retirement), security (prompt-injection and adversarial testing, access control, supply chain), human oversight, third-party risk (including contractual model-change notification and training-use restrictions), monitoring and assurance.

“Show me”: evidence is where most programs fall over

Sooner or later, someone says “we do bias testing,” and an auditor replies with two words: show me.

The difference between an AI experiment and a controlled AI system is whether you can produce a traceable chain from the system in production all the way back to the test that justified putting it there:

That holds up under questioning. “The data science team tested it” doesn’t. The EU AI Act makes the same point in law, with explicit requirements for technical documentation (Article 11) and automatic logging (Article 12) for high-risk systems.⁹

Human in the loop is a control, not a slogan

Almost every AI governance deck I’ve reviewed says “human in the loop” somewhere. Far fewer explain what the human actually does.

A confidence threshold that routes uncertain cases to a person is a start. But that person also needs enough context to judge the case, the authority to reject the recommendation, enough time to review it properly, some understanding of where the model tends to fail, and a way to record their override so it can be audited later. Without those, the reviewer is a rubber stamp with a salary.

If your reviewers approve 99.8% of what the model recommends, you should be asking whether they’re reviewing or just clicking.

Article 14 of the EU AI Act names this directly. It requires that people overseeing high-risk systems stay aware of the tendency to over-rely on AI output, which it calls automation bias.⁹ Override rates, review times and escalation counts are worth tracking for exactly this reason.

Agents: least privilege for software that acts

Generative AI already strained conventional governance. Agents break it further, because they don’t just answer. They plan, pick a tool, call an API, look at the result and decide what to do next, sometimes many times before any human notices.

The industry is betting heavily on this anyway. Gartner has predicted that a third of enterprise software applications will include agentic AI by 2028, up from less than 1% in 2024. It has also predicted that more than 40% of agentic AI projects will be cancelled by the end of 2027, citing costs, unclear value and inadequate risk controls.¹⁴ That second number is the one GRC teams should pay attention to.

The control questions for agents are ones IAM people have been asking about human users for years. Which identity does the agent run under? What can it do, and does it need all of that? Can the same agent initiate and approve a sensitive transaction? Is there a limit on what it can spend? Which actions need a human sign-off? Can we reconstruct every step it took? And can we switch it off immediately?

In practice, that ends up as a permission matrix like this one:

Plug it into the GRC platform you already run

Please don’t build a separate AI governance universe. Whether you’re on IBM OpenPages, ServiceNow IRM, Archer or something else, the objects you need (risks, controls, assessments, vendors, issues, evidence) mostly exist already. AI governance should extend them so an AI risk can sit next to the related cyber, privacy and third-party risks in one register, and roll up into the same board reporting.

The same goes for regulation. Rather than keeping a separate checklist for every framework, map each control once to every requirement it satisfies:

Then the conversation changes. Instead of “are we compliant with regulation X?”, you’re asking “which controls do we run, which requirements do they cover, and where are the gaps?” That’s a much better question.

The three lines model still applies, too. The team building and running the system operates its controls. Risk, compliance and security set the methodology and challenge the assessments. Internal audit provides independent assurance. The people who built a model shouldn’t be the only ones deciding whether its controls are good enough.

Measure whether the controls work

“AI policy published: yes” is not a metric. It tells management nothing about whether AI is under control. The measures I’d put on a dashboard are the share of discovered AI systems that are actually registered, the share with a current risk assessment, the share of models that passed required validation, the share of production systems with active monitoring, the number of open high-risk exceptions, and incident figures, including unauthorised agent actions, time to detect and time to remediate.

Every one of those can go down as well as up, which is exactly why they’re useful.

Gates across the lifecycle

Finally, put a decision gate at each stage of an AI system’s life: identify it, classify it, assess it, approve the use case, build it with security and privacy requirements in, validate it against thresholds, confirm controls are live before deployment, monitor it, reassess when the model, data, vendor or regulation changes, and retire it cleanly, data included. If that sounds like SDLC plus model risk management plus GRC, that’s because it is.

The test

Here’s how I’d check whether an organisation really has AI governance. Can it answer these questions today, with evidence, without a two-week fire drill?

• How many AI systems do we run, and which ones are high risk?
• Which of them process sensitive data or influence material decisions?
• Which models haven’t been independently validated?
• Which AI vendors hold our confidential data, and can they train on it?
• Which agents can take actions in external systems?
• Which AI risks currently sit outside our risk appetite?
• Which models changed last quarter, and did anyone re-validate them?
• Could we reconstruct a specific AI decision from six months ago?

If the answer to most of these is “we’d have to find out,” then what you have is an AI policy. That’s a reasonable starting point. It just isn’t the same thing as governance.

Nobody serious expects to eliminate AI risk. The goal is to make it visible, measurable, owned, controlled and auditable. As AI moves from copilots that suggest to agents that act, that’s going to be the difference between the organisations that scale AI with confidence and the ones that end up explaining to a tribunal why the chatbot wasn’t really speaking for them.