AI governance

The term sounds like corporate bureaucracy, but it describes something very practical. Without governance, every AI project is a collection of individual decisions: one team buys a tool, another builds an AI agent, a third feeds a language model with customer data. Nobody has an overview, nobody is accountable, and the first incident hits a company without a process. With governance, there is a defined answer to all of this before the incident happens.

Why AI governance is becoming important right now

As long as AI in the company was a chatbot that suggested texts, the risk was manageable: a person read the suggestion and decided. With agentic AI, this changes fundamentally. Agents do not just answer questions, they act: they write records, send emails, trigger orders. Output errors thus become action errors, and a quality problem becomes an operational risk.

The figures show both sides of this development. On the one hand, adoption is growing fast: according to the AI Index for SMEs by Salesforce and the Deutscher Mittelstands-Bund (March 2026), 16.6 per cent of German SMEs already use AI agents, almost twice as many as in 2024. On the other hand, Gartner expects more than 40 per cent of all agentic AI projects to be cancelled by the end of 2027. One of the three reasons given: inadequate risk controls. These are exactly the core of AI governance.

Then there is a phenomenon that many companies underestimate: shadow AI. Employees have long been using AI tools, whether or not there is a policy, and in case of doubt upload customer data to services that nobody has ever checked. Governance therefore does not mean preventing use, but giving it a permitted, safe route. A company without approved tools does not have an AI-free workforce; it has an invisible one.

How expensive missing controls can become was shown by a much-cited incident in July 2025: a coding agent on the Replit platform deleted the production database of a SaaS project during a code freeze, containing records on more than 1,200 executives. The agent had write access to production systems and no enforced approval step. No prompt in the world replaces the control that was missing here.

The building blocks of AI governance

A viable governance framework consists of six building blocks. They apply to classic AI applications just as much as to agents, but with agents they become mandatory rather than optional:

• Roles and responsibilities: every AI system has a named human owner who is accountable for granting permissions, reviews and incidents. Without this role, every system becomes orphaned after go-live.
• Permission and access model: AI systems only receive the permissions their task requires (least privilege); for agents, ideally in stages: first read-only, then with approval (human-in-the-loop), then autonomous within narrow limits.
• Identity: every system gets its own distinguishable identity instead of shared API keys. Only then can the question of who did what be answered later.
• Logging and audit: complete, analysable logs of actions, triggers and data basis, plus versioned prompts and configurations. The log is the basis of every review and every promotion decision.
• Testing before and during operation: adversarial testing (red teaming) before go-live, plus repeatable eval suites that run again with every prompt change and every model switch. Technical guardrails catch in operation what tests do not prevent.
• Life cycle management: a defined path from introduction through regular reviews with metrics to an orderly shutdown: revoking permissions, rotating credentials, archiving logs.

Identity as the foundation: the example of Microsoft Entra Agent ID

Microsoft shows how seriously the industry takes the identity building block: Entra Agent ID manages AI agents as separate identities in the corporate directory, with a sponsor, owner and life cycle. Agents appear there like employee accounts: they have an owner, their access is traceable, and access ends when the agent is no longer needed. The pattern makes sense regardless of the vendor stack: governance begins with AI systems being in the same directory as people, not in a shadow spreadsheet.

AI governance and the EU AI Act

With the EU AI Act at the latest, AI governance is no longer optional. Three obligations are particularly relevant in practice:

AI literacy (Article 4), in force since 2 February 2025: companies must promote the AI literacy of the people who operate and use AI systems. The Digital Omnibus softened the wording in 2026 (from "ensure" to "promote"), but the obligation itself remains. Documented, role-based training is the reliable way to meet it.

Transparency (Article 50), in force from 2 August 2026: where people interact with an AI system, this must be disclosed. A customer-facing support agent must therefore identify itself as AI.

High-risk obligations (Annex III), in force from 2 December 2027: for stand-alone high-risk systems, for example in recruiting or credit scoring, requirements including risk management, data quality requirements and human oversight will then apply. The postponement of the original date is a deferral, not an all-clear: anyone planning such systems should build the governance foundations now, not in 2027.

Important for context: the AI Act does not prescribe a particular governance framework. But anyone who has implemented the six building blocks above meets the documentation and oversight obligations almost incidentally, because owners, logs and review processes already exist.

Distinction: IT governance, compliance and AI safety

AI governance overlaps with three neighbouring disciplines but is identical to none of them. IT governance steers the IT landscape as a whole (systems, budgets, security); AI governance is its extension to the particularities of learning, probabilistic systems that can act differently even with the same input. Compliance checks adherence to external rules (GDPR, AI Act, sector law); governance creates the structures with which adherence can be demonstrated at all. AI safety works at model level on the safe behaviour of the AI itself; governance works at organisational level and sets the framework within which even an imperfect model cannot cause unchecked damage.

How to introduce AI governance: four steps

In practice, the set-up almost always follows the same order, regardless of company size:

Step 1, inventory: record which AI systems are already in use, officially and unofficially. For each system: purpose, data access, write permissions, user group. This inventory is almost always longer than expected, and its existence alone ends the shadow AI phase.

Step 2, risk classification: sort the systems by impact: read-only and internal? Customer contact? Write access to business systems? Decisions relating to personnel (caution: a future high-risk class)? The classification determines how much control each system needs and prevents harmless tools from being over-regulated and risky ones from being under-steered.

Step 3, document the framework: put the policy, owners, permission model and review processes in writing, as briefly as possible. A framework that nobody reads does not exist.

Step 4, anchor it in operations: schedule reviews, make logs analysable, document training under Article 4 and define an offboarding process for systems that are switched off. Governance lives in the calendar, not in a folder.

AI governance at mid-sized companies: pragmatic, not bureaucratic

The most common concern at mid-sized companies: governance sounds like a committee that slows innovation down. The opposite is true if you tailor it correctly. A 50-person company does not need an AI board with quarterly meetings. It needs four things that fit on a few pages:

• a short AI policy: which tools are permitted, which data may go into them, what is off-limits;
• one owner per productive AI system, named individually;
• a permission and logging concept for everything that has write access to business systems;
• a fixed review date at which metrics and incidents are put on the table.

These four points fit on two pages, can be drafted in an afternoon and still cover the questions on which projects otherwise get stuck. That sets the framework, and it grows with you: when the first agent with real write permissions arrives, you add graded autonomy and red teaming before go-live. When a multi-agent system arrives, identities and handover protocols are added. Governance that develops in this way tends to speed projects up rather than slow them down, because the same old questions (who is liable? which permissions? what happens in the event of errors?) do not have to be renegotiated in every project.

The best time to build the framework is with the first productive system, not after the first incident. An experienced AI agency therefore does not set up governance as a separate consulting project, but as part of the first introduction: permission design, logging and review process are created together with the system they are meant to control.

Frequently asked questions about AI governance

Is AI governance required by law?

Not as a framework under this name. However, the EU AI Act prescribes specific obligations that can hardly be met without governance structures: AI literacy since February 2025, transparency obligations from August 2026, high-risk requirements from December 2027. Governance is the practical way of meeting these obligations in a demonstrable way.

From what company size is AI governance worthwhile?

From the first AI system that works with real data or real customers. The scope scales with the risk: for a text suggestion tool, a policy plus an owner is enough; for an agent with write permissions, you need permission levels, logs and tests.

What is the difference between AI governance and an AI policy?

The policy is a document, one building block of governance. Governance also includes the processes as they are actually lived: granting permissions, testing, reviews, incident follow-up. A policy without these processes is a PDF that nobody enforces.

Who should be responsible for AI governance in the company?

A person with decision-making authority over IT and processes; at mid-sized companies this is often management, the CTO or the head of IT. More important than the position is clarity: one named person for the framework, plus one owner per productive AI system.

Doesn't governance slow down the introduction of AI?

Tailored correctly, it speeds it up. Projects rarely fail because of too many rules, but because of unresolved questions about liability, permissions and risks that are discussed anew for every project. A lean framework answers them once and makes every follow-up project faster. Gartner's forecast on cancelled agent projects names missing risk controls as a reason for cancellation, not too many of them.