AI Governance Framework: Building It in Practice

Building blocks of an AI governance framework

A good AI governance framework does not have to be heavy on red tape. But it does need a clear structure. These parts have proven useful:

• AI policy: A short paper signed off by management. It sets out goals, rules and banned uses. One example: no confidential customer data in services that are not approved.
• AI inventory: A list of every AI system in use. This includes AI features built into standard software. For each one, it notes the purpose, data types, vendor, business owner and risk class.
• Roles and boards: For instance an AI officer, business system owners and a small approval board. The board draws on IT security, data protection, legal and the business.
• Approval process: A set path from request through risk check to approval on record. The effort grows with the level of risk.
• Risk management: A method to rate risks that are typical for AI. These include wrong output, bias, data leaks, prompt injection or vendor lock-in.
• Running and monitoring: Logging, performance checks, a way to deal with vendor model updates and a channel to report anything odd.
• Skills and training: Training for each role. It follows from the duty in Art. 4 of the EU AI Act to help staff build AI literacy.

Reference frameworks: ISO/IEC 42001 and the NIST AI RMF

You do not need to start from scratch. Two frameworks give you the structure.

ISO/IEC 42001

ISO/IEC 42001:2023 sets out an AI management system (AIMS). You can get certified against it. It shares the high-level structure of ISO/IEC 27001:

• context
• leadership
• planning
• support
• operation
• performance evaluation
• improvement

Its Annex A holds controls. Among other things, they cover AI system impact assessment, data management and the system life cycle. Do you already run an ISMS under ISO/IEC 27001:2022? Then you can reuse much of your document control, internal audits and management review. For risk methods, ISO/IEC 23894:2023 adds advice on AI risk management.

NIST AI RMF

The NIST AI Risk Management Framework (AI RMF 1.0) came out in January 2023. It is voluntary and you cannot get certified against it. But it is a very useful way to think about AI risk. It has four functions:

A practical way is to use ISO/IEC 42001 as the formal frame. Then use the NIST AI RMF as a checklist for the real risk work.

Why a framework makes sense for mid-sized companies

In many companies, AI use grows from the bottom up. Sales tries a text generator. Accounting tries a tool that reads invoices. Developers try a coding assistant. Without an AI governance framework, nobody knows which data flows where. Nor does anyone know which EU AI Act duties apply.

The framework gives you that overview. It makes sure that similar cases are decided in the same way. It also makes customer questionnaires and supplier audits easier. More and more of them ask how you handle AI.

Setting it up in six steps

1. Take stock: Build the AI inventory. Include shadow AI that people use without approval.
2. Classify: Give each system its role (provider or deployer) and its risk class under the EU AI Act: prohibited, high, limited (transparency) or minimal.
3. Define roles: Name owners and write down who may decide what, for example in a RACI matrix.
4. Bring in the policy and approval process: Use a light track for low-risk tools. Use an in-depth review for the rest. Where needed, this includes a data protection impact assessment under Art. 35 GDPR.
5. Build in controls: Set technical rules (access rights, logging, data classification). Add contract terms for AI vendors.
6. Review and improve: Review the framework once a year. Run internal audits. Update it when the law changes.

For high-risk systems, note the deadlines as amended by the Digital Omnibus (in force since 27 July 2026):

• High-risk AI under Annex III: from 2 December 2027.
• High-risk AI in regulated products under Annex I: from 2 August 2028.
• The transparency duties under Art. 50 have applied since 2 August 2026.

Conclusion

An AI governance framework makes AI use easy to manage. Its core is clear roles, an up-to-date inventory and an approval process based on risk. SecTepe helps companies set it up. We focus on fitting it into an existing ISO/IEC 27001 ISMS.

Frequently asked questions

Do we need ISO/IEC 42001 certification?

The law does not require it. It can make sense if customers ask for proof or if you sell AI products yourself. Many companies start by following the standard without seeking certification.

How does the framework differ from the role of senior management?

The framework describes structure, processes and roles. Management approves it, provides the resources and carries overall responsibility. The named roles handle the day-to-day work.