Ask a compliance officer whether the organization has an AI governance policy, and the answer may well be yes. Ask who owns the policy, when it was last approved, which tools have gone through a documented review, and whether the resulting risk decisions can be produced on request — and the answer can get a lot less certain.
That’s the difference mature AI due diligence is designed to surface. A policy proves an organization stated an intention. A paper trail shows how that intention was applied to real tools, real decisions, real risks.
We’ve argued before that AI governance has to be architecture, not paperwork — that a committee meeting quarterly and reviewing a slide deck produces the appearance of oversight, not a defensible record of it. This piece gets more concrete about what “defensible record” means. It isn’t a universal four-document legal checklist. It’s a practical evidence set drawn from the recurring expectations behind ISO/IEC 42001, NIST AI RMF implementation, and mature AI third-party-risk practice.
What Governance Has to Prove Now
Public AI vendor-security questionnaires already ask about data handling, training-data provenance, model transparency, bias testing, and certifications. What’s newer is the expectation sitting behind those questions: not whether an organization can name a policy, but whether it can show how governance actually operates.
Across ISO 42001, NIST AI RMF implementation guidance, and mature third-party AI risk practice, four evidence categories keep recurring. They aren’t a standardized questionnaire every reviewer runs. They’re the records an organization needs if it wants to demonstrate that governance is operating, not aspirational.
Governance policy and version control. An effective date, an owner, an approval history, and a scope that says which AI uses the policy actually covers.
Defined ownership and decision authority. A named accountable executive or a governance body with an actual mandate — and documentation of the decisions or approvals that mandate has produced. A chartered committee is one valid model. A single named accountable executive can be another, particularly for a smaller organization.
Tool intake and approval. A risk-based review with named steps and named owners, involving whichever of legal, privacy, clinical, security, and operational stakeholders are actually relevant to a given tool — not every tool needs every reviewer, but the review itself needs to be a record, not a recollection.
Risk and impact evidence. A tool-specific risk or impact assessment, the mitigation decisions that came out of it, and a plan for monitoring or reassessment — not just the ability to say a framework’s name out loud.
Why the Fourth One Keeps Tripping People Up
Censinet survey findings illustrate the gap this evidence is meant to close: 84% of respondents reported having an AI governance committee, while a separate Censinet analysis found only about 12% of U.S. hospitals had a formal, documented AI governance framework (Source: Censinet, separate publications, 2025). These come from separate publications, not one matched survey cohort, so the honest reading isn’t “72% of the same hospitals have a committee and nothing else” — it’s that having a committee and having an operating framework are different claims, made by different organizations at different rates.
In MGMA’s January 2026 poll of medical-group leaders, a related but distinct pattern shows up: 20% said their organization already had AI governance or a formal AI policy, 22% were developing one, and 56% reported neither (Source: MGMA Stat, Jan. 2026, medical-group leaders). Different sector, same underlying shape — naming an intention and having it operating are not the same milestone, and a meaningful share of organizations haven’t reached either one yet.
NIST AI RMF is voluntary and outcomes-based, not prescriptive. It doesn’t tell a hospital to maintain a particular binder or stand up a particular committee — it organizes governance around four functions (Govern, Map, Measure, Manage) and leaves the specific documents to the organization implementing it. What that means in practice: an organization operationalizing NIST AI RMF still needs enough tool-level evidence — inventories, risk records, oversight logs, monitoring notes — to demonstrate those four functions are actually happening, not just named. Mitratech’s assessment of third-party-risk programs is that many haven’t caught up to that distinction yet, and that pre-2024 vendor AI questionnaires often don’t reflect it.
ISO/IEC 42001 is a useful anchor here because it treats governance as a management system rather than a one-time policy exercise. Its structure calls for organizations to establish processes for AI risk assessment, AI impact assessment, and risk treatment, with documented results and lifecycle controls reviewed when relevant changes occur. In healthcare, those risk and impact assessments are often evaluated through safety, privacy, fairness, security, and clinical-use lenses — that’s a reasonable implementation choice for the sector, not language ISO’s standard itself prescribes.
| Evidence Category | What a Defensible “Yes” Can Include |
|---|---|
| Governance policy | Effective date, owner, approval history, revision history, scope of covered AI uses |
| Decision authority | Named accountable executive or governance body; defined mandate; documented decisions or approvals |
| Tool intake and approval | Risk-based review steps, owners, implementation conditions, approval/exception record |
| Risk and impact evidence | Tool-specific risk/impact assessment, mitigation decisions, monitoring or reassessment plan |
This is a practical evidence model, not a universal legal checklist. Required records vary by tool risk, care setting, data use, regulatory obligations, contracts, and organizational governance model.
Build It or Subscribe to It?
Here’s where the gap gets harder to close than a better policy template. A packaged AI tool bought off your EHR vendor’s own marketplace comes with someone else’s governance answers already baked in, if it comes with any at all. The risk assessment, if one exists, belongs to the vendor’s roadmap, not yours. That works fine for the slice of your workflow the product was built to cover. It’s a problem for everything around the edges — the non-standard intake step, the system that survived a merger, the workflow the packaged product assumes you have and you don’t. Nobody on your side owns an assessment for that layer, because nobody on your side built it.
That’s the fourth evidence category’s actual failure mode, not a paperwork gap. You can’t produce a tool-specific risk assessment for a black box you licensed monthly and don’t control end to end. You can produce one for something you built.
We can figure out and build the custom version of that workflow — the one that doesn’t exist as a packaged product because your configuration doesn’t match what the packaged product assumes. Built against your systems, owned by you, not licensed back to you monthly, the intake step is yours to document because you built it, and the risk assessment stays current because it’s tracking your system, not a vendor’s release notes. Interoperability isn’t a feature line in that version. It’s the deliverable that makes a real fourth-category answer possible in the first place.
The Policy Nobody’s Opened Since It Was Approved
In governance work, a recurring pattern isn’t a poorly written policy. It’s a policy approved once and left disconnected from the AI tools deployed afterward — the policy predates half of what it’s now supposed to cover. The governance group may keep meeting, but unless decisions, exceptions, and approvals are actually retained, the organization can’t easily show what was reviewed or decided, or when.
That’s the version of “we have AI governance” that survives a casual conversation and struggles the moment someone asks for the evidence behind it. The fix isn’t a bigger policy. It’s the same shift we’ve described before — governance built into how tools actually get evaluated, approved, and monitored, so the record is a byproduct of the workflow instead of a project someone starts the week a reviewer calls.
If a due-diligence request landed on your desk this week, which of these four categories could you actually back with a document, by Friday?
Frequently Asked Questions
What is an AI governance “paper trail,” and how is it different from a policy?
A policy is a document that states intent — who’s allowed to approve an AI tool, in principle. A paper trail is the evidence that intent was actually applied: version history on the policy itself, a named executive or governance body with a real mandate, a documented intake and approval record for a given tool, and a tool-specific risk or impact assessment. Mature AI due diligence increasingly asks for the second, not just the first.
Does NIST AI RMF require an organization to keep specific documents?
No. NIST AI RMF is voluntary and outcomes-based — it organizes governance around four functions (Govern, Map, Measure, Manage) and leaves the specific documents to the organization implementing it. An organization still needs enough tool-level evidence — inventories, risk records, monitoring notes — to demonstrate those functions are actually happening, not just named.
What does ISO/IEC 42001 actually require for AI risk assessment?
ISO 42001’s structure (clauses 6.1.2–6.1.4) calls for organizations to establish processes for AI risk assessment, AI impact assessment, and risk treatment, with documented results and lifecycle controls reviewed when relevant changes occur. It doesn’t prescribe an exact review cadence — that’s an implementation choice left to the organization.
What share of hospitals actually have a formal AI governance framework?
Roughly 12%, per a Censinet analysis — compared with 84% of respondents who reported having an AI governance committee in a separate Censinet survey. The two figures come from different publications, not one matched cohort, so the reliable reading is that having a committee and having an operating framework are different, unequally met milestones (Source: Censinet, separate publications, 2025).
Is a packaged AI tool from an EHR vendor’s marketplace enough to pass an AI governance review?
It can cover the risk assessment for the workflow it was built for, but the vendor owns that assessment, not the buyer — and it won’t produce evidence for non-standard intake steps or systems the packaged product wasn’t built to handle. A custom-built tool produces that evidence as a byproduct of the buyer owning the workflow end to end.
Ready to talk?
Could you produce a tool-specific AI risk assessment on request?
If the workflow that would produce this evidence doesn't exist as a packaged product for your configuration, we can build it — owned by you, not licensed back to you monthly.
Sources
- Itirra — “AI Governance in Healthcare Isn’t a Policy Document. It’s an Audit Trail.”
- “AI Risk Management Framework” — NIST — voluntary, outcomes-based structure (Govern/Map/Measure/Manage); does not prescribe specific documents
- “AI Adoption Survey Reveals Healthcare’s Governance Gap and Drive Toward Agentic Usage” — Censinet — 84% AI governance committee, 59% formal documented process; separate Censinet analysis: ~12% of U.S. hospitals have a formal AI governance framework
- “AI Governance in Medical Group Practices: Rules for the Humans in the Loop” — MGMA Stat, Jan 20, 2026 — 20% have AI governance/formal policy, 22% developing, 56% neither, 2% unsure (medical-group leaders, n=328 applicable responses)
- “How to Assess and Treat AI Risks and Impacts with ISO 42001” — Schellman — ISO 42001 clauses 6.1.2–6.1.4 risk assessment, AI impact assessment, risk treatment, documented information, lifecycle review
- “The NIST AI RMF and Third-Party Risk: An Implementation Guide for TPRM Programs” — Mitratech — assessment that many TPRM programs have not caught up to 2025’s expanded NIST AI RMF guidance
- “Third-Party AI Vendor Due Diligence Questionnaire” — Ampcus Cyber — one example of an AI vendor due-diligence questionnaire template (data handling, training data, model transparency, certifications, NIST/ISO alignment)