AI Governance in Healthcare Isn’t a Policy Document. It’s an Audit Trail.

Ask a room of hospital CIOs whether their organization "has AI governance" and almost everyone nods. Ask what happens the day an auditor asks for the decision trail behind one specific AI-flagged denial, and the nodding slows down considerably. That gap, between having a governance story and having a governance artifact, is where most of healthcare IT is sitting right now, and it's a more uncomfortable place to be than the surveys let on.

Three in four health systems have implemented or are actively rolling out at least one AI tool. Governance hasn't kept pace, and not by a little. The rest are running AI in production with no documented review process behind it, and telling themselves that's a next-quarter problem.

We used to think of that gap as a best-practice conversation, the kind of thing you get to when you get to it. In 2026, it became a legal one.

The regulatory stack got real

A single AI-assisted prior authorization or documentation tool can now sit inside four overlapping regulatory regimes at once — FDA, ONC's HTI-1, CMS, and increasingly, state law, which is the one nobody budgeted time for.

More than 240 AI-in-healthcare bills moved through 43 state legislatures in 2026. Read enough of them and one pattern keeps showing up: a licensed human has to own any medical-necessity denial an AI tool influences, on paper and in practice. California's SB 1120 set an early template — AI can inform a utilization review decision, but the tool has to weigh the actual patient's clinical history, not just a population dataset, and a licensed clinician makes the final call, with their name attached to it. Colorado's HB 26-1139, signed this June and effective January 1, 2027, goes a step further, requiring disclosure of AI use to state regulators and flatly barring payment for AI-delivered psychotherapy.

None of this is really a debate about whether AI is allowed. It's a question about whether your organization can produce, on demand, who reviewed a given AI-assisted decision and what its documented limitations were at the time. Most organizations we've talked to could answer that in a meeting. Fewer could answer it with a document.

Why "we have a committee" isn't the answer

Here's the number that stops us every time we look at it: 84% of organizations have a governance committee, closer to 12% have an operational framework that turns what the committee talks about into something a system enforces. A committee that meets quarterly and reviews a slide deck isn't nothing, but it doesn't produce an audit trail. It produces the appearance of oversight, right up until someone, a regulator, a plaintiff's attorney, your compliance officer on a bad Tuesday, asks for the thing behind the appearance.

A committee that meets quarterly and reviews a slide deck doesn't produce an audit trail. It produces the appearance of oversight.

We've sat in rooms where a governance committee genuinely believed its quarterly review was the control. Good people, who had never been asked the follow-up question: show me the log. Nobody built the part of the system that would answer that automatically, so the committee became the entire governance program instead of one piece of it.

What actually holds up

A structure that survives an audit tends to have the same handful of things underneath it. None of them are exotic. What they share is that they're built into the workflow, not written about it after the fact.

Governance as a document that sits next to the system fails the first time someone asks for a specific decision trail, because nobody built the thing that would produce it. Governance built into the architecture produces that trail as a byproduct of how the tool works every day, not as a special project the week before an audit. That's the whole difference between AI governance as a compliance checkbox and AI governance as infrastructure, and it's the difference a growing number of states are now writing directly into law.

We keep finding ourselves early on gaps like this one, a few months before they turn into a requirement, and we'd be lying if we said that felt like foresight every time. Mostly it's just what happens when you spend enough years in the seam between clinical workflow and the systems that get audited for it.

Which AI tools in your organization could produce that audit trail today, on request, without anyone scrambling to reconstruct it?