AI Still Needs a Coach and Human Oversight Needs a Playbook

AI is everywhere you look, and professional football is no exception.

The NFL has introduced the Digital Athlete, which uses data and AI to help teams identify scenarios in which players may face an elevated risk of injury. It combines video and data from training, practices, and games with millions of simulations, giving all 32 teams access to league-wide trend information. Coaches and training staff then use that information to develop individualized injury-prevention, training, and recovery programs.

While the technology reads the field, coaches are still responsible for making the calls. As organizations expand AI use across business processes, including compliance and workplace investigations, the potential to automate feels limitless. But which tasks should AI perform, under what conditions, and with what human oversight?

That’s where an organizational playbook can set the tone: offering coaches the means to leverage AI tools while maintaining human oversight where it counts.

Give AI a Position Rather Than the Whole Field

Investigations consist of varying levels of risk. AI can help organize information from initial reports and:

• Identify missing details
• Summarize information into a final report
• Build timelines
• Surface patterns that investigators may want to examine further

These use cases can reduce manual sorting and free up time to focus on work that requires human judgement.

Consider that AI can summarize an allegation, but it cannot decide whether it’s credible. AI can identify an inconsistency, but cannot determine why it exists. This is where organizations can get into trouble by simply using AI without guardrails.

To maintain oversight, AI needs a defined role. Organizations should decide:

• The tasks the system is meant to perform
• The information it can access
• The boundaries of what it’s allowed to do
• The situations at which a person needs to take control

The National Institute of Standards and Technology’s AI Risk Management Framework provides a useful benchmark. NIST’s roadmap describes AI RMF Profiles as a way to demonstrate how the framework can be applied to particular sectors, technologies, contexts, or other settings. In other words, responsible AI governance becomes more actionable when organizations translate broad principles into the realities of a specific use case.

An AI playbook can do something similar at the operational level. It defines roles, situations, and actions with specificity.

Human Oversight Is a System Design Problem

The phrase “human in the loop” has become commonplace in discussions about responsible AI. But the presence of a person somewhere in a workflow does not automatically make an AI system well-governed.

NIST has identified human-AI teaming as an important area for AI risk management research and guidance. Among the issues it highlights are how people interpret AI-produced outputs and how humans oversee AI systems operating in real-world environments.

Human oversight should not be treated as a temporary workaround until AI becomes sophisticated enough to operate alone. In many high-stakes environments, it is part of the required architecture to use AI responsibly in the long term.

A peer-reviewed examination of human oversight in AI governance notes that human overseers are expected to improve accuracy and safety to uphold human values and build trust. However, humans are not always reliable overseers. Effective oversight depends on competence, incentives, and the design of the oversight role.

For an investigator, being asked to approve an AI-generated recommendation is not enough if they lack the information or authority necessary to challenge it. The playbook must define not only that a human is involved, but how and when that person can intervene.

The Handoff Matters as Much as the Model

There is another challenge: even when a human has final authority, the way AI presents information can influence the decision that follows.

A 2025 preprint study from researchers at the University of California, Santa Barbara illustrates why interface design matters. In a controlled experiment involving 108 participants in a healthcare decision-making scenario, researchers tested six forms of AI decision support. They found that mechanisms, including AI confidence information, textual explanations, and performance visualizations, improved human-AI decision accuracy in their experiment.

The study also highlights the risk of automation bias: people can over-rely on AI suggestions, including incorrect ones.

This experiment demonstrates why the handoff between machine and human requires deliberate design and testing.

When we extrapolate this further and introduce the concept of an investigation, several questions arise:

• Does an investigator see only an AI-generated conclusion, or can they examine the information behind it?

• Can the system communicate uncertainty?

• Is it clear when information is missing?

• Can investigators reject an output and document why?

Those are governance questions as much as technical ones.

Five Questions Every AI Playbook Should Answer

A useful AI playbook should turn abstract principles such as “responsible AI” and “human oversight” into concrete operating rules.

1. What is the AI allowed to do? Organizations should authorize specific tasks rather than grant broad permission to use AI throughout a workflow.
2. What information can it use? For investigations and whistleblowing, organizations need clear boundaries around the information available to an AI system and how that information is handled.
3. How will people evaluate its output? Human reviewers need enough context to understand what the system produced, its relevant limitations, and the information necessary to question the result.
4. When must a human intervene? A playbook should establish escalation triggers rather than relying on individual employees to decide when AI has gone too far.
5. Where does AI’s authority end? Some decisions should require human judgment, and the people responsible for those decisions need genuine authority to disregard, override, or stop the technology.

These ideas are reflected in one of the world’s most significant pieces of AI regulation. Article 14 of the EU AI Act establishes human-oversight requirements for AI systems that fall within the Act’s high-risk category. It says oversight measures should be proportionate to the system’s risks, autonomy, and context of use. It also calls for overseers to be able to understand relevant capabilities and limitations, correctly interpret outputs, remain alert to automation bias, disregard or override outputs, and intervene in or stop a system.

That does not mean every AI tool used in an investigation is automatically a high-risk system. The framework illustrates an important principle for AI governance. Human oversight has to provide humans with meaningful capabilities, not merely a place in the workflow.

AI Doesn’t Replace the Coach. It Changes What the Coach Can See.

The NFL’s Digital Athlete is compelling not because it demonstrates that AI can replace coaches, but because it shows how sophisticated technology can expand the information available to experienced professionals.

AI can help people work through information, surface what deserves attention, and create greater consistency across repeatable tasks. Yet, the most consequential moments in an investigation often require something different: context, skepticism, interpretation, and accountability.

The organizations best positioned to benefit from AI will therefore ask more than what the technology can automate. They will define what it should do, what it should not do, and how humans and AI should work together.