Humans Hang On for Dear Life as AI Accelerates Software Delivery

For most of the history of software development, people have been the control. A developer makes a change, another person reviews it, someone approves it, and eventually it gets deployed.

AI is accelerating that entire system while we’re still trying to keep people in the middle of it. Developers can now create code and changes in seconds. Agents can work across repositories, tools, infrastructure, and other systems with less human involvement.

Our instinct is to add humans back into the process. We review the pull request, approve the tool call, check the change, and confirm the deployment because we want to make sure the AI didn’t do something it wasn’t supposed to do. We’re hanging on for dear life.

That instinct makes sense. Human review has given us a way to maintain control as software moves toward production. But AI is starting to operate at a speed and volume where humans can’t remain the unit of scale for governance.

AI Is Already Moving Faster Than Human Review

The first wave of generative AI in software development focused mostly on helping developers write code faster. That alone changes software delivery. More code means more application changes, infrastructure changes, and database changes moving through testing, security, review, deployment, and production.

The problem isn’t necessarily that AI creates worse changes. It creates more changes, faster. If the control for all of that new output is another person reviewing every change, eventually the math stops working.

We’re already seeing signs of it. Anthropic recently reported that Claude Code users approve about 93% of permission prompts. The company found that repeated prompts can create approval fatigue, with people paying less attention as the number of approvals increases. Anthropic is now using an automated classifier to evaluate actions and stop potentially dangerous ones rather than asking a person to approve everything.

Think about what that says about human oversight. If someone clicks approve 93% of the time, adding another approval doesn’t necessarily give you more control. At some point, the human becomes another step in the workflow.

We can use AI to create more software. We can’t respond by creating an equally large human review operation behind it.

AI Is Moving From Creating Code to Taking Action

Coding assistants gave AI a role in development. Agents give AI the ability to participate in much more of the SDLC. An agent can receive a goal, decide how to accomplish it, use tools, observe the results, and adjust what it does next.

In software engineering, that can mean modifying files, running commands, interacting with repositories, calling APIs, testing code, or working with infrastructure. People are also getting more comfortable letting agents work on their own. In a study of millions of human-agent interactions, Anthropic found that experienced Claude Code users used full auto-approval in more than 40% of sessions, roughly twice the rate of new users.

That doesn’t mean autonomous agents are running production environments everywhere today. They’re not. But software development gives us an early look at where this is heading.

Today, AI creates more change, and human review starts to strain. Next, AI participates across more of the SDLC. Eventually, agents will create, validate, deploy, observe, and remediate changes with much less human involvement.

At each step, we’re removing another place where a person used to provide control. The question changes from whether AI can do the work to what AI should be allowed to do on its own.

Permission Is Not Authority

Agents need access to do useful work. An agent helping deploy software may need access to a repository, CI/CD system, cloud environment, or database. Take that access away, and you also take away much of what makes the agent useful.

But access and authority aren’t the same thing. Giving an agent permission to reach a system doesn’t mean it should have the authority to take every action available inside that system.

Traditional access control can tell us whether an agent has permission to reach something. We also need a way to determine whether the specific action it wants to take should happen. That becomes more important when the system making the decision can interpret a task differently than the person who assigned it, encounter an obstacle and choose another path, or use a legitimate tool in a way nobody anticipated.

OWASP describes a version of this problem as Excessive Agency. It points to excessive functionality, permissions, and autonomy as causes of damaging actions and recommends independent approval for high-impact actions.

NVIDIA is approaching the same problem at the architecture level. Its Open Agent Safety Platform puts policy enforcement outside the agent and makes a simple point: an agent can’t be expected to fully govern its own behavior.

That should shape how we build the AI SDLC. An agent may need permission to access a database, infrastructure environment, or deployment system. That doesn’t mean the agent should decide for itself that every change it wants to make is safe.

AI makes decisions based on probabilities. We shouldn’t let every one of those decisions automatically become an action against a critical system.

Human in the Loop Can’t Be the Whole Answer

The obvious response is to keep a person in front of consequential AI actions. For some decisions, that’s exactly what we should do. The mistake is turning “human in the loop” into the answer for every decision.

If every action an agent takes requires someone to review it and click approve, we’ve recreated the bottleneck AI was supposed to remove. Worse, enough approvals can turn oversight into habit. A person clicking approve all day isn’t necessarily exercising judgment.

We need to be more deliberate about where decisions happen. AI can make decisions within the job we’ve given it. Policy can handle decisions where the rules are already known. People can handle exceptions and decisions that actually require judgment.

A low-risk change that meets established policy shouldn’t need someone staring at it. A change that violates policy should stop automatically. An exception with meaningful business, security, or operational consequences may need a person to make the call.

That’s a very different model from simply putting a human in every loop. The goal isn’t to remove humans. It’s to stop making human attention the thing every action depends on and make the governed path the easiest path.

Put the Control Where the Action Happens

Enterprises aren’t going to standardize on one AI model or one agent. Developers will use different copilots. Teams will experiment with different models. AI will show up inside developer tools, security products, data platforms, and internal applications.

Trying to build a different governance process around every AI tool won’t scale. The control needs to sit closer to the action the AI wants to take.

If an AI-generated change enters a deployment pipeline, it should face the same policies as a human-generated change. If an agent wants to modify infrastructure, data, or a production database, the controls around that system shouldn’t disappear because the actor changed.

The source of the change doesn’t determine the risk. The change itself does. A developer, coding assistant, automated process, or autonomous agent can take a different path to the same action, but that action can still face the same policy before it becomes consequential.

This also lets the technology change without forcing companies to rebuild governance every time. Models will change. Agents will get more capable. The controls around critical systems can remain consistent.

NIST takes a similar risk-based approach in its AI Risk Management Framework, which treats governance as something that has to operate across the AI lifecycle rather than as a single approval at the end. For software delivery, that means putting controls into the path AI already takes instead of bolting another manual process onto it.

When the Human Leaves, the Evidence Can’t Leave With Them

There’s another problem hiding inside the human review model. When you remove the person from the process, you don’t just lose the review. You can also lose the person who helped prove the review happened.

That becomes a serious issue for companies with security, compliance, and audit requirements. They still need to know what changed, who or what initiated it, which policy applied, whether it passed, who approved an exception, where the change ran, and what happened afterward.

You can’t automate the change and leave the evidence manual. In a human-driven process, teams can reconstruct evidence later from tickets, approvals, pipeline logs, screenshots, and conversations. That approach gets harder as the volume of change grows and becomes unrealistic when machines create and execute changes continuously.

The evidence needs to become part of the delivery process. Policy decisions, approvals, exceptions, deployments, and outcomes should create records as the work happens. Audit evidence becomes a byproduct of software delivery instead of something teams assemble after the fact.

That leaves two different jobs for governance in an AI-driven SDLC. Before an action, determine whether it should happen. After the action, prove what happened.

Humans Aren’t Going Away. Our Job Is Changing.

There’s an understandable instinct to measure control by how many times a person gets involved. More reviews feel safer. More approvals feel safer. Keeping a human in every loop feels safer.

AI is going to test that assumption. If AI keeps increasing the amount of software we can create, humans won’t be able to review every change, approve every action, watch every deployment, and reconstruct every decision afterward. Trying to do that will either slow AI back down or turn human oversight into a rubber stamp.

The AI SDLC needs a different division of labor. AI can handle more of the work while policy governs repeatable decisions and people step in when something genuinely requires judgment. The evidence should get created automatically along the way.

We’re going to give AI more access because that’s how it becomes useful. We’re going to give agents more autonomy because that’s how we get more leverage from them. The challenge is making sure greater access and autonomy don’t quietly become unlimited authority.

Humans don’t need to hang on tighter. The goal isn’t less control. It’s a control model that doesn’t depend on us holding onto every decision ourselves. We need to build the controls that let us loosen our grip without losing control.