Thought Leaders

Rethinking AI Resilience: Why It’s Time to Abandon the ‘Set It and Forget It’ Model

Published
Add Unite.AI to your preferred sources on Google

A recent Gallup poll points to a trend of growing AI adoption, as more than half of employed adults say they use the technology at least a few times a year. This data is just a hint of the rapid AI adoption happening across enterprise environments. Although multiple enterprise leaders have begun to question the ROI of AI, it hasn’t stopped organizations from leaning on the technology to streamline manual tasks and maximize productivity.

While this adoption continues, organizations can be tempted to ignore harmful missteps when it comes to automated workflows. They look at AI as a turnkey solution, requiring IT teams to only set the technology, let it run, and “forget” it exists. Some security teams try to combat this thinking, creating automation playbooks meant to promote safe and secure AI use. Still, even the best of IT and security teams leave little room for adjustment in these playbooks. Therefore, enterprises are unable to safely evolve with AI as the technology itself does the same.

As enterprises grant AI more authority in business-critical workflows, operational agility must be the new mandate for today’s organizations. For this to become a universal practice, enterprise security leaders must abandon a “set-it-and-forget-it” mindset and engineer for resilience.

Understanding the Automation Paradox

AI-powered automation is providing massive productivity gains throughout multiple enterprise functions. Let’s use cybersecurity as an example. AI tools allow analysts to take extra steps they never had time for in overworked and overwhelmed security operations centers. This includes tasks such as alert prioritization, event review, and threat detection, cross reference validation, and getting a second opinion. Unlike some C-suite leaders, CISOs remain generally optimistic about the value AI will provide in the future. According to data from “The CISO Report: From Risk to Resilience in the AI Era,” 83% of CISOs believe automation is the investment most likely to exceed expectations.

In the face of these benefits, a paradox remains. The same report highlights the skepticism CISOs carry about AI, fearing the consequences of autonomous errors. 83% of CISOs rank impacts from hallucinations, such as missed alerts or false positives, as their greatest concern about agentic AI. There is sufficient evidence to back these concerns about hallucinations. A major commercial airline was held legally liable and ordered to pay damages after its customer service chatbot hallucinated a bereavement fare refund policy. A threat intelligence firm is facing legal issues because it allegedly released a report connecting a startup to hackers from a foreign power. When AI gets things wrong, the fallout can extend past internal systems to both capital and reputation.

The Problem With Stale Playbooks

AI-focused security playbooks are a baseline for successful, safe AI adoption. Problems arise, however, when those playbooks never change. Threats, IT infrastructure, and business environments are constantly evolving. AI is enabling threat actors to scale attacks, while expanding and increasingly complex IT environments make detection harder. At the same time, as businesses give AI greater responsibility, the consequences of errors become more significant.

If all of these factors continue to evolve, then automation logic must evolve with them. Frozen automation playbooks prevent this from happening. They leave security teams to face current problems with outdated logic. At times, this scenario may even end worse than if the security team had no playbook at all.

For playbooks to become the living organisms fit for current AI implementation, thereby promoting internal resilience, they need to be:

  • Tested continuously against fresh threat intelligence
  • Updated after real incidents occur
  • Retired once they are no longer effective or provide value

This mindset aligns with guidance from the Cybersecurity and Infrastructure Security Agency (CISA) and other global cybersecurity agencies, which promote continuous monitoring of AI systems and continuous human intervention.

Implementing the Resilience Formula

The AI era calls for enterprises to move on from a mindset rooted in AI as a point solution, or simply slapping automation onto existing use cases. Organizations instead need to build a culture of resilience that allows both IT personnel and systems to enhance where AI gets things right — and respond efficiently when the technology gets something wrong.

A three-part practical formula can help embed this resilience into automated actions.

  • Confidence: For each automated workflow, teams should ask themselves: How confident are we in the trigger that initiated the automated response?
  • Reversibility: Is the action reversible? If so, how quickly and safely can we do this?
  • Blast Radius: How far-reaching are the consequences if an error spreads, and how business-critical is the affected asset?

The right observability and monitoring tools are vital to answering these questions. Teams should leverage a unified observability and data management platform so they can completely understand where in an IT environment an automated response may occur, which other assets it affects, and the associated business consequences. Doing this exercise regularly will allow security teams to maintain up-to-date AI playbooks. Moreover, it will allow proper human-in-the-loop practices, as IT teams will know where and how to intervene to prevent improper automated actions from having widespread and costly effects.

Set It and Forget It No More

It’s important for the modern CISO and IT leaders to realize the buck stops with them when it comes to AI behavior. The more AI touches important enterprise work, the greater the burden for chief cybersecurity officers. A good rule of thumb to remember is, “Automation changes who pushes the button. It does not change who’s accountable for the button getting pushed.”

The stakes and onus on today’s CISO are too high to keep set-it-and-forget-it playbooks in place. Instead, resilience must be both a practice and a culture that permeates an enterprise and allows it to bounce back after AI makes a mistake. When teams place continuous monitoring, reversibility, containment, and human ownership into the automation playbook, enterprises can prevent an automated decision from becoming a business-wide problem.

David Dalling is a subject matter expert with over 20 years of Information Security experience and IT Operations. David joined Splunk in February 2024 after 4 years at Accenture as the Cyber Chief Technology Officer. He is an accomplished, motivated, and versatile IT professional across a variety of fields ranging from hands-on systems development, testing, and management to enterprise-level strategic planning and consultation.

David is a man of firsts — he helped get the DHS Enterprise Security Operations program its first ever ATO, wrote the first ever common control package for DHS, and received a security engineering award for developing a SOC metrics program that contributed to DHS’s first ever perfect scorecard. He then led the development of the first ever Managed XDR service to receive its FedRAMP Authorization.

David is now the GVP of Global Cyber Strategy where he drives strategy and go-to-market efforts for Splunk, working with customers on their north star objectives and roadmaps. Outside work, David channels that same drive into Adventure Racing — trail running, mountain biking, and kayaking 100+ miles through the wilderness.