Thought Leaders

Before You Scale AI Agents, Ask What They Don’t Know

Dror Nadler
Add Unite.AI to your preferred sources on Google
Key PointsConfident answers fail when agents omit context like stale filing dates or fiscal…
View more
  • Confident answers fail when agents omit context like stale filing dates or fiscal calendar differences.
  • Nearly 6 in 10 businesses traced a confident-but-wrong AI answer to missing or incomplete business context.
  • Human review must scale with stakes because correct data shared wrongly can still violate regulations.

Picture this: A CFO is getting ready for a meeting with a major institutional investor. To help with prep, the investor relations (IR) team asks an AI tool to pull some background—the date of the last meeting with that firm, priorities expressed, number of shares held, and so on.

The tool confidently states that the investor owns 2.5 million shares. No one bats an eye, and the CFO continues preparing for this important conversation, full speed ahead.

But there’s a catch: The AI pulled the figure from a Form 13F filing, which reports holdings as of the previous quarter-end. The information was accurate back when it was reported… but now, it’s months old. In the meantime, the investor has quietly sold 2 million shares, reducing their position significantly. The meeting agenda (and potentially the meeting itself!) no longer seems relevant.

What went wrong? In this scenario, the AI didn’t hallucinate or fabricate a number. It retrieved legitimate information but failed to qualify it or explain the limitations.

While in this case, the consequences might be an awkward conversation or missed opportunity, in other situations, a similarly definitive answer that lacks critical context can carry even greater risks.

Better Models Aren’t Necessarily the Answer

Foundational models are getting better and better. So, why does it seem like confident-but-wrong answers are popping up more and more—something businesses across industries are grappling with?

For starters, generative AI (GenAI) has become completely pervasive. According to McKinsey’s The State of AI in 2026 report, nearly 9 in 10 business professionals say their organizations regularly use AI in at least one business function.

And as organizations move beyond experimentation—increasingly relying on AI for everyday business tasks and decisions— they’re also encountering the limitations of what these tools know and understand.

A recent industry survey shows what’s often at the root of it: In the past six months alone, nearly 6 in 10 businesses (57%) traced a confident-but-wrong AI agent answer to missing or incomplete business context.

With all the data available to AI, this may come as a surprise. But there’s an important distinction to be made: Context isn’t the same thing as data.

What Context Is—And Isn’t

Context can encompass institutional knowledge, exceptions to the rules, relationships, processes and expectations around how work gets done—nuances that are often unique to each company.

In my work with investor relations teams, I see just how important these distinctions can be. Consider a few seemingly straightforward examples:

  • Fiscal calendars: One company’s first quarter might align with the calendar year, while another’s begins in September. Without that context, an AI agent could compare results from the wrong reporting periods.
  • Financial metrics: An organization may report both GAAP and non-GAAP earnings per share. An agent that doesn’t understand which is relevant to cite (and when) could misrepresent business performance.
  • Company narratives: A company’s messaging evolves over time, reflecting deliberate decisions about what to emphasize and leave out. Without understanding the history and reasoning, an AI agent could generate content that actually undermines a carefully developed narrative.

Given all this, it’s no surprise that the same underlying model can deliver very different results at different companies, depending on the context and specialized knowledge available to it.

Purpose-built, industry-specific (or “vertical”) AI can help bridge some of these gaps, recognizing the specialized knowledge, workflows and nuances of a given sector. But even the most sophisticated technology can’t capture everything an organization knows, which is why humans have a part to play too.

What AI Doesn’t Know About Your Business

But humans don’t stick around forever. Consider what happens when an experienced investor relations officer (IRO) leaves their company after, say, five years. They’ve likely spent their time identifying and targeting prospective investors, analyzing shareholder activity, cultivating relationships, preparing for earnings, prepping execs… and a whole lot more.

The company retains the earnings scripts, presentations, and other files. But the institutional knowledge behind them? Much of that walks out the door too, especially when you consider that nearly eight in 10 IROs (78%) don’t have a successor identified for their roles.

That’s why organizations need to put in the work to preserve knowledge and understanding in a way that AI systems can use. Simply connecting an AI agent to a shared drive often isn’t enough.

Having a way to systematically capture key knowledge and workflows—like the reasoning behind key decisions, the history of relationships, and other mission-critical information that never made it into a formal document—is important to businesses across industries.

And powering such a knowledge base requires people: both for recording information and applying judgment around it. AI can help organize knowledge, but it’s employees who provide the key context that technology often misses.

In High-Stakes Environments, Context Gaps Really Hurt

We’ve discussed what happens when AI gets an answer wrong. But sometimes, an AI agent can retrieve information that’s right—it’s accurate, current and relevant—and still create problems by sharing it under the wrong circumstances.

In highly regulated industries, those missteps are especially costly.

In investor relations, for instance, sharing material nonpublic information (MNPI), such as unreleased financial results, with an analyst or investor can violate the U.S. SEC’s Regulation Fair Disclosure (Reg FD)—potentially resulting in financial penalties and reputational damage.

Other industries like healthcare, financial services, and legal services face similar regulatory risks, highlighting the need for guardrails when AI agents are privy to sensitive information.

This also shows why humans remain crucial—an essential part of reviewing and approving what gets communicated. Companies may consider making the level of human review proportionate to the stakes, with different types of AI workflows operating with different levels of autonomy.

Three Questions to Ask Before Scaling AI Agents

We’ve probably all encountered situations, in work or life in general, where we don’t know what we don’t know. Sometimes the ramifications aren’t consequential; other times they’re more serious.

With such massive datasets available to AI today, AI models know a lot more… but still need to “speak up” when holes exist. So, as businesses use AI to tackle more strategic priorities and workflows, it’s important to ask these questions before scaling:

  • Can our agent show where its answers came from—and when that information was last updated? Can it provide this information without being asked?
  • Does the agent have the necessary business context to interpret and respond to requests? Will it outline its limitations?
  • Do we know when a human needs to step in for review and approval? Under what circumstances will that occur, and how will the review be triggered?

The Answer Isn’t to Avoid AI

I want to be clear: I’m bullish on AI. I use it daily in my role, see the current and future benefits, and think that—in many ways—we’re just scratching the surface of AI’s potential. I have no doubt that companies that avoid AI, and those that fail to implement it in an organized, effective way, run a real risk of falling behind.

But adopting AI doesn’t mean giving a general-purpose model access to enterprise data and letting it operate willy-nilly.

Right now, and as far as I can see, fully autonomous AI isn’t appropriate for every business task or decision. Organizations need to understand where AI can operate independently, where it needs additional context, and where human judgment is non-negotiable.

So, before asking what an AI agent knows, ask what it doesn’t. The answers shape how you deploy it, what you trust it to do, and how you set it up for success.

Dror Nadler is Chief Product Officer at Q4 Inc., where he leads product strategy and innovation, helping investor relations teams harness AI, data and technology to work more effectively. He brings more than 25 years of experience in enterprise technology, SaaS and financial services, with previous leadership roles at Intralinks, Cellrox, Rapid7 and EMC. Throughout his career, Dror has focused on developing and scaling technology products, driving innovation and accelerating business growth. He holds an executive MBA from Boston University, a master’s degree in information technology from Harvard University, and a law degree from Reichman University.