Babak Hodjat, Chief AI Officer at Cognizant: A Return Conversation

Babak Hodjat, Chief AI Officer at Cognizant, leads the company’s AI Research Labs, where his work spans agentic AI, evolutionary computation, distributed intelligence, and the development of AI systems designed for large-scale enterprise deployment. A longtime AI researcher, inventor, and entrepreneur, Hodjat previously co-founded Sentient Technologies, where he helped build one of the world’s largest distributed AI systems, and earlier co-founded Dejima, developing agent-oriented natural language technology that contributed to the foundations of Apple’s Siri.

When we previously spoke with Babak Hodjat, our discussion explored his career in AI, evolutionary approaches to complex decision-making, Cognizant’s work applying AI to climate and land-use challenges, and the growing democratization of AI across the enterprise. In this follow-up conversation, the focus shifts toward the rapidly emerging agentic enterprise. Drawing on themes from his recent book, The Agentic Enterprise, Hodjat examines why architecture may matter more than model capability, how organizations can coordinate increasingly complex networks of AI agents, where centralized trust and governance fit into decentralized agent ecosystems, and what still needs to change before autonomous systems can reliably manage long-running objectives with minimal human intervention.

Your career spans some of the earliest work on intelligent agents, from Dejima’s natural-language technology that helped inspire Siri to building massive distributed AI systems at Sentient and now serving as Chief AI Officer at Cognizant. How has your definition of an “AI agent” evolved over that journey, and what can today’s developers learn from those earlier generations of agent-based systems?

The definition has stayed the same, but the capabilities of agents today mean that many of the basic functions of an agent that we had to code back in the 90’s are available out of the box with LLMs. This is why LLM-based agents are the standard now, and when people talk about agents, they assume there’s an LLM driving its interface and reasoning.

In your recent book, The Agentic Enterprise, you argue that enterprise AI systems often succeed or fail because of their architecture rather than the raw capability of the underlying model. What are companies getting wrong when they assume that upgrading to a more powerful model will solve their agentic AI problems?

Just as harnesses (i.e., non-LLM parts of an agent) can make or break a coding agent, the architecture of a multi-agentic system is critical to its success. This is because the architecture reflects the context within which agents should operate – sort of like the scaffolding that holds the agentic enterprise together. Without it, agents remain ungrounded in the realities of the enterprise operations, data, workflows, and processes. This is somewhat analogous to the organizational structure of a business, which includes the job descriptions, process definitions, and communication protocols for the employees.

You have emphasized that the surrounding system, including access controls, accountability, orchestration, observability, and oversight, can matter more than the model itself. What does a production-ready agentic architecture actually need that most proof-of-concept deployments are missing?

Proof of concepts (PoC) are often ungrounded, so to start, we need the agents to operate in a setting where they have access to the company’s internal data and applications. Important questions need to be asked, such as what data and applications are we prepared to allow the agents to access, and what decisions are we comfortable allowing the agents to make so that the system is safe and secure, while the quality and productivity gains we expect can still be achieved? It is hard to determine the answer to these questions in a PoC setting. Furthermore, guardrails and oversight can only be meaningful when an agentic system is operating in conjunction with other human or agentic processes within the company. Sandboxing, therefore, is an important step in the process of effectively moving from a PoC to deployment.

Cognizant has described the emergence of an “Agent Operating System” for coordinating networks of AI agents. What functions should such a layer perform, and do you expect agent orchestration to eventually become as foundational to enterprise computing as operating systems and cloud platforms are today?

The coordination of agents is important: make it too rigid, and your system is degraded and brittle. Too open with coordination completely delegated to the agent, and you risk making the agent’s task too complex or risky. A good coordination mechanism allows agents to spontaneously form teams in response to various situations, resulting in a robust and fluid organizational discipline. I’m a proponent of distributed coordination. It is analogous to empowering a human organization with some level of authority over its domain of responsibility. When a context presents itself, members of the organization communicate and coordinate over the organizational structure to determine the individuals responsible for handling the situation, and the response is collated, coordinated, and consolidated between the organizational nodes. The good news is that strict controls and safeguards can be applied to agentic systems so that such flexibility does not result in risky behavior.

I do think, just as MCP has rapidly become a de facto standard for agentic application access, coordination protocols will also emerge and become standardized across agentic platforms to allow for spontaneous multi-agent processes across agentic system boundaries. Operating systems and cloud platforms are probably not the best analogy here; rather, widely agreed-upon standards for multi-agent definition and coordination protocols will have to emerge to allow wider penetration and flourishing of agentification in the enterprise.

As enterprises move from individual copilots to networks containing dozens or even hundreds of specialized agents, how should organizations decide which responsibilities belong to individual agents and which should remain centralized?

Just like the internet firewall and security machinery, the establishment of trust through live auditing against policies and enforcing guardrails on agentic systems should be a centralized function within a business, defined and maintained by a single entity within the company. All other agentic systems can be provisioned or implemented in a decentralized manner, but these systems will have to register with the trust system to be allowed to operate.

Cognizant has deployed a multi-agent system internally that connects roughly 200 agents across an organization of about 350,000 employees. What did operating at that scale reveal about agentic AI that would be difficult to discover through smaller pilots?

We had to design for scalability, extensibility, cost efficiency, and reasonable response times. In projects of such scale, you often must invent things that are simply not needed in small, contained agentic systems that have few users. For example, you have to make inter-agent coordination personalized and efficient, so extra steps and cost are avoided. Also, the system should be able to pull in custom-built apps as well as third-party systems and agents, so interoperability is important and can save time and money.

When autonomous agents can call APIs, access enterprise systems, communicate with other agents, and take actions at machine speed, traditional AI governance becomes much harder. Where should human oversight remain mandatory, and where can organizations safely allow agents to operate independently?

I think the ultimate determinant of where human oversight is needed and mandatory is trust. As these systems become more trustworthy, we will allow them more autonomy, and unless legally mandated (e.g., due to lack of personhood for AI), we will be comfortable deferring to agents. Measuring trust and risk is therefore important and can help draw the line between autonomy and human oversight — a line that might start moving as these systems become more trustworthy and less risky to run autonomously.

Much of your earlier research focused on evolutionary computation and systems that improve through selection and adaptation. How might evolutionary approaches complement large language models and reinforcement learning as we try to build agents that can continuously improve rather than simply execute predefined workflows?

The mainstream in AI today is dominated by gradient descent approaches, which are a form of hill-climbing search that is not very creative. Evolutionary computation is highly efficient at searching complex search spaces and can complement mainstream approaches. Our lab, for instance, has had a recent breakthrough in this regard: we’ve been able to show that Evolutionary Strategies are superior to Reinforcement Learning for most LLM post-training, resulting in equal or better-quality models at much lower costs. Evolutionary approaches might also help with Test-Time Training, which aims to allow LLMs to learn post-deployment.

Cognizant’s research has explored environments where interacting agents develop cooperation, competition, shared knowledge, and even deceptive strategies. As multi-agent systems become more complex, could emergent behavior become one of the biggest challenges in keeping enterprise AI predictable and aligned with organizational goals?

This is already the case, as evidenced by the much-publicized recent OpenAI/Hugging Face incident. We might think that agents, confined to their prompts and contexts, and utilizing LLMs that don’t learn at runtime, will remain more or less consistent in their behavior. However, the fact is that an agent’s environment and motivations can change, often as a result of the agent’s own actions in the environment, resulting in unforeseen behavior. This is even more likely to happen as multiple agents interact, and so it is critical for us to observe autonomous agents in safe sandbox settings so we can determine the scope and scale of their emergent tendencies, and to account for that in our agentic designs and safeguards. Our publicly available system, TerraLingua, where you can introduce agents, artifacts, and even impersonate agents and observe their individual and collective behavior, is a step in this direction.

Looking beyond today’s wave of agentic AI, what technological breakthrough or architectural shift do you believe is still missing before we can build AI systems capable of reliably managing long-running, complex objectives with minimal human intervention?

Many of the limitations of today’s agents stem from the limitations in the state of the art in AI. LLMs are pretty much fixed after training and cannot ‘change their minds’ about things as they encounter new information. LLMs are everything and everyone all at once, and so they need to be primed with context and prompts and are quite sensitive to this priming. LLMs have limited context windows and need to be fed everything all over again for every single word they produce or action they take. At a multi-agent level, I think we need widely accepted distributed coordination protocols, trust guardrails and policies, and transaction arbiter systems in order for the agentic future to meet its full potential.

Thank you for the great interview, readers who wish to learn more should visit Cognizant or our our previous interview.