Bing Xu, Founder and CEO of INT21 – Interview Series

Bing Xu, Founder and CEO of INT21, is an AI infrastructure engineer and entrepreneur with deep experience spanning GPU optimization, machine learning systems, and autonomous AI agents. Before founding INT21 in 2026, Xu was a Distinguished Engineer at NVIDIA, where he worked on agent-driven software development and created multiple generations of coding agents, including work behind VibeTensor and AVO. He joined NVIDIA following its acquisition of HippoML, the GPU inference startup he founded and led as CEO. Earlier, Xu served as a Senior Staff Software Engineer at Meta, where he created AITemplate and helped improve GPU inference efficiency across production workloads, and held engineering and research roles at OctoML, Facebook AI, Apple, and Turi. His career has consistently focused on improving the software and infrastructure underlying modern AI systems.

INT21 is building self-improving AI infrastructure designed to use autonomous agent swarms to continuously develop, test, benchmark, and optimize the software that powers AI workloads. Its current focus is the Inference Engine Factory, which uses specialized agents to explore optimization strategies in parallel and build complete inference engines spanning CUDA and PTX kernels, drivers, and serving infrastructure, with performance validated directly on target hardware. The company’s technology is built on SwarmOS, a cloud-native platform that allows large numbers of agents to coordinate around measurable engineering objectives while retaining evidence and lessons between generations. INT21 initially demonstrated this approach through its PTX Kernel Factory, which produced and benchmarked GPU kernels across NVIDIA Hopper and Blackwell hardware; the company has since shifted its compute resources toward optimizing complete inference engines.

You founded INT21 after repeatedly encountering the difficulty of hiring specialized infrastructure engineers fast enough to build and optimize AI systems. What convinced you that the solution was not simply better developer tools, but autonomous, self-improving agent swarms capable of taking on this work themselves?

It’s a well-known problem that the demand for AI infrastructure-focused talent across different specializations far outpaces supply, and it’s a challenge I faced firsthand. That’s what motivated me to look at self-improving agent swarms as an option. Over the last few years, I’ve been working on self-improving AI solutions, and the technology is now moving at such a rapid pace that these systems are now truly sufficient to build, run and optimize infrastructure autonomously.

The core philosophy behind INT21 is that like humans, the agents become smarter through accumulated knowledge. This is fundamentally different from legacy developer tools, which are still constrained by how many hours someone can work and the problems they can physically solve. Using self-improving agent swarms means that we are faster, more accurate, and more efficient with every production cycle. Because our agents are capable and unconstrained by existing frameworks, we can build solutions directly from the ground up for each specific workload.

Before INT21, you founded HippoML, which was acquired by NVIDIA just 14 months after launch, and you later became a Distinguished Engineer at NVIDIA. What did those experiences teach you about the bottlenecks in AI infrastructure that ultimately shaped the architecture and mission of INT21?

At HippoML, we specialized in high-performance generative AI inference, building software optimization tools to run large language models faster and more efficiently. We realized we couldn’t scale without relying on talented contractors from Eastern Europe and elsewhere, just to keep pace with the demand for AI inference optimization.

After NVIDIA acquired us, I began building an agent-based development approach inside the company. We published research proving that the harness (the infrastructure layer around a model) is what actually determines agent performance, not the model itself. That was evidence that the real bottleneck in AI infrastructure isn’t finding better models. It’s optimizing the systems that run them.

As models improve, the infrastructure around them has to scale accordingly. But human-driven optimization cycles struggle to keep pace. You optimize a training loop for one model architecture, deploy it, then if the model architecture changes or a new GPU generation arrives, you need to start over. When nowadays there seems to be a new model update each week, this old approach is unsustainable. That realization convinced me that the next breakthrough is in self-improving infrastructure, which led to me founding INT21.

INT21 describes its approach as “self-improving infrastructure,” which is quite different from recursive self-improvement where researchers try to make the underlying AI model itself more capable. How does your approach work, and why do you believe improving the systems around existing models may deliver meaningful gains much sooner?

The recursive self-improvement approach focuses on training proprietary models, assembling research teams, and raising massive capital. It’s a decade-long, expensive journey, because you’re asking the model to reason about its own training process.

Our approach is different. At INT21, we’re not trying to improve the frontier models themselves, which would take years and billions in capital. Rather, we’re building agent swarms that optimize the harness, the infrastructure layer that sits between the model and performance. That’s what my research at Nvidia proved: that the harness, not the model, determines agent performance on complex tasks. An agent doesn’t need to understand the frontier model capabilities to find a 10% efficiency gain. It just needs to explore the design space, measure, validate, and keep what works.

A way to think about the two different approaches is that one is building a power station while the other is building with electricity that already exists. We can move faster and address demand that’s real and immediate. And it means we’re already seeing significant, measurable gains in self-improving infrastructure today.

INT21 recently launched the Inference Engine Factory, extending the concept beyond individual GPU kernels to complete inference engines. What does it actually mean for an agent swarm to autonomously build and optimize an inference engine, and which parts of that process have traditionally required the most specialized human expertise?

An inference engine is the software that runs a model on hardware. It’s a complex system that makes dozens of design decisions at once, such as which operations run together in a single kernel, or how work schedules across cores. These decisions are interconnected, so if you change one, you have to retest everything else.

Traditionally, an engineer hand-tunes all of this for each model. For video, music, and speech generation, that’s a challenge because they’re multistage, multiscale architectures with no off-the-shelf framework. INT21’s agent swarms build the complete stack end to end, from individual kernels through to the full inference engine. Our video, and audio generation stacks require no human code review and outperform state-of-the-art inference solutions under the same concurrency settings.

Your PTX Kernel Factory has produced implementations that outperform existing baselines by as much as 59% on certain workloads. At a technical level, where are the agents finding performance improvements that highly optimized human-written implementations or conventional compilers are missing?

Our PTX Kernel Factory outperformed the best available baseline by up to 59% against KDA (Kimi Linear Attention). This boils down to our self-improving agent swarms being able to scale to thousands of variations in a fraction of the time, a process that’s too arduous and expensive for human engineers to do.

What’s different is that human-written kernels rely on domain-specific languages (DSLs), templates, and compilers that were prebuilt for GPUs. A DSL works well for common patterns, but when you hit a new workload, such as KDA, the general-purpose compiler often can’t perform optimally. Our agents bypass those abstractions entirely, which means that they’re not constrained by DSL or compiler assumptions. That freedom to explore the full design space without being locked into predefined patterns is where the improvement comes from.

AI-generated infrastructure creates an unusual verification problem: an optimization is not useful if it is faster but subtly incorrect. How do your agent swarms test, benchmark, reject failures, and retain successful discoveries, and how important is this feedback loop to making the system genuinely self-improving rather than simply an AI coding agent?

The feedback loop is critical, because an optimization is worthless if it’s only correct under specific assumptions or only for a specific input distribution. At INT21, we make sure the feedback loop is rigorous.

We start from the model, deployment constraints and the serving metric that matters. Then, the self-improving agent swarms explore configuration and optimization paths at the same time, and evaluate every candidate against correctness and performance targets. Only implementations that work are retained.

What makes this genuinely self-improving is that the validation data itself improves. That data feeds back into the agent swarms, so that they refine their understanding of what works, and continuously close the feedback loop. Without this rigor, you’d just have a system with plausible-looking, AI-generated code that can ship faster, but with technical debt that invisibly accumulates and erodes trust in what is being built.

At NVIDIA, you worked on projects including VibeTensor and Agentic Variation Operators, where agents generated substantial amounts of system software and autonomously searched for GPU optimizations. What did those projects reveal about the types of engineering problems AI agents are already capable of solving that many developers may still assume require human experts?

VibeTensor taught me something that I call the “Frankenstein Effect.” The AI did correct work at every level, but when you put all those pieces together as a system, it fell short of what human experts could build.

The Agentic Variation Operators work was the opposite. It showed that AI can outperform human experts in tightly scoped problems. Kernel generation is a good example, because the problem is narrow, measurable, and has a single objective.

The “Frankenstein Effect” is exactly what INT21 is solving. The engineering samples from our Inference Engine Factory outperform high-performance, open-source inference engines such as SGLang and vLLM, because we’ve learned to structure the problem correctly. We give agents clear constraints, measurable objectives, and tight feedback loops. Human experts are still essential, but instead of conducting optimization work, they give valuable direction and interpret the results. AI amplifies the expert’s impact by executing faster and more systematically than what any human can do.

INT21 uses multiple specialized agents rather than relying on a single agent with an increasingly large context window. Why do you see multi-agent orchestration as a more scalable approach to complex engineering problems, and how do the agents divide work, share discoveries, and avoid duplicating or conflicting with one another?

A bigger context window lets you hold more information at once, but it doesn’t help you solve multi-dimensional problems where everything is interconnected. We use a cloud-native platform for running specialized agents, all working toward the same measurable goal. The agents explore in parallel, and continuously converge on stronger solutions.

That’s more scalable for several reasons. Firstly, each agent stays focused. Secondly, you’re not asking a single agent to reason about everything at once, so the process is more efficient. And finally, it mirrors how infrastructure teams actually work, where each team owns their domain, but sync when decisions affect each other.

If agent swarms become capable of continuously optimizing kernels, inference engines, compilers, and other layers of the AI stack, how does the role of the infrastructure engineer change? Do you expect these systems primarily to address the shortage of specialized engineers, or eventually to automate a substantial portion of infrastructure development itself?

An infrastructure engineer’s job is set to change completely. They’re not grinding through kernel code and memory tuning anymore. Instead, they’re setting objectives, defining the design space, establishing constraints, and interpreting results.

Over time, fewer infrastructure engineers will be needed to keep systems running and improving. But the strategic work that remains becomes more valuable, not less. Practically, it means organizations can operate complex AI infrastructure with far fewer specialized engineers than they need today. That’s significant for every company trying to build production AI systems.

Your long-term vision is for self-improving agent swarms to operate across every layer of AI infrastructure. What does that stack look like if this vision succeeds, and could we eventually reach a point where AI infrastructure continuously rewrites and optimizes itself as models, workloads, and hardware change?

Right now, every layer of the AI stack optimizes independently, from the low-level kernels to the foundational frameworks, to the interfaces. But they’re not coordinating with each other. So you could optimize for one layer, but then accidentally break something else downstream.

At INT21, the vision is that agent swarms can coordinate autonomously across all of these layers, constantly adjusting and adapting. When a new model arrives, the entire stack reoptimizes for it, so that infrastructure never falls behind what’s running on it.

This hasn’t been feasible, because human engineers can’t move fast enough. But with agent swarms continuously optimizing, self-improving infrastructure is emerging as a distinct computing category. The infrastructure that can adapt as fast as models and hardware change will win. Everything else becomes a liability.

Thank you for the great interview, readers who wish to learn more should visit INT21.