Interviews

Rahul Nambiar, Co-Founder and CEO of Propensity Labs – Interview Series

Antoine Tardif
Add Unite.AI to your preferred sources on Google
Rahul Nambiar smiling in a green shirt against an outdoor urban background.
Rahul Nambiar, Co-Founder and CEO of Propensity Labs.
Key PointsAdapting prompts to a cheaper model let it match a premium model at 7% of the cost.
View more
  • Adapting prompts to a cheaper model let it match a premium model at 7% of the cost.
  • Cheaper models often fail from fixable behavioral patterns rather than insufficient raw capability.
  • Teams should measure cost per successfully completed task rather than cost per token alone.

Rahul Nambiar is Co-Founder and CEO of Propensity Labs. He previously built large-scale infrastructure at AWS before moving to Meta, where he led work on an exabyte-scale user data inventory using machine learning classification. His work now focuses on understanding how AI models behave and combining model routing with instruction adaptation to improve the cost and reliability of agentic workflows.

Propensity Labs develops systems that help teams choose and direct AI models for their specific workloads. Its approach combines evaluations of model behavior, automated prompt adaptation, and routing between models, with the aim of helping users get more from available models without having to become experts in each one.

Your career has taken you from Meta, where you led privacy tech on an exabyte-scale data inventory project, to AWS and now as CEO of Propensity Labs. How has that technical background shaped your perspective on the limitations of current AI models and the potential of specialized instruction routing?

I actually started at AWS and then moved to Meta. One thing I’ve learned from building large scale systems is that you can often get massive gains from understanding and optimizing how the underlying components behave rather than treating them as black boxes that are interchangeable. The same applies to AI models. We’ve found that even very capable models have behavioral quirks that can be understood and worked around, often with surprisingly large improvements in performance and cost.

In your recent experiment, you demonstrated that routed, smaller models outperformed premium ones at a fraction of the cost. What are the primary engineering hurdles to implementing this kind of model routing in production environments versus a benchmark test?

By adapting prompts to Luna and its weaknesses, we were able to get it to match Sol’s performance at 7% of the cost in the complex Tau3 agentic benchmark from Sierra. Routing between Luna with adaptation and Sol enabled us to match Astra’s performance at 16% of tis cost.

A big learning for us was that both routing and prompt adaptation need to be very specific to a workload. Benchmarks come with neat environments and graders that you can use to study how a model performs at any given task and evaluate how well it did. This informs how you need to adapt prompts to the model and when to route to which model.

Production workflows don’t come so neatly packaged. You have traces of models completing tasks, but no reliable way to tell whether they actually succeeded and what good looks like. What we’ve learned to do is take existing usage data, either evals or production traces, and reconstruct an evaluation setup that lets us test different models and adaptations on the same workload.

The interesting part is that routing and adaptation need to be considered together. A cheaper model might fail at a task with your existing prompts, but succeed once you adapt them to how that model behaves. That changes which model you should route to in the first place.

You mention adapting instructions to specific models. How do you distinguish which instructions are universal versus those that must be uniquely tailored to the strengths and weaknesses of a specific model architecture?

Many instructions that seem universal need to be tailored to the model. Here’s a simple example – hallucination in single turn responses. We studied two different models here – DeepSeek V4 Pro and GPT-6 Luna. Both have a strong tendency to make up things when they don’t know an answer. Some artifact of their training process makes them very hesitant to abstain and say “I don’t know”. This behaviour is similar, but how we mitigate it through prompts is quite different.

DeepSeek V4 Pro can be induced to abstain to the levels of Claude models if you layer it with many variations of “It’s okay to say I don’t know”. The same tactic has little effect on Luna. Luna responds best to being given a cost for being wrong. Something along the lines of “A wrong answer counts as -90 points, while a right answer only gives +10” makes it abstain to the same extent as its bigger brother, Sol. Same behaviour, but very different levers to push two models away from it. To adapt instructions to a specific model, you need to deeply understand both the model and the workflow its deployed in.

As companies move toward agentic workflows, what evidence do you see that routing between cheaper, specialized models is a more durable strategy than waiting for a single, increasingly capable frontier model to handle everything?

A new frontier model comes out every week or so. Say you pick the biggest and best model right now for your entire workflow. In a couple of months, or even weeks, there will likely be smaller models out there that can do large parts of, if not all of, your workflow. At the same time, there will likely also be larger or more specialized models out there that are more performant in your workflow. By not switching or routing across models, you’re leaving both performance and cost on the table.

But keeping up with all this is a hassle and people shouldn’t have to become experts in all the different models, which is where we believe we can help by ensuring you always have the best model for your workload at the lowest cost for the performance you want. Sometimes, that’s just picking the most capable model out there for parts of your workload and at other times, it’s adapting most of your workload so that it can be handled by much cheaper models.

When managing a system that routes tasks across multiple models, how do you handle data consistency and privacy, especially when those tasks touch sensitive information?

We function as a layer between users and model providers. Users select which providers they want to use and bring their own API keys. We pick the best model for their workloads, or part of their workloads, from these providers, adapt their workload, and send it over to the right models. We only use the evals or traces a customer chooses to share, and we don’t train on or keep their data.

You co-founded Propensity Labs with Mana Azarm. How do you divide the focus between the research-heavy side of model behavior and the practical, systems-engineering side of building these agentic pipelines?

Our skills complement each other well here. Mana, who is our Chief Scientist and also a Professor, focuses more on the part around model behaviour and I focus on the system engineering side of things. Or rather, this is how we want it to be. Reality is messier and we each end up working on whatever is the main blocker at the moment.

For CTOs currently spending heavily on premium model API calls, what is the ‘low-hanging fruit’ for testing model routing and instruction adaptation in their existing workflows?

Come talk to us! But seriously, I would just try running a smaller model, from the latest generation, on your specific workload and see what it’s able to handle. I suspect a lot of folks are stuck on larger models from previous generations and at the rate at which models are improving, just haven’t realized that there are smaller and cheaper alternatives that came out recently that can handle their workloads.

What are the most common misconceptions developers have about what cheaper models can achieve when they are properly constrained or guided?

The most common one is assuming that when a cheaper model fails, it just isn’t capable enough. I think it’s beneficial to analyze in depth how and where a cheaper model fails in your workflows. A chunk of those failures are patterns the model tends to slip into that if recognized and nudged away from, might enable it to succeed in your workflow. Another chunk are tasks that a cheaper model just lacks the raw capabilities to perform correctly, which is where you need a bigger model.

Your work touches on the intersection of privacy, scale, and cost. When looking at the next 12-18 months of AI infrastructure development, where do you see the biggest disconnect between what businesses want and what the current ecosystem of tools provides?

From talking to practitioners deploying models in production, a common feeling I’m seeing is FOMO and to some degree, exhaustion from keeping up with new models and increasing capabilities. Most technologies tend to get easier to use and deploy with time. LLMs have gotten harder, with a plethora of models and settings (temperature, reasoning effort, caching) to select from. I think there’s a lot of value in simplifying and abstracting them away for teams using models, so that they can just focus on what they want to get done from models and not on how to get it done.

What metrics should a technical team prioritize when they start experimenting with multi-model routing to ensure they aren’t just chasing cost savings at the expense of reliability?

I’d start with task success rates, failure rates, latency, and cost per successfully completed task. Cost per token alone isn’t particularly useful if the cheaper model fails more often, so you want to count the entire cost of a task – including retries and any classifiers or decisions models you call to do routing.

The hard part is measuring reliability. We’ve seen a common pattern where teams struggle with migrating their agents from one model to another, go all in on evals for a few months, and then find the process of switching models much easier. I’d strongly advise investing in solid evals for your workflows as early as possible and keeping them up to date as your workflows and how your users use them change.

Thank you for the great interview, readers who wish to explore model routing, instruction adaptation, and evaluation methods for more reliable and cost-effective AI workflows should visit Propensity Labs.

Antoine is a visionary leader and founding partner of Unite.AI, driven by an unwavering passion for shaping and promoting the future of AI and robotics. A serial entrepreneur, he believes that AI will be as disruptive to society as electricity, and is often caught raving about the potential of disruptive technologies and AGI.

As a futurist, he is dedicated to exploring how these innovations will shape our world. In addition, he is the founder of Securities.io, a platform focused on investing in cutting-edge technologies that are redefining the future and reshaping entire sectors.