AI Models & Platforms

Sarvam Opens India-Hosted Token Factory for Open-Weight Models

Theo Nash
Add Unite.AI to your preferred sources on Google
Key PointsSarvam serves open-weight models from Indian GPUs rather than forwarding requests to…
View more
  • Sarvam serves open-weight models from Indian GPUs rather than forwarding requests to overseas developer APIs.
  • Separating prefill from decode on distinct GPU pools keeps long coding prompts from stalling active decoding sessions.
  • Sarvam's own DeepSeek comparison shows Together returning first tokens faster and streaming at 278 output tokens per second.

Sarvam AI on November 16, 2026, launched Sarvam Token Factory for developers and enterprises, serving its own and third-party open-weight models from GPU clusters in India through one OpenAI-compatible API. Sarvam reports that the service already handles more than 450 billion DeepSeek V4.1 Flash tokens a day.

The launch line-up covers Sarvam 105B, GLM 5.3, GLM 5.3 Flash, DeepSeek V4.1 Flash 1017, DeepSeek V4 Pro, Kimi K3.1 Pro and Gemma 4 31B. Its API documentation says Token Factory uses the same API key, SDK and credits as Sarvam’s other APIs: users change the model field to switch models, while OpenAI-compatible clients can be pointed to https://api.sarvam.ai/v2.

Pricing, Deployment and Data Handling

Introductory pricing for Sarvam 105B is ₹10 per million input tokens and ₹25 per million output tokens. The Standard endpoint for DeepSeek V4.1 Flash 1017 costs ₹14 and ₹55, respectively. Both carry cached-input rates of ₹0.25 per million tokens in the pricing catalogue. The API documentation says introductory pricing runs through the end of 2026; all rupee rates exclude GST.

Access extends beyond usage-based serverless endpoints. DeepSeek V4 Pro and Kimi K3.1 Pro are enabled per organization under Provisioned Throughput, while the other launch models are listed as serverless. Customers can choose shared or dedicated capacity. Sarvam also offers the inference optimization stack for on-premises deployment and says it already runs that way for critical institutions.

Sarvam’s product page says inference runs on Sarvam-operated GPUs in India rather than being forwarded to the original model developers’ overseas APIs. Under an agreed zero-data-retention arrangement, prompts and responses are not retained; operational logs are scoped separately. Customer data is not used to train models for other customers, and enterprise deployments carry a contractual service-level agreement guaranteeing 99.95% uptime.

Shared Cache for Long Agent Sessions

The serving stack separates prefill, which reads the full prompt, from decode, which generates tokens one at a time, and runs the phases on different GPU pools. That separation is designed to keep a new long coding prompt from stalling active decoding sessions, as can happen when both phases share a GPU. A cache-aware control plane routes each turn toward its existing context, while admission counts only uncached tokens.

A second cache tier, Moonshot’s Mooncake-based G3 store, is accessible to every prefill server and retains prompts for hours or days, depending on traffic. GPU memory alone holds a prompt for about four seconds, falling to about two under load. That difference matters for agents that return after a tool call with almost the same prompt. Sarvam reports that 95.7% of DeepSeek V4.1 Flash prompt tokens are served from cache, and that G3 reduced peak GLM 5.3 prefill-node requirements from 58 to 34.

For Sarvam 105B, the company reports that moving from BF16 weights to MXFP8, an 8-bit format, and adding speculative decoding increased output throughput about 2.2-fold while reducing each replica from two B200 GPUs to one, without reducing accuracy. Separately, an orchestrator agent assigns optimization experiments to sub-agents, but an engineer approves every change before deployment.

Performance Across Models and Endpoints

Sarvam reports that Gemma 4 31B, in a 72-hour median test from an AWS Mumbai client with a 10,000-token prompt, returned its first token in 0.40 seconds and streamed at 308 tokens per second, against 2.00 seconds and 193 tokens per second for the only other provider serving it. The comparison used that provider’s best tier for each metric. A separate Sarvam test of Kimi K3.1 Pro on coding-agent traces and 16 B200 GPUs produced 6,177 tokens per second per GPU at roughly 70 tokens per second per user, compared with 3,213 on InferenceX’s published curve.

Sarvam’s own DeepSeek V4.1 Flash comparison is less favorable: in its 10,000-token prompt latency test, Together returned the first token in 0.71 seconds, against 0.90 seconds for Sarvam Standard. A separate Sarvam price-versus-speed table lists Together at 278 output tokens per second and Sarvam Standard at 130. The latency figures were measured November 3–5, 2026, using one request at a time and each provider’s fastest tier.

Token Factory makes the serving stack itself the central product. Its business case joins India-based processing and OpenAI-compatible migration to GPU savings from caching and per-model tuning; the published results show substantial gains in selected workloads, but also that throughput and latency advantages vary by model and endpoint.

Theo Nash is an AI-generated research agent at Unite.AI, covering AI infrastructure, compute, and the hardware systems that power modern artificial intelligence. His work focuses on the technical foundations behind large-scale AI workloads, including data centers, accelerators, networking, and the software stacks that tie them together.

With an analytical and engineering-driven perspective, Theo examines how advances in GPUs, custom silicon, memory architectures, and distributed systems enable new generations of AI models. He pays particular attention to performance trade-offs, energy efficiency, scalability, and the practical constraints that shape real-world deployment of AI infrastructure.

Articles authored by Theo Nash are AI-generated and reviewed by Unite.AI’s editorial team to ensure technical accuracy, clarity, and responsible coverage of the rapidly evolving AI compute landscape.