Supermicro Launches Shared KV Cache Storage for Long-Context AI

Supermicro announced a KV cache storage solution with IBM and NVIDIA on November 5, 2026, designed to move reusable AI inference context into shared storage beyond GPU memory. The companies report up to 56 times faster time to first token on a 130,000-token prompt in joint testing.

The offering combines Supermicro Petascale all-flash storage servers using AMD EPYC 9005 processors, IBM Storage Scale software, NVIDIA’s Dynamo inference framework and Spectrum-X Ethernet networking. It is aimed at multi-turn and agentic applications, coding assistants, multimodal AI and long-context retrieval-augmented generation. Large codebases, documents, images and conversation histories can create more key-value state than GPU and CPU memory can retain.

Reusing Context Across Requests

Prefill turns a tokenized prompt into the intermediate key-value states used by the model. Persisting those states allows later requests to reuse context instead of recomputing it. The architecture provides a highly available storage tier with shared access across GPU nodes and clusters, plus non-disruptive expansion of cache capacity.

Dynamo adds cache-aware routing, separates prefill from decode, and offloads cached blocks through its KV Block Manager and the NIXL transfer library. IBM Storage Scale Erasure Coding Edition supplies the shared G4 storage tier in NVIDIA’s memory hierarchy.

Benchmark Results and Test Conditions

An IBM Redbooks reference architecture, published June 5, 2026, and last updated August 20, 2026, reports 32.14 seconds for GPU recomputation at 130,000 tokens, versus 0.570 seconds using the storage-backed cache. Those existing measurements underpin the launch rather than constitute a newly published November study.

A separate 200-request run used 28 concurrent connections and drew from 100 unique prompts representing 24 million tokens and 825 GB of KV cache data. The Redbook reports throughput rising from 0.19 to 4.26 requests per second and total processing time falling from 1,048.56 to 46.94 seconds. The cache began empty, so the full-run average includes a cold-start period as reuse built up.

Under a noisy-neighbor test, four other clients generated 200 GB/s of competing network I/O. The cached system sustained 3.6 requests per second and finished 200 requests in 55.56 seconds, which the Redbook describes as an 18% throughput reduction from its clean-network cached result.

All inference measurements came from one Supermicro GPU client node with four NVIDIA RTX PRO 6000 GPUs, connected to an eight-node Petascale storage cluster. The software stack used Dynamo v0.9.0+, vLLM v0.14.1, NVIDIA AIPerf and the openai/gpt-oss-120b model. GPU-memory and system-DRAM cache tiers were disabled to isolate the shared G4 tier.

Network Headroom and Deployment Sizing

Three Spectrum-X SN5600 switches formed the test’s spine-leaf network. The Redbook reports client-facing bandwidth of up to 80 GB/s, while the eight-node storage system achieved up to 315 GB/s of read throughput, making client-side networking rather than the storage tier’s full read capacity the configuration’s constraint.

The Redbook’s sizing examples use four storage nodes for small proof-of-concept deployments, eight for the tested medium configuration and 16 or more for large AI factories. Capacity, bandwidth, concurrency, model size and workload all affect the appropriate design. Supermicro says its factory rack-integration and on-site deployment services can reduce deployment time from weeks to days.

The launch packages a previously documented, jointly validated architecture for deployment, but its gains belong to a specific model, prompt mix, concurrency level and network. The central deployment question is whether storage capacity and client bandwidth are balanced to actual context-reuse demand: additional storage read headroom cannot improve inference if the client network cannot deliver it.