Cohere Launches Confidential Computing on Model Vault in Public Beta

Cohere on October 22, 2026, announced the public beta of Confidential Computing on Cohere Model Vault, its dedicated, fully managed model inference platform, running on NVIDIA H100 GPUs on Microsoft Azure for enterprises processing sensitive data.

The launch extends Model Vault to workloads involving clinical records, financial information, privileged documents, intellectual property, and proprietary source code. Cohere said the goal is to let enterprises use the capabilities and operational simplicity of managed AI while keeping their sensitive plaintext data out of reach of the infrastructure running the service. The company names candidate workloads including financial-services fraud investigations, privileged document search and due diligence for legal teams, sensitive public-sector information, patent and research material, proprietary engineering and interface design documents, and search and analysis across proprietary source code.

Under the beta’s division of roles, Microsoft Azure provides the underlying confidential computing infrastructure, NVIDIA Confidential Computing carries that protection onto the H100 GPUs, and Cohere supplies managed model serving, an encrypted request path, remote attestation, and customer-side verification through Model Vault. Customers can set up and manage encrypted vaults through Model Vault, connect them through the same Cohere APIs, and begin at vault.cohere.com, according to the company.

Confidential Execution and the Encrypted Request Path

Confidential computing targets data in use, meaning information while a model is actively processing it, a third state beyond the stored and in-transit data that encryption traditionally covers. In Model Vault, the model-serving stack, including the operating system, drivers, and inference server, runs inside a confidential virtual machine, where hardware-backed isolation shields its memory from the host environment and policy limits the environment to approved software while giving operators no interactive path into the running workload, according to a companion technical deep-dive Cohere published alongside the launch. NVIDIA Confidential Computing protects GPU memory and encrypts traffic between the CPU and GPU, so the confidential boundary covers the accelerator performing the inference instead of ending at the CPU coordinating it.

The request itself receives separate protection. Model Vault uses Oblivious HTTP, or OHTTP, to encrypt the sensitive contents of a request to a public key tied to the attested confidential environment. The encrypted message crosses intermediary infrastructure as ciphertext, only the OHTTP gateway inside the environment holds the matching private key, and responses receive the same protection on their way back. Cohere says this splits the job of operating the platform from the ability to read what customers send through it, because intermediaries can still route traffic without seeing plaintext payloads. Applications keep the Model Vault API and familiar request formats, with the protection implemented in the client and infrastructure layers rather than requiring redesigned payloads.

Remote Attestation as an Active Control

Before a protected request is sent, the confidential environment generates cryptographically signed evidence that captures measurements and security-relevant properties of its hardware, firmware, operating environment, and application software, and customer-side software checks that evidence against an approved policy. The evidence records what environment is actually running; the policy encodes whether that environment should be trusted with the data. When a required property does not match, for instance a changed software measurement or a GPU that is not in its expected confidential mode, the client fails verification, refuses to transmit the payload, and the request stays within the customer’s own environment. That evidence is refreshed on a frequent cycle, and the client re-checks the new measurements and security state against the approved policy each time before continuing to send protected requests, which Cohere describes as converting attestation from a record kept for audits into a control that actively decides whether protected data is released.

In Cohere’s launch announcement, CTO Phil Blunsom said: “With hardware attestation, customers not only have cryptographic proof that workloads are running in a secure, confidential computing environment, they also can verify that their data is protected from unauthorized access, even from Cohere.” Vikas Bhatia, head of product for Azure Confidential Computing at Microsoft, said the work with Cohere demonstrates how confidential computing can combine the simplicity of managed AI with independently verifiable technical controls. John Fanelli, vice president of enterprise software at NVIDIA, said NVIDIA Confidential Computing on its GPUs provides hardware-backed protections that help platforms like Model Vault bring confidential AI inference to enterprise workloads.

Threat Model, Stated Limits, and Open-Source Plans

Within its stated threat model, the system shields prompts, documents, responses, and model execution during processing, and the cloud provider, host operating system, hypervisor, infrastructure administrators, and Cohere’s own operators are blocked from reading protected workload memory. The trusted computing base still includes the hardware and firmware providing the confidential computing foundation, the measured software running inside the environment, the reference measurements that define an approved configuration, and the attestation infrastructure that evaluates and signs evidence. Cohere says the design shrinks what must be trusted into an explicit set of components that produce measurable, checkable evidence.

Cohere also spells out what the boundary excludes. Physical and side-channel attacks, such as inferring information from timing or other shared hardware behavior, remain outside it, and an operator who cannot read a workload can still stop, delay, or prevent it, so the design carries no availability guarantee on its own. Application-level risks such as prompt injection, insecure tool use, authorization errors, and other software vulnerabilities are not solved by placing the software inside confidential computing hardware.

Observable metadata is another exclusion: while payload contents are encrypted, message size, request timing, traffic volume, and routing details remain visible, and in the current architecture the API credential and model identifier also sit outside the OHTTP-protected payload. The company states confidential computing is not a replacement for application security, not a data-residency mechanism, not a substitute for compliance or contractual controls, and not homomorphic encryption, because the model still operates on plaintext inside the protected environment.

The implementation builds on the open-source Kata Containers and Confidential Containers projects, and Cohere said it plans to release the confidential inference stack as open source, allowing customers and security researchers to examine the code, see which policies are enforced, and reproduce verification measurements independently. The beta currently operates on NVIDIA H100 GPUs in Microsoft Azure, and Cohere said support for NVIDIA Blackwell infrastructure, including B200 GPUs, is in progress on the path to general availability.