Salesforce’s SFR-AutoR&D Automates Model Research and Validation

Salesforce introduced SFR-AutoR&D on November 11, 2026, saying its agents produced SFR-Embedding v4 in about 80 hours rather than the several weeks that human experimentation often requires. The system is designed to run the research cycle autonomously, both by changing existing codebases and by developing and testing new training methods for AI models.

The work extends beyond a standalone research demonstration. SFR-AutoR&D had already improved SFR-Embedding V3, which Salesforce uses for retrieval in Data Cloud, before researchers pointed it at the open-source EmbeddingGemma 2 baseline to create v4. Embedding models convert text into numerical representations so systems can retrieve records, documents or case histories. In Salesforce’s architecture, that retrieval supplies customer information on which Agentforce answers are grounded, making retrieval quality consequential for the agent’s subsequent response.

Two Agents, One Controlled Research Loop

The SFR-AutoR&D project divides the work into four stages: discover a testable hypothesis, build a candidate and baseline comparison, evaluate correctness or model quality, and learn from both successful and failed experiments. A candidate that cannot demonstrate a verified gain leaves the baseline unchanged.

AutoR&D-Engineer accepts a repository, target and constraints, then produces a candidate patch accompanied by benchmark evidence. TrainForge starts with a baseline model, task, evaluation and compute budget, producing the best verified checkpoint and its evaluation record.

TrainForge’s method-discovery process generates training strategies instead of choosing only from a fixed recipe list. Codex writes implementations from an empty workspace, and checks verify that the running code matches the proposed idea. Promising candidates advance from a fast LoRA test, which updates a compact set of trainable parameters while retaining the base model, to a short full-parameter run and then full training. Frozen held-out datasets are reserved for evaluation.

Both agents run on AutoInfra, Salesforce’s shared training and inference infrastructure. It assigns GPU workers to parallel experiments and keeps requests, progress and results persistent so work can resume after gateway interruptions. AutoInfra also separates the algorithm, training-service and sandbox environments to protect system integrity and guard against reward hacking.

“The scientific method still governs, and the loop runs faster,” said Silvio Savarese, Salesforce’s executive vice president and chief scientist.

Embedding Gains and Codebase Results

On the public embedding evaluation, Salesforce reports that the research checkpoint scored 57.21 against EmbeddingGemma 2’s 51.07, a reported 6.13-point gain on a 0–100 scale. The production-supported checkpoint scored 55.76. The scores average nDCG@10 across 10 retrieval tasks within the same 41-task MTEB suite; the metric assesses relevance among the top 10 results, giving greater weight to higher-ranked items.

All three models were evaluated with an 8,192-token context limit. Salesforce explicitly labels the measurements as local checkpoint comparisons, not official MTEB leaderboard ranks. They therefore document improvement over the selected baseline under a controlled protocol without establishing a position against the wider field.

A separate autonomous session lasting roughly nine hours targeted serving efficiency. Salesforce reports 36% higher throughput and 17% lower latency with almost no quality loss, connecting the research result to the cost and responsiveness of retrieval across large customer datasets.

The engineering results also report up to 1.84× faster pandas reductions on nullable data types, which explicitly represent missing values. They record a 4.1% median increase in hnswlib queries per second across three index builds and 17.5% higher single-core USearch throughput with GCC 11.4. The latter two libraries perform vector search, a central operation in AI retrieval.

The same engineering record documents 134,064 differential cases for the nullable-dtype pandas patch, with zero differences in values, masks, data types or exceptions. Inputs outside its fast path fall back to pandas’ existing implementation, and all three pandas optimizations have been submitted as upstream pull requests. The hnswlib patch reuses scratch memory per worker while preserving byte-identical search results. The USearch patch caches stored vector norms, adding four bytes per vector while preserving recall in the reported evaluation.

Limits on Autonomy and Deployment

Researchers still set the goals, budgets, evaluations and data constraints. Models intended for product use draw only on training datasets that have cleared ethical and legal review, and an improved model must also satisfy production trust and safety requirements. Salesforce said that production work is underway, so SFR-Embedding v4’s research results do not amount to a completed customer rollout.

SFR-AutoR&D combines autonomous experimentation with human-set constraints and evidence requirements. Its immediate significance is the compression of that controlled research cycle from weeks to days; the next substantive test is whether the v4 gains satisfy production review and reach the retrieval systems on which Salesforce’s enterprise agents depend.