Baidu’s ERNIE Evolve Automates AI Research and Inference Optimization

Baidu introduced ERNIE Evolve on November 9, 2026, reporting better results than public comparisons on three NanoChat and NanoGPT metrics and 33.7% higher throughput in one deterministic ERNIE-5.5-Flash inference test. The automated research system selects research directions, edits AI code, and retains only improvements that pass separate evaluation and review.

The announcement also covers active inference-engineering work for the in-development ERNIE-5.5-Flash. The system’s role is broader than writing code: it selects lines of inquiry, executes tests, and uses both failures and successes to guide later iterations.

A Research Loop With Independent Checks

Given a repository, an optimization objective, and evaluation criteria, ERNIE Evolve modifies code and runs experiments in isolated environments. An outer-loop Director chooses a direction and starting version using an Experience Bank of previous experiments and a Knowledge Base of papers, open-source code, and domain knowledge. Parallel lanes hold trees of candidate versions; within each, a Selector chooses a candidate and hypothesis, and an Executor implements the change.

Acceptance depends on a fixed Evaluator and a separate Reviewer. The Evaluator sits outside the repository, is invisible to agents, excludes out-of-scope changes, and requires gains above a noise threshold. The Reviewer checks adherence to the research direction and discards evaluation-loophole exploits. Off-direction candidates cannot become the best version.

Public Benchmark Results and Released Code

Baidu reports these outcomes against the public comparison results used in its evaluation:

On NanoChat, ERNIE Evolve added two prediction branches alongside the neural network: one uses n-gram statistics from the training corpus, and the other retrieves matching documents and predicts from their continuations. A learned gate adjusts the mixture using each branch’s performance at earlier token positions.

The results repository publishes final code and measurement logs, with executed code embedded in each log or identified by its SHA-256. It lists seven SOL-ExecBench kernel artifacts scoring 0.6669–0.9920; NVIDIA’s task definitions and workload data are not redistributed.

The SOL-ExecBench row qualifies the broader results: ERNIE Evolve did not lead every listed metric, although Baidu’s comparison places it ahead of Recursive and Hyra-1.0 on that task.

Inference Optimization for ERNIE-5.5-Flash

Baidu applied ERNIE Evolve to low-latency inference on Kunlunxin P800, deterministic inference throughput on GPUs, and single-request decoding speed. Most baselines had already been optimized by engineers, and changes had to pass the same load tests and correctness checks as engineer-submitted work.

On Kunlunxin P800, Baidu reports that four research rounds raised generation speed to 623.6% of baseline while passing correctness checks, with the model and deployment setup unchanged. Early rounds enabled fused manifold-constrained hyper-connections and speculative decoding, then reduced host-side scheduling and synchronization overhead. A fourth-round dispatch fix enabled DeepEP’s low-latency path.

For reinforcement-learning training, the deterministic task required generated tokens and log probabilities to be bitwise identical regardless of batch composition and to match full-sequence recomputation. SGLang’s deterministic mode supports those requirements by fixing reduction order, but that limits GPU utilization. After roughly three days, Baidu reports that the work also cut average time per output token by 26% and time to first token by 12%, with determinism preserved. The largest individual gain came from replacing deterministic sampling, which had consumed about 14% of GPU time per step, with a Triton kernel that processes vocabulary chunks in parallel.

For single-request GPU decoding, Baidu reports a 14.9% speed increase over an already engineer-tuned service, with unchanged accuracy. Three of seven research directions produced gains. They included using native FP16 instructions for FP8 matrix multiplications with fewer than eight input rows and replacing sequential expert scans with a two-level mixture-of-experts scheduler.

These tests also exposed tighter module and determinism constraints, measurement noise comparable to individual gains, and environments spanning inference engines, kernel libraries, hardware plugins, and service startup scripts. Some useful changes were smaller than the noise, so candidates had to prove their value in end-to-end load tests rather than isolated kernel measurements.

Baidu plans to apply the loop to training-system optimization, then use model-capability metrics to guide automated hypothesis generation, experiment design, and validation. Its longer-term aim is for AI to identify capability gaps and generate and refine evaluation tasks.

The company says recursive self-improvement must remain human-controlled and interruptible, within explicit boundaries for goals, permissions, and resources, with continuous safety evaluation and human oversight. Moving from fixed engineering metrics toward model capabilities and benchmark design makes those evaluation and oversight controls central to how far the research loop can expand.