Valory Unveils Open-Weight Olas Predict Model With Gains in Real-World Forecasting

Valory has unveiled Olas-Predict-Qwen3.8-27B, an open-weight forecasting model that the company says comes within 1.2 percentage points of its GPT-5.6 comparator on a test of real-world prediction markets. The 27-billion-parameter specialist reached 67.3% balanced accuracy in Valory’s internal evaluation, compared with 68.5% for GPT-5.6, when both systems received the same market evidence.

The result points to a practical question in AI development: how much of a specialist task’s performance comes from a model’s broad knowledge, and how much can come from relevant evidence and focused training? Valory’s latest experiment suggests that a smaller, adaptable model can be competitive on a narrowly defined forecasting task, while also showing that a stronger foundation model accounts for much of the improvement.

What Valory’s Forecasting Comparison Measures

The announcement reports results across 6,824 researchable prediction markets. The systems used common prompts and evidence at temperature zero, and Valory applied a date filter tied to the GPT-5.6 comparator’s stated February 16, 2026 knowledge cutoff. The comparison evaluates how models reason over supplied evidence rather than asking each system to gather its own information.

That scope matters. These are company-reported results on selected markets, rather than an independent demonstration of general equivalence between the models. The final comparison is also separate from an earlier 10,916-sample experiment and from Valory’s previously published 14-billion-parameter study.

Balanced accuracy gives equal weight to performance on the two outcome classes, helping prevent a common outcome from dominating the result. Valory also reports Brier score, which evaluates probability forecasts and penalizes confident errors. Lower Brier scores are better; a forecaster should be rewarded for well-calibrated uncertainty, not just for selecting the eventual winner.

The table reproduces Valory’s reported final comparison subset. Its historical GPT-4.1 baseline should be read as that specific baseline, rather than as a new universal score for GPT-4.1.

A Stronger Foundation Provides Most of the Gain

The new model improves on Olas-Predict-R1-14B by approximately 22% on Brier error and 8.4 percentage points on balanced accuracy in this comparison. But the untrained Qwen3.8-27B starting point already reaches 66.4% balanced accuracy and a 0.2235 Brier score, accounting for much of the step forward.

Domain training supplies an additional improvement: Brier error falls by approximately 5%, while balanced accuracy rises by 0.9 percentage points. Expected calibration error, which summarizes the alignment between stated confidence and observed outcomes, falls from 0.1630 to 0.1429—an improvement of approximately 12.3%.

This is a more useful finding than treating the entire generational gain as the effect of fine-tuning. A stronger base model and specialized training contribute separately. The latter appears to improve how the model expresses probabilities, an important capability when a downstream system must decide whether the evidence is strong enough to act.

The Forecasting Tool Supplies Evidence; the Model Estimates Probability

Valory describes a training process that uses reviewed forecasting examples and reinforcement learning to improve probability estimates. A separate tool gathers market intelligence and evidence, then supplies that material to the model.

This separation is central to the architecture. The language model is one component of a forecasting system: evidence retrieval determines which information is available, while the model interprets it and produces a probability. Strong reasoning over weak or outdated evidence can still produce a poor forecast.

The approach also creates opportunities to improve the components independently. Developers can refine retrieval, change how evidence is presented, or update the forecaster without necessarily rebuilding the entire agent. The reported scores evaluate a particular combination of evidence and model behavior, so they should not be transferred automatically to a different retrieval pipeline.

How Olas Turns Agent Activity Into Training Data

Valory is a core contributor to Olas, whose Predict ecosystem connects agents that create markets, trade, provide forecasts, and finalize outcomes. Traders can purchase forecasting services from specialist agents through the Mech Marketplace. Once events resolve, forecasts can be compared with outcomes, creating records for evaluation and future training.

That feedback loop gives the project a source of task-specific evidence. Instead of judging a forecast solely by whether its explanation sounds convincing, the system can eventually compare its probability estimate with a resolved event. Market resolution still defines the target, so question wording and resolution quality remain consequential.

Valory’s earlier published work used more than 200,000 agent prediction records to specialize Olas-Predict-R1-14B. A separate later-market evaluation reported 75.8% accuracy across 2,628 markets. That figure is ordinary accuracy from a different evaluation; it is not directly comparable with the new announcement’s 67.3% balanced accuracy.

What Open Weights Change for Deployment

An open-weight forecaster gives developers the option to host and adapt the model rather than depend entirely on a remote proprietary service. For an organization building a forecasting workflow, that can create more control over model updates, serving infrastructure, and the handling of supplied evidence.

A 27-billion-parameter model still requires suitable hardware, with memory, latency, and throughput depending on precision, context length, and concurrency. The announcement does not provide measured deployment costs or serving benchmarks for the new model, so its parameter count alone does not establish a particular operating-cost advantage.

Forecasting quality also differs from trading performance. Returns depend on market prices, transaction costs, execution, and strategy as well as predictions—a distinction reflected in the limitations of Valory’s earlier model card.

The significance of Olas-Predict-Qwen3.8-27B is therefore focused but meaningful: Valory reports a specialist model approaching its frontier comparator on supplied-evidence forecasting, with measurable gains from domain training. Its next opportunity is to turn the continuing stream of resolved predictions into improvements that hold up across later events and real deployments.