AGIBOT Releases Physical Reasoning Dataset to Teach Robots What to Do Next

A robot can pick up a bottle perfectly and still fail at restocking a minibar. It might keep an empty bottle, add too many replacements, or repeat a step it has already completed. The missing skill is deciding what the situation requires before choosing the next movement.

AGIBOT is targeting that gap with the Physical Reasoning dataset, announced October 22 as Release 4 of AGIBOT WORLD 2026. The release connects task goals, changing scene conditions, execution history, and real robot actions. Its central idea is to collect different routes to the same goal, giving researchers material for training robots to adapt their decisions as a task unfolds.

One Goal, Several Correct Action Sequences

Release 4 changes the emphasis of task design from difficult movements to difficult decisions. Instead of collecting only demonstrations of a fixed sequence, AGIBOT varies object quantities, obstructions, available space, and task progress. Those changes can make a previously correct action inappropriate, even when the user’s request stays the same.

Consider the minibar example. Reaching a target of three full bottles requires identifying which bottles are empty, counting the full ones already present, and deciding what to remove and replenish. A successful grasp is only one part of the job. The robot also needs a representation of the desired final state and a way to compare it with the current one.

Another scenario involves retrieving a key behind a box. A direct grasp may work when there is enough room. When the box blocks access, moving it becomes a prerequisite. Searching drawers adds a memory problem: a camera image alone cannot tell the system which drawers it has already checked.

These examples expose a practical challenge for physical AI. A policy that imitates the visible motion of an expert can still make the wrong decision if it lacks the context that made that motion appropriate. By varying conditions around a shared goal, the dataset is intended to make those dependencies available for training.

Nine Reasoning Abilities, With Increasing Complexity

AGIBOT organizes the release around nine capabilities: selecting objects by attributes, choosing according to context, assessing whether a manipulation is feasible, reasoning about action prerequisites, searching toward a goal, tracking states and quantities, planning over longer sequences, maintaining memory, and ordering subtasks.

These abilities overlap in ordinary work. Restocking requires recognition, counting, and progress tracking. Finding an item combines a search strategy with a record of previous attempts. A longer housekeeping task can require several of these judgments in succession, with each action changing the conditions for the next.

The dataset separates tasks into Easy, Medium, and Hard levels. Individual selection judgments sit at the simpler end; quantity constraints and prerequisites add complexity; longer tasks combine multiple goals, memory, and contextual requirements. That structure gives researchers a way to study individual skills before testing how well they work together.

Execution outcomes are annotated too, including object choice, placement, and whether the required quantity was achieved. The release includes both successful and erroneous examples, supporting research into recognizing deviations and adjusting subsequent decisions.

Connecting Instructions to Physical Actions

The technical contribution lies in how the annotations connect different levels of a task. A high-level instruction states the overall objective. Task frames, subtask segments, and action steps describe its structure and progress. Researchers can use those relationships to study which subgoal is active and what should happen next.

At the movement level, AGIBOT’s Dense Prompt annotations specify the target object, manipulation region, roles of the two arms, motion, gripper state, and expected result. Bounding boxes, points, and trajectory lines connect language to locations and movement paths. This gives a learning system more explicit guidance about what an instruction refers to and how its execution is represented.

The company also describes success and adversarial segments for distinguishing targets from distractors and checking results. Here, adversarial examples concern task interpretation and execution. They help frame questions such as whether the robot chose the right object or actually finished the requested step.

Together, these annotations are intended to support training that connects a decision-making model with an action policy. The decision model selects a subgoal, the policy carries out the manipulation, and the resulting state informs the next decision. That feedback matters whenever a robot must recover from an imperfect attempt or respond to a changed scene.

How the Release Fits AGIBOT’s Robotics Platform

The broader AGIBOT WORLD 2026 repository describes real-world data collected on the company’s G2 platform. Its documented format follows LeRobot conventions, with episode data, metadata, and separate head and wrist camera video streams. That provides useful context for developers connecting visual observations with recorded robot behavior.

AGIBOT’s G2 product page positions the hardware for embodied intelligence and force-controlled manipulation, alongside interactive capabilities. Release 4 addresses the decision and task-memory layer around those physical capabilities. It also extends the data infrastructure discussed in Unite.AI’s earlier coverage of AGIBOT’s robotics platform.

The repository lists a Creative Commons Attribution–NonCommercial–ShareAlike 4.0 license. The noncommercial condition is a meaningful distinction for developers considering how to use the released data in a product.

What Physical Reasoning Could Change

For hospitality, household services, and retail operations, the attraction is a robot that can work toward an outcome across changing conditions. Restoring a table or replenishing a shelf requires checking what is already there, remembering completed work, and recognizing when further action is unnecessary. Those judgments become more important as robots move from isolated demonstrations into complete workflows.

Release 4 supplies training material for that direction. It does not, by itself, establish that a particular model can reliably perform those workflows in unfamiliar environments. AGIBOT says the data has already supported its upcoming agent framework and next-generation foundation model; performance results for those systems remain a separate question.

The useful shift is toward training robots on the relationship between a goal, the current situation, and the consequences of previous actions. Researchers can explore that approach through the AGIBOT WORLD project and its linked dataset resources.