Humanoid Robots Are Coming to Factories. Who Is Responsible When They Fail?

A robot may successfully complete a performance test and yet become a safety hazard if it faces an unexpected obstacle, a change in how it behaves due to a software update, or a real worker. The question is not whether humanoid robots can perform a task anymore, but how they can do it. Whether they can do it safely or not—and who is responsible if something goes wrong.

Agility Robotics and FORT Robotics announced a partnership in October to enhance the safety architecture of Agility’s Digit 5 humanoid robot. They have planned their approach by incorporating onboard safety systems and external interfaces, communications, and manual override. The development marks a significant shift: Robots have to be safe in human workplaces, and that means more than just the robot itself.

However, technology is not enough to answer the question of accountability. When a robot makes a mistake based on a faulty component misidentified by an AI system, the manufacturer, the software developer, the system integrator and the employer who is using the system could be held liable.

When AI Decisions Become Physical Risks

Traditional industrial robots are typically found in enclosed work cells, where they are isolated from workers by barriers and safety devices. Humanoid robots pose another challenge: they are supposed to navigate through environments constructed for humans, perform different tasks, and adapt to different environments.

That flexibility creates new risks. If a false detection occurs, it may lead to an unnecessary movement. A software update could alter a robot’s behavior. Changes in the factory layout may create hazards that were not present in previous tests.

Dana R. Wolfe, Vice President and Head of Industry, GFT Americas, told Unite AI to prevent unsafe interventions, multiple layers of protection are necessary, such as deterministic software guardrails, factory-specific confidence thresholds, sensor fusion and staged validation prior to deployment.

There is no way to completely avoid false-positive results, Wolfe said. Rather, manufacturers need to make sure that mistakes are not translated into unsafe physical movements.

Confidence thresholds and human authority should be part of the system’s safety architecture, as should be hardware interlocks, said Wolfe.

Safety Standards Must Keep Pace

Current industrial robotics standards offer a valuable foundation, but are not necessarily the solution to all the problems presented by mobile humanoids.

The International Organization for Standardization (ISO) has published safety requirements for industrial robots, and ISO 10218-2 covers the integration of robots into complete systems.

These standards, however, do not address all applications or hazards of mobile robots working with humans.

Independent testing, scenario-based validation and phased workplace pilots are particularly important as industry practices evolve, said Wolfe.

Safety evaluations must also be repeated if changes in the AI model, software, movement restrictions, tasks or factory environment may render previous safety evaluations invalid.

After a major change, a robot that was safe in one set of conditions may not be safe in the new set of conditions.

Human Oversight Must Mean More Than Approval

It is impractical to monitor every decision an AI system makes, but when people are not involved in critical decisions, another challenge arises: accountability becomes more difficult to prove.

Adrian Figueroa Ramos, co-founder and CEO of Flexio and Routina, said that the focus of oversight should be on decisions that have the most uncertainty or risk.

“In today’s AI world, the volume of outputs is more than a human can process. Reviewing everything makes the human the bottleneck, but human judgment is still necessary for high-risk or low-confidence decisions,” he told Unite AI.

This is particularly true for physical AI. Human oversight should include named individuals with the power to override a system and ensure that decisions are documented and checked and not just click an approval button.

Who Takes Responsibility When a Robot Fails?

The accountability should be shared based on the control of each party over the system.

For instance, manufacturers are responsible for creating and documenting safety measures. Changes that may impact perception or movement should be tested by software developers.

The whole system needs to be validated in the intended environment by the integrator. Employers are responsible to train workers, ensure safeguards are in place and procedures are established for stopping unsafe operations.

Cybersecurity also matters. Operational technology can be vulnerable to sensor, software and external system issues connected to robots. A security failure that impacts physical behavior is a workplace safety problem, as well as an IT problem.

Humanoid robots are expected to become more flexible and productive in factories in the future. However, adoption will not be based solely on performance measures. There is a need for clear responsibility, independent validation and safeguards that are sustained throughout the working life of a robot.

The question is, can a robot make mistakes? It is, if the individuals who are using it have created a system that will hold them accountable for those errors.