VicOne warns robot safety systems may fail with manipulated inputs
VicOne LAB R7 tested how false or malicious data can alter robot behavior, undermining safety protocols. In experiments, adversarial images, audio, and fabricated sensor messages caused robots to misinterpret instructions or bypass protective controls. For example, a robot-dog using Gemma 4 E4B followed text on a poster as an instruction, while a hospital-service robot simulated in NVIDIA Jetson…
Key points
- VicOne LAB R7 found adversarial images, audio, and sensor messages altered robot behavior in tests with Gemma 4 E4B and NVIDIA Jetson AGX Orin
- Current safety systems may fail if inputs are manipulated, as independent safeguards could share vulnerabilities or rely on the same compromised data
- VicOne recommends testing for coordinated attacks and revisiting safety evidence after software, model, or sensor changes
The findings challenge assumptions in safety assessments, which often assume faults occur randomly. Deliberate manipulation—such as repeated triggers or coordinated attacks—could exploit vulnerabilities in sensor fusion or shared safeguards. VicOne highlights gaps in current standards like China’s GB/T 45502-2025 and the EU’s upcoming Machinery Regulation, which require protecting safety-relevant data but lack clear testing frameworks for adversarial scenarios. The company’s tools, Radeis and Rthena, aim to help teams validate protections before deployment and monitor risks during operation.
Your Robot’s Safety Functions Already Work. What If the Input Lies?
IEEE Spectrum AI · 27 October 2026
Loading the full article…
This text was published by IEEE Spectrum AI and written by VicOne. It is reproduced here with attribution so you can read it in full; the rights remain with the publisher. Read it at the source ↗
The headline, key points and digest above were generated by Digest AI's editorial model from the linked sources. Automated summaries can contain errors: the sources are the record. Spotted a mistake? Tell us. Published by Martin K., who runs Digest AI and handles corrections.
More in Robotics & Physical AI
All →- Google releases Genie 3 world model with 720p, 24fps limits · 1 src
- ANYbotics ships ANYmal X for hazardous zone inspections in oil, gas, chemical plants · 2 src
- Machenta raises $40M seed to cut robot training time with one demonstration · 2 src
- AgiBot synchronizes 110 humanoids in Guinness-recorded dance routine · 2 src
- BlackBerry’s QNX backlog hits $1.2B as CEO bets on physical AI · 1 src
Comments
via GitHub Discussions