Robotics AI Faces On-Device vs Datacenter Inference Trade‑offs

Current generalist robot models sit in the billions of parameters. Physical Intelligence’s π0 class is around 3 B, while NVIDIA’s DreamZero tops at 14 B and requires two GB200 GPUs to run in real time. In contrast, RoboTTT trades model size for on‑device feasibility, using a 3 B policy that updates its weights at test time and can fit on a Jetson Thor. These examples illustrate how model size is constrained by the hardware that can be shipped with a robot.

Silicon and memory supply further shape the decision. Jetson Thor delivers only a fraction of the FLOPs of a datacenter GPU, and the cost of advanced‑node wafers is higher for edge silicon. When a fleet exceeds a handful of units, offloading the planning layer to a shared datacenter GPU can reduce silicon and DRAM per robot, making the approach more scalable. The article argues that the optimal strategy will likely involve a mix of onboard cognition and off‑robot compute, depending on latency, jitter, and cost.