A robot does not necessarily need to carry the largest GPU it uses. Microsoft Research argues that mapping, planning, navigation and manipulation inference can be moved to a nearby edge server or the cloud, allowing the robot to use larger models while carrying lighter, less power-hungry hardware. The accompanying measurements show why the idea is attractive—and why it changes the safety problem.[1][2]
Once inference leaves the machine, the network is no longer ordinary background infrastructure. Delay, packet loss, bandwidth contention and a broken connection can alter what the robot perceives or when it acts. In physical AI, a slow answer can be a wrong answer even when the model output would have been correct a second earlier.
The reported trade-offs
| Measure | Value | Context |
|---|---|---|
| Mapping and planning slowdown | Up to 383% versus an A100 | A 383% increase means the operation can take about 4.83 times the baseline duration, not merely 3.83 times.Source 1 |
| Timely obstacle detection | 30% lower on lighter GPUs | The result concerns detection arriving in time, not only whether an obstacle is eventually recognised.Source 1 |
| VLA-model accuracy | Up to 50% lower under slowdown | Microsoft links slower inference to lower task accuracy in the tested setting.Source 1 |
| Battery lifetime | More than 100% improvement in one offloaded setup | The comparison replaced onboard GPU inference with a Raspberry Pi 5 and remote compute; it is not a universal robot-battery guarantee.Source 1 |
| Kairos latency reduction | 31.8%–66.5% | Reported average end-to-end task-latency improvement over digital-AI serving practices across tested robots and models.Source 3 |
383% slower is a control consequence
If a planning operation takes one second on the reference server, a 383% slowdown means roughly 4.83 seconds under the slower configuration: the original second plus 3.83 additional seconds. During that interval, a person can move, a door can open or a handover target can change position. Physical performance therefore depends on end-to-end freshness, not only tokens per second.
Where different work belongs
| Workload | Best default location | Reason |
|---|---|---|
| Emergency stop and collision envelope | On the robot | Must remain available under disconnection and meet a bounded response time |
| Fast local perception | Robot or nearby edge | Benefits from low latency and local sensor access |
| Large vision-language-action inference | Edge or cloud when the link permits | May need more memory and compute than the robot can carry |
| Long-horizon task planning | Edge or cloud | Can often tolerate more latency and use broader context |
| Audit, fleet optimisation and model updates | Central infrastructure | Benefits from shared data, capacity and coordination |
The robot needs a timing contract
A remote inference call should specify more than a timeout. The system needs the maximum acceptable age of sensor data, expected network delay, compute deadline, action validity window and a safe outcome if the deadline is missed. A response that arrives after its action has become unsafe should be rejected rather than executed because it is technically complete.
The local fallback also needs a defined scope. It may stop the robot, maintain balance, avoid obstacles or complete a limited motion safely while the remote service recovers. It should not quietly continue a complex task with a materially weaker model unless that degraded mode has been tested and authorised.
Fleet scale changes the economics again
Shared accelerators can improve utilisation across many robots, but they also create queues and correlated failure. Kairos treats the generate-execute loop as a first-class serving problem because physical agents alternate between inference and action rather than generating one continuous response. Scheduling must account for the deadline and state of each robot, not simply batch requests for throughput.[3]
The correct conclusion is not that every robot should become a thin client. It is that compute placement is now part of physical-AI design. Battery, model size, link quality, action deadlines and failure recovery must be evaluated together. Moving the AI off the robot can improve the machine—but only if the network is engineered as part of the machine.
Sources & further reading
- Offloaded inference for real-world physical AI robotics Microsoft Research · 23 Sep 2026 · Accessed 9 Oct 2026
- Offload or Overload: A Platform Measurement Study of Mobile Robotic Manipulation Workloads Microsoft Research · 1 Mar 2026 · Accessed 9 Oct 2026
- Kairos: A Scalable Serving System for Physical AI Microsoft Research · 1 May 2026 · Accessed 9 Oct 2026