Contact targets improve transfer of human demonstrations to robot hands

CARD, an arXiv preprint on robot retargeting, converts human demonstrations into both motion targets and contact-force targets. That gives different robot hands a way to preserve a demonstration’s physical purpose even when they cannot copy its exact posture. In a 30-trial hardware comparison, tactile tracking succeeded 25 times against four successes for position-only replay.[1][2]

The framework accepts several forms of human data: ordinary video, motion capture and tactile gloves. Those inputs are translated into a shared hand-and-contact representation. When video provides no direct force measurement, a model called Vid-HACO estimates contact from visual evidence. In simulation, the retargeter searches for robot motion that satisfies the contact targets without requiring a known object model or object pose.[1]

Why contact changes the solution

Pose matching is underdetermined across embodiments. A five-finger human hand, a robot hand with fewer joints and a parallel-jaw gripper cannot reproduce the same geometry. They may still accomplish the same task if they preserve the functional contacts: pinching an edge, supporting a base or maintaining pressure while rotating. By optimising for those interactions, CARD can choose a different-looking robot posture that is mechanically more useful than the closest visual imitation.[1]

This also changes what can be learned from video. A demonstration clip contains rich human dexterity but no force sensor. Vid-HACO estimates where contacts occur and supplies targets that the simulator can use. The paper reports that the method works across four robot hands ranging from a 22-degree-of-freedom dexterous hand to a parallel-jaw gripper. That cross-embodiment range is the central promise: human data need not be collected through every robot's own control interface.[1][2]

Hardware needs feedback even when the demonstration is fixed

Simulation cannot guarantee that the real fingertips arrive with the intended pressure. CARD therefore adds a tactile tracking controller for hardware replay. The high-level trajectory remains open loop, but fingertip force feedback corrects contact during execution. In the reported 30-trial hardware comparison, position-only replay succeeded four times and tactile tracking succeeded 25 times. The result attributes much of the improvement to preserving forces at execution, not only to a better offline trajectory.[1]

That distinction also defines the boundary of the claim. The robot is replaying a demonstrated clip; it is not deciding what task to perform, planning a novel sequence from language or recovering from every unexpected object movement. The system does not constitute a general autonomous manipulation policy. Its object-free formulation avoids requiring a precise object model for retargeting, but real deployments still need perception, task selection, safety constraints and failure recovery.[1]

The comparisons are constrained by what can be reproduced. Some recent retargeting systems do not release code, and methods can differ in data access, hand model and object information. Contact percentage is an informative intermediate measure, but the relationship between maintaining contact and completing a useful task depends on whether the right contact and force were specified. A robot can press consistently in the wrong place.[1]

Contact targets give human demonstrations a reusable description across different robot hands. The reported results support that representation for retargeting and replay, while the hardware comparison shows why maintaining force during execution matters as well as finding a suitable trajectory beforehand.[1][2]

Report an error or suggest a correction