Physical AI glossary
What is UMI?
In the guide: 4.4 Wearable capture
A team can dispense with the robot entirely, hand people an inexpensive capture device, and send them out into the world. One effort built thousands of these devices and collected about 500,000 hours of data, mostly of people performing everyday tasks in their own homes and workplaces across many countries (a reported result). The approach derives from an academic design called UMI (Universal Manipulation Interface), a hand-held gripper with a camera mounted on it. The camera records what the gripper sees, and the device's motion through space is reconstructed from the video, typically with visual tracking of the camera's position and orientation, so each recording yields both observations and a trajectory of gripper poses and openings that a robot can learn to reproduce.