Polymorph/Physical AI, ExplainedGlossary

Physical AI glossary

What is Wearable capture?

Wearable capture: collecting demonstrations from people wearing or holding capture devices, with no robot involved.

In the guide: 4.2 Five sources of robot data, and three tests

  1. Robots practising on their own. A robot attempts tasks autonomously and records the results, often as part of reinforcement learning, in which behaviour is improved by trial and error against a reward signal. The data is complete, since every sensor and motor is logged, and it requires no human operator. Its weaknesses are cost and signal. Every sample occupies a physical robot for real time, and early random attempts at a complex task almost never succeed, so most of the data records failure and teaches little. Large installations of robot arms grasping objects around the clock produced real progress on grasping, then reached a ceiling where additional robot-hours yielded little further improvement.
  2. Teleoperation. A person controls the robot through the task, and the robot records its own sensors and commands. This gives the highest-quality data, because it is produced by the robot's own body and contains exactly the observations and actions the robot will later use. It is slow and confined to wherever a robot happens to be (4.3).
  3. Wearable capture. A person wears or holds a gripper or glove fitted with a camera, and sometimes other sensors, and performs the task directly, with no robot present. The data is far more varied and cheaper per hour, but it sits one step further from the robot: the device approximates the robot's hand, and the person's arm and body differ from the robot's (4.4).
  4. Simulation. Virtual worlds can produce trillions of examples in a day, run faster than real time on many processors in parallel, and expose quantities, such as every contact force, that are hard or impossible to measure on hardware. Their weaknesses are diversity, since building many varied and realistic virtual scenes is laborious, and fidelity, since simulated physics never matches reality exactly (4.8).
  5. Human video. Video of people doing things, from head-mounted cameras or the open internet, is extremely varied and plentiful. It is also the furthest from the robot's body. It shows a human hand rather than a gripper, it usually lacks precise 3D positions, and it carries no force information. A viewer can see a hand close on an egg but cannot see how hard it squeezes (4.5).

Read 4.2 in context →

In the guide: 4.2 Five sources of robot data, and three tests

Ranked by quality, teleoperation usually comes first, then wearable capture, then simulation and generated data, with head-camera video last. The lower-quality sources clearly help in early training, when the model is still learning general structure. Whether they improve final performance on a specific robot, once enough robot data is available, remains genuinely disputed.

Read 4.2 in context →

In the guide: 4.4 Wearable capture

Why it matters: Data collection can itself be a company's core capability, and wearable capture is a third route to robot data alongside teleoperation and simulation, trading some closeness to the robot for far greater scale and diversity.

Read 4.4 in context →

All 54 terms in the glossary · Read Physical AI, Explained