Physical AI glossary
What is Reinforcement learning?
In the guide: 1.3 From programming robots to training them
In the last few years, robots have begun to learn behaviour from data instead of following hand-written instructions. The most common approach is imitation learning: a person demonstrates a task many times, usually by teleoperation, and a neural network learns to map what the robot sensed at each moment to the action the person took. A second approach, reinforcement learning, lets the robot improve by trial and error against a reward signal, often in simulation. In both cases the engineer's job shifts. Less time goes into specifying motions and enumerating failure cases, and more goes into where the data comes from, how good it is, and how to obtain more of it.
In the guide: 2.4 Imitation learning and VLAs went mainstream
For the first time in RCSV's survey, more teams named imitation learning as their primary training method (61%) than reinforcement learning (31%). Imitation learning trains a policy to reproduce the actions in human demonstrations, treating control as a supervised prediction problem; reinforcement learning (RL) instead has the robot learn by trial and error, improving its behaviour to maximise a reward signal, usually in simulation because real-world trial and error is slow and can damage hardware. The shift reflects the falling cost of demonstrations (2.3) and the practical difficulty of RL for manipulation: writing a reward function that captures what a task actually requires is hard, and policies trained in simulation must then cross the gap to the physics and appearance of the real world. Imitation learning has its own well-known weakness, compounding error: small mistakes carry the robot into states the demonstrations never showed, where its predictions are worse, so errors accumulate. Practitioners counter this with more diverse data, recovery demonstrations and policies that predict short sequences of actions rather than one step at a time. These are self-reported answers about a team's primary method, and many teams combine the two, for example by pretraining with imitation and refining with RL.
In the guide: 3.5 Three families of robot model
VLAs have improved in clear stages. The first versions joined a vision-language model to an action generator (see 3.6) and were never meant to be final. The next versions could split a large instruction such as "clean the kitchen" into steps before acting, and worked in homes they had never seen. After that, teams fine-tuned versions for specific jobs like making espresso or building boxes, then improved them with reinforcement learning. Most recently, a single general model matched or beat those fine-tuned specialists with no extra training (a reported result).