Physical AI glossary
What is Chain-of-thought?
Chain-of-thought: the robot works out a plan in steps, in words or pictures, before acting, so it can use general knowledge.
In the guide: 3.8 Reasoning before acting
- Step-by-step reasoning. Before acting, the robot generates a written plan for itself. Told to clean a kitchen, a reasoning model first produces "pick up the dish, pick up the towel, wipe the spill," and then executes each step. Coding assistants do the same, writing a plan before writing code. This technique is called chain-of-thought. In robotics it is often implemented hierarchically: a high-level component, which may run only every few seconds, produces the next sub-task in words, and a low-level component converts that sub-task into motion at the control rate. Writing the sub-task down has a further benefit, in that a person can read it and see why the robot did what it did.
- Teaching with words. Because the model works in named steps, a new skill can be taught by describing its steps, without recording any new motion data, provided the robot already has the underlying motions. A robot has been taught to use an air fryer this way.
- Reasoning with pictures. Some information is hard to convey in words. "The power button is about three inches left of the edge" helps little, while a generated picture of the robot's finger on the button specifies exactly what to do. Recent models can generate images of where the robot should be a second or two from now, currently using a separate image model, and then act to reach that pictured state. The predicted image serves as a precise visual sub-goal.
- Detailed labels. In one model, labelling every training example in detail, with the current sub-step, how good the demonstration was and how fast it was performed, made a large difference. Video generators work the same way when they expand a short prompt into a long description: a more detailed prompt carries more information about the desired output, so the model learns more precisely what the right answer looks like. Quality labels also address the imitation problem described in 2.7, since at run time the model can be conditioned on "high quality" and so imitate the best demonstrations rather than the average one. In that case, detailed labelling mattered more than making the model bigger.