Polymorph/Physical AI, ExplainedGlossary

Physical AI glossary

What is World model?

World model: a responsive simulation close enough to reality to train, test, or imagine in; increasingly used inside the robot's decisions, not only offline.

In the guide: 3.5 Three families of robot model

2. World action models. This approach holds that a robot should predict what will happen next in the world, not only its own next move. A world model is a model that predicts how an environment will evolve, often conditioned on an action. One such model predicts future video of the scene together with the robot's actions, and was trained on more than a million hours of video, mostly of people (a reported result). It works like a language model reasoning step by step, except in pictures: the model imagines how the scene will change, then acts to bring that future about. The reasoning is that any physical system has to decide what to do next, which requires predicting the future; the robot is part of the world, so the model should predict as much of the world as it can. A practical attraction is that video of people is plentiful while robot data is scarce, so a model that learns physics from human video can draw on a much larger pool. The open difficulty is that generating video is computationally expensive, and the model must still translate imagined scenes into the specific commands of a particular robot body.

Read 3.5 in context →

In the guide: 4.9 Simulation spectrum and world models

"World model" has at least 20 definitions in current use. It helps to treat them as points on a spectrum of methods for creating a virtual world, ordered from hand-built and explicit to learned and implicit.

Read 4.9 in context →

In the guide: 4.9 Simulation spectrum and world models

A working definition: a world model is a simulation close enough to reality, and responsive enough, that a robot can act in it and the world reacts appropriately. A reactive (closed-loop) world responds to what the robot does, so a policy can be tested or trained inside it. An open-loop world simply replays a recording regardless of the robot's actions, which is useful for evaluating perception but cannot show the consequences of a different decision.

Read 4.9 in context →

All 54 terms in the glossary · Read Physical AI, Explained