Polymorph/Physical AI, ExplainedGlossary

Physical AI glossary

What is Foundation model?

Foundation model: one broadly trained model adapted to many tasks; in robotics, one model for many robot bodies and tasks.

In the guide: 3.3 The case for general models

The leading research groups are betting on robot foundation models. A foundation model is a single large model, trained on broad data, that can be adapted to many downstream uses; in robotics, the aim is one model that controls many kinds of robots and performs many tasks, instead of a separate system engineered for each. The idea comes from language processing. Researchers first built narrow systems, one for translation, another for sentiment analysis, another for question answering, each trained on its own curated dataset. General language models trained on vast amounts of text then outperformed all of them. Training on everything produced a broad internal representation of language and the world on which specific skills could build.

Read 3.3 in context →

In the guide: 3.12 Data and scaling laws

Data is the limit, for four reasons: - The gap is very large. One estimate puts the data a robot foundation model needs at around 100 trillion examples. The largest robot teleoperation datasets hold around a million, so almost the whole 100 trillion is missing. The figures are one company's estimate, and the conclusion holds even if they are far off. - More data does not mean more variety. In language, more data brings more variety automatically, because the internet covers so much of human life. In robotics it does not. Trillions of hours of self-driving data are still all driving. A hundred robots in one room learn only about that room, and 100 trillion tokens are useless if they all come from the same environment. - Variety matters more than volume, and this has been measured. When the most varied data was removed from one training set, performance on new tasks dropped sharply. Removing a random fifth of the less varied data barely changed anything. - Even unrelated data helps. Adding data unrelated to a task, even data collected in another country, has been reported to improve performance on it, with no sign of the effect levelling off.

Read 3.12 in context →

In the guide: 4.3 Teleoperation

Read 4.3 in context →

All 54 terms in the glossary · Read Physical AI, Explained