Physical AI, Explained
A structured introduction to physical AI and robotics that explains how each concept works and marks the points on which researchers still disagree.
How to use this
Read the parts in order the first time, since each one builds on the last. Turn pages with the arrow keys or a swipe. Open Contents, or press C, to jump to any part or search for a topic. Hover over or tap an underlined word to see its definition; the Glossary (Part 12) lists them all.
The guide rests on a simple model of a physical AI system: three layers and a feedback loop. The layers are the body (the hardware that senses and moves), the brain (the AI model that turns sensor readings into actions), and the data from which the brain learns. The loop is deployment: robots working at real sites generate new data, and that data trains the next, better version of the robot. Nearly every concept in the field belongs to one of the three layers or to the loop, so a reader who loses the thread can usually recover it by asking which of the four a passage concerns.
The parts follow that model. Part 1 sets the scene and Part 2 gives the industry numbers for 2026. Parts 3, 4 and 5 take the brain, the data and the body in turn. Part 6 is about reliability, the hardest problem in the field today, and Part 7 about deployment. Part 8 covers markets and capital, Part 9 compute and the supply chain, and Part 10 what the robotics market rewards. Part 11 applies all of it to strategy for a company whose models, compute, and hardware keep getting cheaper.
Researchers and builders frequently disagree. Where they do, the guide presents both positions rather than choosing one. Figures that companies report about their own robots are marked as reported results, because they have rarely been reproduced under independent conditions.
Part 1. Physical AI vs. digital AI
1.1 Two kinds of AI
Digital AI operates entirely on information: conversation, code, search, ad ranking, image and video generation. It is trained mostly on data gathered from the internet, its inputs and outputs are both symbols (text, pixels, numbers), and an error usually costs little, because a wrong answer can be ignored, corrected or regenerated before it affects anything.
Physical AI controls machines that move in the real world: cars, trucks, drones, factory arms, humanoids. Such a system runs a continuous cycle of perception (reading cameras, joint encoders, force and touch sensors), decision (choosing the next action) and actuation (sending commands to the motors). Its output is a change in the physical world, which often cannot be undone: a dropped part stays dropped, and a collision has already happened by the time anyone notices it.
Four differences shape almost everything else about the field:
- There is no internet to train on. The useful data, such as how a particular factory line runs or how a hand closes on an egg without crushing it, is not on the web. Video of people working shows what happened but not the forces involved, and it is recorded from a human body, not a robot's. Robot data is also tied to the machine that produced it: a recording of one arm's joint angles does not transfer directly to an arm with different link lengths or a different gripper, a problem known as the embodiment gap. Someone has to go and collect this data, usually one demonstration at a time, which is why data is the field's biggest bottleneck (Part 4).
- Safety comes first. A heavy machine, or a humanoid that could fall on a child, needs far more testing than software does. What matters is behaviour in the long tail, the large set of rare situations that each occur seldom but together occur often: a reflective surface that confuses a camera, a person stepping into the work area, an object of an unfamiliar shape. A system can perform well on average and still fail badly in exactly these cases, so its average performance says little about whether it is safe to deploy.
- It runs in real time. A chatbot waits for a prompt and then answers. A robot works continuously and must react within milliseconds; the low-level control loops that hold a joint at its commanded position typically run at hundreds of hertz to a kilohertz. The robot also depends on hardware that changes over time: motors overheat and lose torque, sensors drift out of calibration, gears wear, and cables fatigue. The software has to keep working as the body underneath it degrades.
- It is tied to national interests. Physical AI touches defence, critical infrastructure, export controls and national rules on where data may be stored. Robots also record video inside factories, warehouses and homes, which raises questions of security and privacy that pure software rarely does. As a result each country tends to develop its own market rather than adopting one global product.
Why it matters: These four constraints, more than any shortage of ideas, explain why robotics has been "10 years away" for decades. Most of world GDP comes from physical industries, so companies that succeed in putting intelligence into machines could eventually become larger than purely digital ones.
1.2 Seventy years of slow progress
Robotics was one of the first goals of artificial intelligence research, and it has shown the least visible progress. Old footage makes the point. In 1957 a robot at a public exhibition was controlled by a person behind a curtain using a leader-follower rig: a second, matching arm that the operator moves by hand while the real arm reproduces each motion. Today this method is called teleoperation, and most labs still collect their robot training data this way. In the 1960s a university robot looked at a picture of stacked blocks and assembled the same stack from real ones, a task researchers called the "blocks world". Colorized and given a modern voiceover, those films would be hard for most viewers to tell apart from a 2026 demo.
Over the same period, chips, phones, computer vision and language technology advanced dramatically. Robotics did not, for two related reasons. First, the field treated robots mainly as a hardware problem, investing in better arms, motors and sensors while lacking a general-purpose way to decide what the machine should do. Second, researchers consistently underestimated how hard everyday physical skill is. AI researchers have long observed that tasks humans find intellectually demanding, such as chess or algebra, turned out to be comparatively easy for computers, while tasks a small child performs without thought, such as picking up an unfamiliar object or walking over uneven ground, turned out to be extremely hard. Human perception and motor control are the product of a very long evolutionary history, and they feel effortless precisely because so much of the work happens below conscious awareness, which hid their difficulty from the people trying to replicate them.
1.3 From programming robots to training them
For about 65 years, people programmed robots by hand. An engineer studied a task and specified every motion explicitly: the sequence of positions the gripper should pass through, the speed along each segment, and the conditions under which to stop. Converting a desired gripper position into the joint angles that produce it is called inverse kinematics, and classical industrial robotics is built on solving it precisely for positions known in advance. This works inside a cage, where every part sits in a known position to a fraction of a millimetre, because the robot is replaying a trajectory, not reacting to what it sees. It fails as soon as something moves. The cage, the fixtures (custom clamps and jigs that hold each part in exactly the expected place) and the measuring equipment around a classic factory robot can cost several times more than the robot itself, and any change to the product means redesigning the whole setup. This is why hand-programmed robots spread through high-volume, low-variation work such as car body welding and painting, and almost nowhere else.
In the last few years, robots have begun to learn behaviour from data instead of following hand-written instructions. The most common approach is imitation learning: a person demonstrates a task many times, usually by teleoperation, and a neural network learns to map what the robot sensed at each moment to the action the person took. A second approach, reinforcement learning, lets the robot improve by trial and error against a reward signal, often in simulation. In both cases the engineer's job shifts. Less time goes into specifying motions and enumerating failure cases, and more goes into where the data comes from, how good it is, and how to obtain more of it.
Until recently, robots had no meaningful intelligence in this sense: they could repeat, but not adapt. In roughly the last 18 months, learned systems have started to cope with uncertainty. Because the model reads its cameras continuously and chooses each action from the current scene, it can notice that an object has shifted, react, and still finish the task when things move around it. The change is in the source of the behaviour, which now comes from examples rather than from code, and this lets the same software approach extend to tasks that no engineer could have specified step by step.
Why it matters: Earlier "year of the robot" announcements came with no new method; the current wave comes with the learning method that already works for language and images.
1.4 Robots have to act without a person checking
Almost every successful AI product so far, from product recommendations to chatbots to coding assistants, leaves the final decision to a person. A recommendation can be skipped, a chatbot answer reread, a suggested line of code rejected before it runs. This arrangement, called human-in-the-loop, is why imperfect models are still useful: the person catches most errors, and the remaining ones are usually cheap.
A robot earns its keep only when nobody has to check each move. If an operator must watch the machine and step in whenever it hesitates, the labour saving largely disappears. Physical tasks also compound errors in a way single answers do not. A task such as unloading a bin is a long chain of steps (locate, grasp, lift, move, place), and the task succeeds only if every step does, so a small per-step failure rate multiplies into a much larger per-task failure rate. A learned policy has a further weakness: after one small error it finds itself in a situation that looks slightly unlike its training examples, which makes the next error more likely. Practitioners therefore measure deployed robots by how often a person has to intervene, typically as interventions per hour or the time between failures, rather than by accuracy on a test set.
Why it matters: Reliability without supervision is the central problem in the field today (Part 6), and a demo video, which shows a successful attempt and not the failure rate, says little about it.
1.5 Why it has to be a robot
Almost none of the universe is inside a computer: the internet is very small, and the universe is very big. Text and images record what people chose to write down or photograph, not how objects actually behave when pushed, lifted, squeezed or dropped. On this argument, a machine that understands physical things has to learn from the physical world, through a body that can act and observe the consequences. Cognitive scientists call the general idea embodied cognition: the claim that intelligence is shaped by, and partly built from, interaction between a body and its environment.
A stronger version of the argument starts from evolution. Animals spent hundreds of millions of years developing physical skill and then vision, while human language is only tens of thousands of years old. On this view, language is a late product of intelligence and not its foundation, and the capacity to perceive and act in the physical world is the main route to general intelligence. A model trained only on text would then be learning the thin final layer of intelligence without the deep structure beneath it.
The debate: That position is philosophical and unproven. The opposing bet is that text and video, at sufficient scale, can teach a machine most of what it needs, with robot data required only to fill the remaining gap. The evolutionary view remains a useful check on the common assumption that robotics will simply follow the path of language models.
Part 2. The industry in numbers, 2026
The figures in this part come from a single source: RCSV Research, State of Robotics 2026 (Robotics Center, March 2026). They should be read as one firm's estimates rather than as official statistics. The report is free and broad in scope, and it also serves partly as a pitch for the publisher's own data services, which gives it an interest in emphasising the importance of data. Industry reports of this kind typically combine surveys of companies that chose to respond, public funding announcements and the analysts' own modelling, so each figure inherits the biases of its method: survey respondents tend to be the more active and visible firms, self-reported numbers are rarely audited, and market totals depend heavily on where the authors draw the boundary of "robotics". For these reasons the individual numbers are less reliable than the direction they point in. That direction is consistent across the report: hardware and data are getting cheaper, and economic value is moving into the data and model layer, meaning the datasets, trained policies and the infrastructure for producing them.
2.1 Market size and growth
The report estimates that the global robotics market reached $38B in 2026, up 34% on 2025, which it describes as the fastest growth in a decade. A figure like this is best treated as an order of magnitude. Market sizing requires decisions about what to include (industrial arms, warehouse vehicles, service robots, components, software, integration services), and two analysts using different definitions can arrive at very different totals from the same underlying activity. Growth rates computed within a single report are more trustworthy than the absolute level, because the same definition is applied to both years.
The more interesting claim is qualitative. In RCSV's reading, companies now differentiate themselves less by the robot itself than by whether they can run a repeating loop: collect data from real tasks, train a policy (the learned function that maps what the robot senses to the actions it takes) on that data, redeploy the improved policy, and do this again, quickly and at a cost the business can sustain. A company that can turn each deployment into training data for the next one improves with use, whereas a company selling a fixed machine improves only when it ships new hardware. This is the deployment flywheel seen from the market side.
2.2 Robot parts are becoming commodities
A commodity, in the economic sense, is a good that is standardised enough that buyers can switch between suppliers mainly on price. The components a robot is built from, such as actuators (the motor, gearbox and sensor units that move each joint), cameras, batteries and embedded computing boards, are increasingly of this kind: standard, mass-produced parts that any team can buy off the shelf rather than design in-house.
The effect already shows in finished robots. According to the report, fourteen manufacturers sell six- and seven-DoF arms for under $10K, eight of them Chinese. Degrees of freedom (DoF) count the independent ways a mechanism can move, which for an arm usually means the number of actuated joints. Six DoF is the minimum needed to place a tool at any reachable position and orientation in space; a seventh joint adds redundancy, letting the arm reach the same pose in more than one configuration so it can avoid obstacles or awkward joint limits. Twelve humanoid platforms can now be bought or leased, up from three in 2024, priced from $28K for an upper-body torso to $245K for a full biped, with lease programmes starting at $3.5K a month. The spread between those prices reflects real engineering differences: a torso mounted on a fixed base or wheeled platform avoids the hardest problems of a walking robot, namely dynamic balance, the high-torque leg actuators it requires and the cost of falls. Despite the growth in available platforms, full humanoids remain a small share of the robots actually shipped, which are still dominated by arms and mobile bases doing well-defined jobs.
These are list prices, and they should be read with that limit in mind. A purchase price excludes the grippers or hands, sensors, integration work, safety measures, maintenance and the engineering time needed to make a robot do a particular task, which in practice often exceed the cost of the hardware itself.
Why it matters: When capable parts are cheap and standard for everyone, owning them stops being an advantage. Value moves to what no supplier sells: knowing which tasks to automate, making robots work reliably at customer sites, and the data those deployments produce (11.2).
2.3 Data got 60% cheaper
The report puts the cost of a fully loaded hour of labelled teleoperation data at about $340 in early 2024, falling to $118 by March 2026. Teleoperation is the practice of a human operator controlling a robot remotely while the system records what the robot sensed and what it was commanded to do; each recorded attempt at a task is an episode, and a collection of good episodes becomes training data for imitation learning. "Fully loaded" means the figure includes everything required to produce a usable hour: operator wages, equipment, the robot's time, supervision, annotation and quality control, rather than the operator's time alone. "Labelled" means the episodes have been annotated with information such as task boundaries, success or failure, and language descriptions of what was done.
The report attributes the decline to three developments. The first is low-cost leader-follower teleoperation rigs, available for under $2K, in which the operator physically moves a small replica arm (the leader) and the robot arm (the follower) copies its joint positions in real time. The joint-to-joint mapping needs no motion capture or inverse kinematics. The second is annotation pipelines that cut labelling labour by 40 to 60%, typically by using models to propose segment boundaries and descriptions that humans then check rather than write from scratch. The third is the field settling on a few standard episode formats, so that recordings from one rig or lab can be loaded by common training tools without bespoke conversion code.
RCSV estimates that most manipulation tasks need 300 to 1,200 good demonstrations to generalise across 80% of in-distribution variation, meaning variation of the same kind the training data covered (different object positions, lighting or clutter within the expected range), as opposed to genuinely new situations. On that basis it puts a pilot's data budget at roughly $50K to $150K. Two qualifications apply. The demonstration counts depend strongly on task difficulty, on how much the scene varies and on the policy architecture, so they are a planning range, not a law. And the word "good" carries much of the weight: demonstrations that are hesitant, inconsistent between operators or full of corrections can teach a policy contradictory behaviour, which is why data quality often matters as much as quantity (2.4).
2.4 Imitation learning and VLAs went mainstream
For the first time in RCSV's survey, more teams named imitation learning as their primary training method (61%) than reinforcement learning (31%). Imitation learning trains a policy to reproduce the actions in human demonstrations, treating control as a supervised prediction problem; reinforcement learning (RL) instead has the robot learn by trial and error, improving its behaviour to maximise a reward signal, usually in simulation because real-world trial and error is slow and can damage hardware. The shift reflects the falling cost of demonstrations (2.3) and the practical difficulty of RL for manipulation: writing a reward function that captures what a task actually requires is hard, and policies trained in simulation must then cross the gap to the physics and appearance of the real world. Imitation learning has its own well-known weakness, compounding error: small mistakes carry the robot into states the demonstrations never showed, where its predictions are worse, so errors accumulate. Practitioners counter this with more diverse data, recovery demonstrations and policies that predict short sequences of actions rather than one step at a time. These are self-reported answers about a team's primary method, and many teams combine the two, for example by pretraining with imitation and refining with RL.
VLA models (vision-language-action models) are large neural networks that take camera images and a natural-language instruction as input and output robot actions, typically built by extending a model pretrained on internet-scale image and text data. Almost absent from production in 2024, they now underpin about 40% of new deployments, according to the report. The report also finds that fine-tuning a base VLA, meaning continuing to train an already pretrained model on a small task-specific dataset, on 200 to 500 task demonstrations now outperforms training a task-specific policy from scratch on 1,000 or more. The mechanism is transfer: the pretrained model already encodes general knowledge of objects, scenes and language, so the fine-tuning data only has to teach the specifics of the new task and robot. The comparison depends on the tasks and benchmarks chosen, and the report's summary does not allow it to be checked independently.
The debate: The report argues that beyond roughly 7B parameters, curation beats scale: a well-curated 500-demonstration fine-tune of a 7B model outperforms a poorly curated fine-tune of a 70B model on most manipulation benchmarks. If it holds, competitive advantage lies in collecting and cleaning task data rather than in training ever larger models, though benchmark results do not always carry over to real deployments.
2.5 Where robots are deployed
Logistics, semiconductors and food service account for most commercial deployments by unit volume, that is, by the number of robots installed rather than by the revenue they generate. These sectors share features that suit current robots: repetitive tasks, structured or controllable environments, and high labour costs or shortages.
The fastest growth is in the long tail of smaller sectors: healthcare and lab support (+94%) and food service (+61%). High percentage growth from a small base is common for young markets and can reflect a modest absolute number of new units, so these rates indicate where interest is forming rather than where most robots are. Lab and hospital work is attractive for the same reason as logistics, since tasks such as moving samples, loading instruments and delivering supplies recur constantly, but it adds stricter requirements for safety, hygiene and validation.
2.6 Where the money goes
Venture investment in robotics reached $9.4B in 2025, up 41%, but the ten largest rounds took 58% of it. That concentration means the headline total says little about the typical company: a handful of very large rounds, often for humanoid and foundation-model companies, can lift the aggregate while most startups raise far less. Median Series A pre-money valuations rose to $42M. A Series A is usually a company's first major institutional funding round after its seed stage, and the pre-money valuation is the value investors assign to the company immediately before their new money is added.
The report also finds that companies with proprietary data collection command a valuation premium of 1.4 to 1.8x, and that those with ten or more paying customers in one use case earn a further 1.3x. These multipliers are correlations drawn from the deals the analysts observed, not controlled measurements. Companies with their own data pipelines or many paying customers may differ in other ways too, such as team experience or revenue, that also raise valuations.
Why it matters: Investors pay a premium for proprietary data and for depth in one use case, but that premium is under pressure as fine-tuning foundation models shrinks the data advantage incumbents once had.
Reference: Robotics Center (2026). State of Robotics 2026: Annual Report on the Global Robotics Industry. RCSV Research. roboticscenter.ai/state-of-robotics-2026
Part 3. Robot intelligence
A robot model is a learned function that maps camera images, joint positions and an instruction to motor commands. It acquires that mapping from human demonstrations and from its own practice, and the current evidence is that models trained across many robots and many tasks outperform models built for a single one. This part explains what such a model consumes and produces, how the main families differ, how they generate motion, how they learn, and where their limits currently lie.
3.1 Capable bodies with no brain
For decades the mechanical side of robotics was, for most purposes, adequate. Industrial arms could already move with sub-millimetre repeatability, carry heavy payloads and run for years. What they could not do was cope with a world that had not been arranged for them. A classical industrial robot replays a programmed trajectory; if the part arrives rotated or a box is left in its path, it has no way of noticing, let alone adapting. The missing ingredient was the intelligence to perceive a scene, interpret it, decide what to do and act on that decision while the scene changes.
Earlier attempts to supply that intelligence relied on hand-written rules and explicit geometric models of objects and environments. These worked inside controlled cells and broke down in open settings, because no engineer can enumerate every situation a robot will meet. The modern field rests on a different bet: that this intelligence can be learned from data, in the same way that language and image models learned their abilities, rather than programmed case by case.
3.2 Three sub-problems: semantics, planning, control
A useful robot needs three abilities working together. Each is partly solved in isolation, and most of the remaining difficulty lies in combining them so that a failure in one does not undo the others:
- Semantics means understanding the instruction and what is in the scene: that "the mug" refers to the blue object on the left, that a towel is soft and a glass is fragile. Large language models and vision-language models made rapid progress here, because this kind of knowledge is abundant on the internet.
- Planning means putting steps in the right order to reach a goal. Clearing a table requires picking up the cups before wiping the surface, and opening a drawer before placing something in it. Planning errors are often invisible in short demos, which rarely require more than a few steps.
- Control means moving in real time while the world keeps changing. The robot repeatedly measures what actually happened, compares it with what it intended, and corrects the difference. This feedback cycle is called closed-loop control, in contrast to open-loop control, which executes a planned motion without checking the result. Closed-loop control is what allows a robot to keep its grip when an object turns out to be heavier than expected, or to adjust its foot when a step is slightly higher than predicted.
The third ability is often called sensorimotor common sense: the continuous coupling of perception and movement that lets a body adjust to its surroundings without deliberate thought. Humans tend to undervalue it because nearly everyone has it, and because it operates below conscious awareness. It is also the ability for which the internet provides almost no training data, since text and video record what people did, not the forces they felt or the corrections they made.
The three abilities operate on different timescales. Semantic understanding may be updated once per instruction, planning every few seconds as sub-tasks complete, and control many times per second. Many system designs mirror this split, with a slower reasoning component setting goals for a faster component that produces motion.
Why it matters: A claim that "the robot works" should prompt the question of which of the three abilities is meant, since a polished demo can conceal a serious weakness in any one of them.
3.3 The case for general models
The leading research groups are betting on robot foundation models. A foundation model is a single large model, trained on broad data, that can be adapted to many downstream uses; in robotics, the aim is one model that controls many kinds of robots and performs many tasks, instead of a separate system engineered for each. The idea comes from language processing. Researchers first built narrow systems, one for translation, another for sentiment analysis, another for question answering, each trained on its own curated dataset. General language models trained on vast amounts of text then outperformed all of them. Training on everything produced a broad internal representation of language and the world on which specific skills could build.
For a sufficiently difficult intelligence problem, enlarging the problem can make it easier. A model trained on all of language translated better than a model trained only on parallel translations, because it had learned grammar, idiom and world knowledge that the translation data alone could not supply. The effect is expected to be stronger in robotics, because physical situations vary so widely that any single task's data covers only a small fraction of what a robot might encounter, while a general model sees far more unusual situations across its training set.
Three further arguments favour generality:
- Single-task robots stall. A robot built for one job, such as welding, typically reaches 80 to 90 percent performance quickly. It then meets rare situations, such as a box left in the way or the lights going off. A single task never produces enough examples of these events to learn from, so a person has to remain nearby to handle them, and the economics stop working. The last stretch of reliability is where most of the cost of automation sits, because each remaining failure mode is individually rare and therefore expensive to collect data for.
- Rare cases transfer between settings. The corner case of one industry is often the central case of another. An object falling from a shelf may be rare in a factory and routine in a warehouse; a reflective or transparent item may be unusual in one kitchen and common in a laboratory. A model trained across many settings can learn each setting's rare cases from another setting's everyday data.
- Recovery skills transfer. In one experiment, a robot that recovered from a mess it had never seen drew much of that ability from recovery data recorded on other tasks. Recovery behaviours, such as regrasping a slipping object or backing away from a collision, share structure across tasks even when the tasks themselves differ. A model trained on one task sees few such episodes and therefore copes worse with surprises.
The practical pattern that has emerged resembles the language field: a general model is pretrained on large, varied data and then fine-tuned, meaning trained further on a smaller set of examples from the target task, so that it specialises without discarding what it learned during pretraining.
Why it matters: Companies increasingly build or adopt one general model and adapt it to each task, from dishes to laundry, rather than engineering a new system per application.
The debate: General models can outperform specialists without a single general robot winning every market, and both general and task-specific robots are likely to be large markets, much as specialised language models for code or mathematics are regaining ground. Whether the general model itself becomes a company's durable advantage is a separate question (7.9).
3.4 What a robot model takes in and puts out
- Inputs: camera images (sometimes supplemented by depth or lidar data, which is usually converted into an image-like grid as well), the robot's own joint angles, and an instruction, usually in natural language. The robot's sense of the position of its own joints is called proprioception, by analogy with the human sense that allows a person to touch their nose with closed eyes. Some systems also take in force or touch readings, the gripper's opening width, or a short history of recent actions.
- Outputs: a vector of numbers specifying where each joint or gripper should move next. A typical model outputs roughly 30 to 50 numbers per step, and one semi-humanoid robot uses about 100. That robot's model produces a new set of numbers 10 to 20 times a second. One walking model processes vision and produces actions 100 times a second.
Before a transformer-based model (3.6) can process these inputs, each must be converted into tokens, the fixed-length vectors that the model operates on. Text is split into word fragments, each mapped to a learned vector. Images are typically cut into a grid of small square patches, and each patch is projected into a token, so a single camera frame becomes a few hundred tokens. Joint angles are scaled into a standard range and projected into tokens of their own. The model then attends across all of these tokens together, which is how it relates the word "cup" to a region of the image and to the current position of the arm.
The output side involves a design choice called the action space: what the numbers represent. The common options are target joint positions; a target pose for the gripper in space (its position and orientation), which a separate inverse kinematics routine converts into the joint angles that achieve it; joint velocities; or, more rarely, joint torques. Most learned models output target joint positions instead of driving the motors directly. A small classical controller then pushes each motor toward its target hundreds of times a second. This controller is usually a PD controller (proportional-derivative): it applies a torque proportional to the gap between the target and the current position, minus a term proportional to the joint's velocity that damps oscillation. The rule is older than modern robotics and needs no learning.
This layering resolves a timing mismatch. A large model cannot run hundreds of times a second, but the motors need commands at that rate. The model therefore sets targets at its own slower rate, and the low-level controller interpolates and tracks them in between. The choice of output level does not currently appear to be a bottleneck; systems using joint targets and systems using gripper poses both reach high performance.
Some teams build strictly end-to-end systems, which take camera pixels in and send motor targets out, with no separate modules for object detection, mapping or motion planning in between. Any planning happens implicitly inside the model, learned from the data. The advantage is that no hand-designed interface between modules can throw away information the task needs; the cost is that failures are harder to diagnose, because there is no intermediate output to inspect.
Why it matters: Every robot "brain" in a demo is this mapping from inputs to numbers, and what separates good models from poor ones is whether they produce the right numbers in situations they have never seen.
3.5 Three families of robot model
The leading teams build robot models in three distinct ways and disagree about which will prevail.
1. Vision-language-action models (VLAs). A VLA is a large language model adapted to control a robot. It is built in three stages:
1. Train it on text, so it learns language and general knowledge.
2. Train it further on web images paired with text, so it learns to connect words to what it sees. A model at this stage is called a vision-language model.
3. Train it further on varied robot data, so it learns to output actions instead of words.
VLA builders start from a language model because the two kinds of knowledge complement each other. Robot data comes from the real world but contains little general knowledge: a few hours of arm motions say nothing about what a colander is for. A language model contains a great deal of knowledge but has never touched anything. Each supplies what the other lacks. In practice the third stage must be handled carefully, because training heavily on robot data can erode the knowledge acquired in the first two, an effect known as catastrophic forgetting. Builders mitigate it by continuing to mix web data into robot training, or by attaching a separate action-generating component so that the language backbone is disturbed less.
There are two main ways to make a language model emit actions. The first discretises each action dimension into a fixed set of bins and treats each bin as a token, so the model "writes" an action exactly as it would write a word. The second attaches a dedicated action expert, a smaller network that reads the language model's internal representation and produces continuous actions, usually with diffusion or flow matching (3.6). The second approach generally yields smoother and more precise motion at high control rates.
VLAs have improved in clear stages. The first versions joined a vision-language model to an action generator (see 3.6) and were never meant to be final. The next versions could split a large instruction such as "clean the kitchen" into steps before acting, and worked in homes they had never seen. After that, teams fine-tuned versions for specific jobs like making espresso or building boxes, then improved them with reinforcement learning. Most recently, a single general model matched or beat those fine-tuned specialists with no extra training (a reported result).
2. World action models. This approach holds that a robot should predict what will happen next in the world, not only its own next move. A world model is a model that predicts how an environment will evolve, often conditioned on an action. One such model predicts future video of the scene together with the robot's actions, and was trained on more than a million hours of video, mostly of people (a reported result). It works like a language model reasoning step by step, except in pictures: the model imagines how the scene will change, then acts to bring that future about. The reasoning is that any physical system has to decide what to do next, which requires predicting the future; the robot is part of the world, so the model should predict as much of the world as it can. A practical attraction is that video of people is plentiful while robot data is scarce, so a model that learns physics from human video can draw on a much larger pool. The open difficulty is that generating video is computationally expensive, and the model must still translate imagined scenes into the specific commands of a particular robot body.
3. Models trained mainly on physical data. This approach trains the model almost entirely on physical data. In one reported case, about 99 percent of the model was trained from scratch, keeping only about 1 percent from a pretrained model. On this view, language pretraining is a crutch once enough physical data exists, much as reading about skiing never makes anyone a good skier. Physical interaction is the foundation, and language is added on top so that people can give the robot instructions.
The debate: VLA advocates argue that web knowledge lets a robot handle the unexpected; world-model advocates argue that "actions" are an abstraction carrying human assumptions and that the bitter lesson, the observation that simple methods with more data beat hand-designed ones, favours predicting raw video; physical-data advocates argue that at enough scale such data teaches dexterity better than web text. A fourth position cuts across all three: the data matters more than the architecture, so the data available now, and the data the robot will generate as it improves, should determine the model design.
3.6 How the models make smooth, precise motion
Two technical terms appear in almost every description of a current robot model.
- Transformer is the neural network design used in large language models. Its core operation, attention, lets every token in the input weigh its relevance to every other token, so the model can relate a word in the instruction to a region of an image and to the arm's current position in a single computation. Transformers keep improving as they are given more data and more parameters, which is the main reason they have spread across fields as different as gene therapy and robotics, applied to very different data.
- Diffusion, and its simpler relative flow matching, are methods for generating an output by starting from random noise and refining it over a series of steps until it is sharp and precise. Image generators use them to produce crisp edges. Robots use them because their actions also need to be precise and smooth.
The reason a robot needs a generative method, rather than a network that simply outputs one best guess, is that demonstrations are often multimodal: there are several valid ways to perform the same action. Asked to move around an obstacle, one demonstrator goes left and another goes right. A network trained to minimise average error on such data learns to predict the average of the two, which drives the arm straight into the obstacle. A generative method instead learns the full distribution of valid actions and samples one coherent option from it.
Diffusion achieves this as follows. During training, the model is shown real action sequences from demonstrations, corrupted with varying amounts of random noise, and learns to predict the noise that was added (equivalently, to recover the clean actions). At run time it starts from pure noise and repeatedly removes the predicted noise, conditioned on the current camera images, joint positions and instruction, until a clean action sequence remains. Flow matching reaches the same goal more directly. It learns a velocity field that transports noise to data along paths that are close to straight lines, and generation consists of following that field from a noise sample to an action. Straighter paths can be followed accurately in fewer steps.
Most of these models also use action chunking: instead of predicting one action at a time, they predict a short sequence of future actions in one pass, execute part or all of it, and then predict again from the new observation. Chunking has three benefits. It produces smoother motion, because consecutive actions are generated together and are therefore consistent. It reduces the number of separate decisions at which an error can creep in. And it copes better with demonstrations in which the person paused or hesitated, which would otherwise confuse a model predicting one step at a time. Some systems overlap successive chunks and blend them to avoid a visible jolt at each boundary.
Combining the two ideas gives a diffusion transformer: the transformer interprets the inputs and learns from large amounts of data, and the diffusion or flow-matching process turns its output into fine, smooth motion. Most of the field now uses some version of this combination. Flow matching has become especially popular because it needs less tuning and fewer refinement steps, which matters when a robot must recompute its actions many times a second on hardware that also has to run everything else.
3.7 Imitation vs. reinforcement learning
- Imitation learning, also called behavioural cloning, means recording demonstrations of a task, usually a person controlling the robot remotely (teleoperation), and training the model to reproduce the demonstrated actions from the same observations. Almost every system starts this way, because it is the most direct way to give a model a working behaviour. It tends to top out near human level, because the model is copying people, mistakes included. In one case, robots learned to always miss first and then pick up the object, because the people giving demonstrations sometimes fumbled and tried again.
- Reinforcement learning (RL) means the robot attempts the task itself, receives a numerical signal called a reward when it does well, and adjusts its behaviour to increase the total reward it collects. RL can exceed human performance because it is not limited to moves a person made. The famous example is "move 37," a move by a Go-playing program that no human player would have chosen.
Imitation learning has a characteristic failure mode called compounding error. A cloned policy is never perfect, so at each step it deviates slightly from what the demonstrator would have done. Those small deviations carry the robot into states that never appeared in the demonstrations, because the demonstrator never made those particular mistakes. In such unfamiliar states the model's predictions are worse, so it deviates further, and the errors accumulate over the course of a task. The underlying mismatch between the states seen in training and the states encountered at run time is called covariate shift. The standard remedy is to collect data in the states the robot actually reaches: a person watches the robot run, takes over when it drifts, and the corrective actions are added to the training set. This interactive procedure, known as dataset aggregation, teaches the model how to get back on track from its own typical mistakes. The "miss first" example illustrates a related problem: imitation copies whatever is in the data, so practitioners filter poor demonstrations or label each one by quality so that the model can be asked to imitate only the good ones (3.8).
RL has its own difficulties. The first is reward design. A sparse reward, given only when the task succeeds, is easy to specify but gives the robot almost no information during the many early attempts that fail. A dense reward, which scores partial progress such as distance from the gripper to the object, speeds learning but can be exploited: the robot may find a way to increase the score without doing the task, a behaviour called reward hacking. A common modern approach is to train a separate model to judge from camera images whether a task has succeeded or how much progress has been made, and to use its judgement as the reward. The second difficulty is sample efficiency: RL typically needs very many attempts, which is cheap in simulation and slow, expensive and potentially damaging on real hardware, where a person may also have to reset the scene after each attempt. The third is safe exploration, since trying random movements near people or fragile objects is unacceptable. For these reasons much RL is done in simulation, with domain randomisation (varying friction, mass, lighting, sensor noise and other simulator parameters during training so that the real world looks like just another variation) used to make the learned behaviour survive the transfer to a physical robot, the sim-to-real gap.
Over a robot's working life the two methods combine. First people teach it, then people correct it, and once it is clumsy but autonomous, RL removes the clumsiness. The division of labour is precise: a human correction teaches the robot how to recover after dropping something, while practice teaches it to stop dropping things. Teams are now finding ways to make real-world RL affordable (Part 6).
The grand challenge: The two largest results in AI are generative AI and deep RL. Generative AI, such as large language models, reproduces human abilities and holds an enormous amount of world knowledge; deep RL, such as the Go-playing programs, finds new solutions that surpass humans. The current frontier in robotics is combining them: begin with the world knowledge of generative AI, then use RL to go beyond human performance.
Why it matters: The mix a team bets on determines where its money goes: progress driven mostly by demonstrations calls for investment in data collection, while progress driven by a few demonstrations plus extensive practice calls for investment in deploying robots and letting them practise safely.
3.8 Reasoning before acting
The hardest part of the real world is the long tail of unusual situations that no training set anticipated. People handle them with common sense learned elsewhere. Robots now obtain some of this from language models, and several techniques have emerged for putting it to work.
- Step-by-step reasoning. Before acting, the robot generates a written plan for itself. Told to clean a kitchen, a reasoning model first produces "pick up the dish, pick up the towel, wipe the spill," and then executes each step. Coding assistants do the same, writing a plan before writing code. This technique is called chain-of-thought. In robotics it is often implemented hierarchically: a high-level component, which may run only every few seconds, produces the next sub-task in words, and a low-level component converts that sub-task into motion at the control rate. Writing the sub-task down has a further benefit, in that a person can read it and see why the robot did what it did.
- Teaching with words. Because the model works in named steps, a new skill can be taught by describing its steps, without recording any new motion data, provided the robot already has the underlying motions. A robot has been taught to use an air fryer this way.
- Reasoning with pictures. Some information is hard to convey in words. "The power button is about three inches left of the edge" helps little, while a generated picture of the robot's finger on the button specifies exactly what to do. Recent models can generate images of where the robot should be a second or two from now, currently using a separate image model, and then act to reach that pictured state. The predicted image serves as a precise visual sub-goal.
- Detailed labels. In one model, labelling every training example in detail, with the current sub-step, how good the demonstration was and how fast it was performed, made a large difference. Video generators work the same way when they expand a short prompt into a long description: a more detailed prompt carries more information about the desired output, so the model learns more precisely what the right answer looks like. Quality labels also address the imitation problem described in 2.7, since at run time the model can be conditioned on "high quality" and so imitate the best demonstrations rather than the average one. In that case, detailed labelling mattered more than making the model bigger.
The main cost of these techniques is latency. Generating text or images before acting takes time, so most systems reason at the slower, higher level and leave the fast control loop untouched.
Why it matters: No team can collect data for every situation, so words and pictures become inexpensive ways to steer a robot alongside demonstrations.
3.9 Compositional generalization
A capable language model can write whole paragraphs in the phonetic alphabet, even though dictionaries show that alphabet only one word at a time. Combining known pieces in a new way like this is called compositional generalization. An image generator can likewise draw an armchair shaped like an avocado. The model never needed a picture of that exact combination, because it had learned "armchair" and "avocado" as separable concepts that can be recombined.
The distinction that matters here is between interpolation, producing something similar to what was seen in training, and composition, assembling familiar parts into a configuration that never occurred. Classical robot learning rarely achieved the second: a policy trained on one object in one place often failed when either changed. Robots trained on broad data now show it:
- A rare appliance. One general model used an air fryer that appeared in only three training examples, by combining skills it already had: open, place, close.
- A known task on a new robot. A large industrial two-armed robot folded a shirt with no folding data from that robot, and came close to the quality of a person controlling it remotely (a reported result). The folding skill came from other robots and the control of this body from other tasks.
- Switching hands. A robot trained to always hold a pin in its right hand ended up with the pin on the wrong side, and used its left hand instead. Nothing like that was in its training data.
- The same effect elsewhere. On other robots, one taught to shake a bag with its left hand once used its right. Another robot, knocked with a hockey stick, reached across with its other hand to catch a washer in mid-air.
Composition depends on the model having learned representations in which skills, objects and bodies are somewhat independent of one another. It is not guaranteed, and current models still fail at combinations that seem trivial to a person, so these results show that the capability exists, not that it is reliable.
Why it matters: Composition is the reason general models pay off, because data collected for one task can improve tasks it was never collected for.
3.10 Cross-embodiment transfer
Embodiment means a specific robot body, with its own sensors, motors, joints and proportions. Cross-embodiment means one model controlling many different bodies. Surprisingly, a single model has worked on robots with different numbers of joints, and even on multi-fingered hands, without being told which robot it was running on.
The technical obstacle is that bodies differ in their degrees of freedom (the number of independent ways a robot can move, roughly one per joint), their camera placements, their joint limits and their dynamics. Practitioners handle this in a few standard ways. Action vectors from different robots can be padded to a common length, with unused entries ignored. Actions and joint readings can be normalised to a shared range so that one robot's large numbers do not dominate another's small ones. Some designs give each embodiment its own small input and output layers around a shared core, so that the core learns skills common to all bodies while the edges handle each body's specifics.
Neuroscience suggests why a body-agnostic model is plausible. When monkeys use tools, neurons that track the position of the hand begin to respond at the tip of the tool. The brain treats the tool as part of the body, which implies that a good robot brain should not depend heavily on which body it inhabits. The strongest evidence comes from the Open X-Embodiment project, which pooled data from many institutions. One general model trained on about 10 types of robot beat 10 specialist models, each tuned to one robot, by 50 percent on the specialists' own robots. Part of the explanation is that even a single robot changes over time, through worn motors, recalibrated cameras and hardware revisions. Training on many bodies teaches the more general skill of controlling whatever robot the model finds itself in.
Some teams take this as far as it goes, with one model meant for any robot and any task. Their argument is partly about data, since robot data is so scarce that no robot's data can be wasted, and partly about safety. When a joint breaks, a 32 degree-of-freedom robot becomes a 30 degree-of-freedom robot. A model tuned to one exact body fails at that point, while a model trained on many bodies treats the damaged robot as a new one and adapts. In reported demos, a four-legged robot with a failed knee keeps walking on three legs within milliseconds, a dog-like robot with useless front legs stands up and walks on two, and a wheeled robot with jammed wheels starts walking.
The model works out its new body from the last few moments of experience, without retraining. Given a short history of the commands it sent and the motion that resulted, it infers how this body responds and adjusts accordingly, an implicit form of what control engineers call system identification. This is called in-context learning, the physical counterpart of a language model picking up a pattern from examples in its prompt: the model's weights do not change, only the context it conditions on. The ability was reportedly not designed in; it was noticed when a robot kept walking with broken ankle motors.
Why it matters: Data and models should be planned for many robot types from the start, since a model tied to one body has no fallback when a joint fails.
The debate: Some teams deliberately master a single robot body before attempting cross-embodiment, which is a decision about focus rather than evidence against the approach. The humanoid race shows the cost of skipping it: humanoid hardware is meant to be general, yet each company's model works only on its own humanoid.
3.11 Memory and context
Most state-of-the-art robot foundation models have no memory or context. They act only on what the cameras show at that moment, a property called being memoryless or reactive. That suffices for short, repetitive motions, where the current image contains everything needed to choose the next action, but long jobs require memory. Halfway through opening a microwave, a single frame cannot show whether the robot is opening or closing it. A dryer's blinking indicator may appear off in one frame even though the machine is running. In both cases the correct action depends on information that is no longer visible, and a memoryless policy can oscillate, repeat a completed step or abandon one halfway.
Remembering everything is too expensive. Ten seconds of video, at 50 frames a second, from four cameras, at about 256 tokens per image, comes to around half a million tokens. That is far too much to process in real time, particularly because the cost of attention in a standard transformer grows with the square of the number of tokens: doubling the context roughly quadruples that part of the computation. Even at one frame a second the total is about 10,000 tokens, and that still covers only 10 seconds. The constraint is latency as much as cost, since a robot that takes too long to decide is acting on a scene that has already changed.
One solution keeps memory at two levels of detail, much as a person remembers the last few seconds vividly and the last quarter of an hour as a sequence of events. The last ten seconds are stored as compressed video, preserving the visual detail needed for fine motion. Anything older becomes a short text summary covering the past 10 to 15 minutes, which records what has been done ("dishes rinsed, counter wiped") at a tiny fraction of the token cost. With this setup, robots finished 10 to 15 minute kitchen clean-ups on their own, including wiping, drying, putting things away and washing dishes (a reported result).
Why it matters: Together with reliability, memory is what separates a 30-second demo from a robot that can do a real job.
3.12 Data and scaling laws
Scaling laws are empirical relationships showing that a model's performance improves predictably, typically as a smooth power-law curve, as training data, computing power and model size increase. In language models they allowed researchers to forecast the benefit of a larger training run before paying for it. The Open X-Embodiment project described in 2.10 found scaling laws for robotics, and they are now starting to appear more clearly, but slowly. Because the curves are smooth only when plotted on logarithmic axes, data must grow by orders of magnitude before a scaling law becomes visible, and each 100x increase in robot data takes about a year to collect. Progress is nonetheless real at the level of individual deployments: in one reported case, the robot data needed for a live factory task fell from more than a month to under 24 hours within a year.
Data is the limit, for four reasons:
- The gap is very large. One estimate puts the data a robot foundation model needs at around 100 trillion examples. The largest robot teleoperation datasets hold around a million, so almost the whole 100 trillion is missing. The figures are one company's estimate, and the conclusion holds even if they are far off.
- More data does not mean more variety. In language, more data brings more variety automatically, because the internet covers so much of human life. In robotics it does not. Trillions of hours of self-driving data are still all driving. A hundred robots in one room learn only about that room, and 100 trillion tokens are useless if they all come from the same environment.
- Variety matters more than volume, and this has been measured. When the most varied data was removed from one training set, performance on new tasks dropped sharply. Removing a random fifth of the less varied data barely changed anything.
- Even unrelated data helps. Adding data unrelated to a task, even data collected in another country, has been reported to improve performance on it, with no sign of the effect levelling off.
Data can be scarce in two distinct ways. In some cases the data does not exist and has to be generated, through teleoperation, simulation or the robot's own practice. In others it is being produced somewhere, for example in the motions of workers or in the logs of existing machines, but nobody has built a way to capture it in a form a model can learn from. Each source has a characteristic weakness. Teleoperation is accurate but slow and labour-intensive. Simulation is cheap and unlimited but suffers from the sim-to-real gap (3.7), which is widest for contact-rich manipulation. Video of people is abundant but contains no motor commands or forces and shows human hands, not robot grippers. Most serious efforts therefore combine several sources.
Why it matters: Data, more than algorithms or hardware, is the main bottleneck, with one caution: even if all the data appeared tomorrow, today's algorithms might not be able to use it.
3.13 Moravec's paradox
Tasks that are easy for people, such as picking up a cup or climbing stairs, are hard for machines, while tasks that are hard for people, such as calculus or chess, are easy for machines. This inversion is known as the paradox in this section's title. A computer beat the world chess champion in the 1990s, yet no robot can reliably pick up the pieces and set up any chess board.
There are two standard explanations:
- Evolution. People are excellent at physical tasks because ancestors who were poor at them did not survive. Abilities refined over that long a period are deeply optimised and run without conscious effort, so tasks that feel effortless in fact rely on a vast amount of hidden processing. Abstract reasoning is recent in evolutionary terms, which is why it feels hard to people and why its explicit, rule-based structure suits computers.
- Interaction with an uncertain world. A backflip is complete control of one's own body in free space. Computers excel wherever everything is known and controlled. Airplanes have flown on autopilot for decades because in the air they touch nothing, and their dynamics can be modelled accurately. The hard part is continuously sensing an uncertain world and correcting for it, such as placing a foot on a stair never seen before or closing a hand around an object of unknown weight and friction.
Contact is the core of the second explanation. When a hand touches an object, the forces change abruptly, friction is hard to predict, and the hand itself hides the object from the cameras at exactly the moment the details matter. These effects are also difficult to simulate accurately, which is why simulation transfers less well to manipulation than to motion in free space.
Two practical observations add to this:
- Walking corrects itself, but each grasp is different. Walking works like a pogo stick. The same motion repeats, recovers on every step and absorbs variations in the ground. Each grasp is a separate problem, beginning with considerable uncertainty about the object's shape, position and weight and ending with the object held, and there are many valid ways to achieve it.
- Legs get more practice signal than hands. A walking robot that tries random movements falls in every possible way, and every fall teaches it something, since the outcome is clear and the dynamics are continuous. A hand touches an object only briefly, so most random attempts produce no contact at all and teach it little. That is why simulation plus RL solved walking before it solved grasping.
The same logic applies to evaluating demos. Dancing, martial arts and backflips replay a planned motion in free space, so they are easier than they look, and so is blind parkour, which relies on proprioception rather than vision. Stairs are the honest test, especially descending stairs while being pushed, or steep fire-escape stairs, because the robot must see and react. A humanoid that cannot climb stairs has little reason to have legs.
How much error a task tolerates is the other test. Folding laundry is relatively easy for learned robots because a shirt still looks fine if a fold is off by a centimetre, and cloth forgives imprecise contact. Putting a wireless earbud into its case with a two-fingered gripper is hard, because there is no room for error and the gripper cannot reorient the part in the final moment, as human fingers do without thought.
Why it matters: The two tests, the need to react and the tolerance for error, are better guides than intuition when choosing which tasks to automate first, since how hard a task looks to a person predicts little.
3.14 Where the bottleneck is moving
Some models improved simply from adding high-level language labels to existing data, with no new demonstrations. The main limit had moved up from the lowest level, whether the robot can physically perform a motion, to the middle level of interpreting the scene and choosing the right next step, a level that language can supervise. A later model confirmed the trend, since more detailed descriptions of the data helped more than a bigger model did (3.8).
The shift changes which failures dominate. Robots in current systems more often fail by choosing the wrong sub-task, forgetting what they have already done or losing track of a long sequence than by being unable to execute a grasp they have chosen. These are problems of decision-making and memory (3.11), and they compound over long tasks: a small chance of a wrong step at each stage becomes a large chance of failure across a task with many stages.
Why it matters: Effort now pays off most in choosing the next step, in memory, and in reliability over long tasks.
Part 4. Data, simulation and world models
Learned robot behaviour is only as good as the data it is trained on, and robot data is scarce, expensive and uneven in quality. This part describes where robot data comes from, what each source is good for, how teams select and label it, and how simulation and world models stretch a limited supply of real experience further. It closes with the problem of measuring whether any of it worked.
4.1 Why robot data is hard to get
Language and vision models were trained on data that already existed: text and images accumulated on the internet over decades. No comparable archive exists for robots. A useful training example for a robot pairs what the robot sensed (camera images, joint angles, forces) with the actions it took, recorded at the rate the controller runs, and almost nobody records such data unless they are building a robot. Teams therefore have to produce or capture robot data themselves (1.1), which requires hardware, trained operators, physical sites, storage and quality control. The result is that data collection in robotics behaves like an operations business with research attached, and its cost is measured in people, robots and hours rather than in downloads.
Two further difficulties are less obvious.
- Finding the right data is a research problem. Whether a given kind of data improves a model usually cannot be determined from a small sample. Effects that are invisible at a few hours of data can become decisive at thousands of hours, and the reverse also happens. Teams therefore learn which data works partly by collecting more of it, which makes every bet on a data source expensive to test.
- The data a model needs changes as it improves. In statistical terms the requirement is non-stationary, meaning that its distribution shifts over time. A model that knows nothing benefits from almost any data, including large amounts of loosely related video, because it is still learning basic regularities such as how objects move and what hands do. A more capable model has already absorbed those regularities, and further improvement comes mainly from data aimed at its specific failures: the grasps it fumbles, the lighting it misreads, the situations it has never seen. A data strategy that was right at the start of a project is therefore rarely right a year later.
4.2 Five sources of robot data, and three tests
Every source of robot data has a weakness, and three tests show where it lies.
- Scalability: whether large volumes can be collected quickly and in many places at once.
- Diversity: whether the data covers many tasks, objects, environments and conditions. Diversity matters because learned policies generalise poorly outside the range of situations they were trained on.
- Closeness to the robot: how closely the data matches the robot's own embodiment, meaning its particular body, sensors, joint angles, forces and timing. The mismatch between the body that produced the data and the body that must use it is called the embodiment gap.
The five sources are as follows.
- Robots practising on their own. A robot attempts tasks autonomously and records the results, often as part of reinforcement learning, in which behaviour is improved by trial and error against a reward signal. The data is complete, since every sensor and motor is logged, and it requires no human operator. Its weaknesses are cost and signal. Every sample occupies a physical robot for real time, and early random attempts at a complex task almost never succeed, so most of the data records failure and teaches little. Large installations of robot arms grasping objects around the clock produced real progress on grasping, then reached a ceiling where additional robot-hours yielded little further improvement.
- Teleoperation. A person controls the robot through the task, and the robot records its own sensors and commands. This gives the highest-quality data, because it is produced by the robot's own body and contains exactly the observations and actions the robot will later use. It is slow and confined to wherever a robot happens to be (4.3).
- Wearable capture. A person wears or holds a gripper or glove fitted with a camera, and sometimes other sensors, and performs the task directly, with no robot present. The data is far more varied and cheaper per hour, but it sits one step further from the robot: the device approximates the robot's hand, and the person's arm and body differ from the robot's (4.4).
- Simulation. Virtual worlds can produce trillions of examples in a day, run faster than real time on many processors in parallel, and expose quantities, such as every contact force, that are hard or impossible to measure on hardware. Their weaknesses are diversity, since building many varied and realistic virtual scenes is laborious, and fidelity, since simulated physics never matches reality exactly (4.8).
- Human video. Video of people doing things, from head-mounted cameras or the open internet, is extremely varied and plentiful. It is also the furthest from the robot's body. It shows a human hand rather than a gripper, it usually lacks precise 3D positions, and it carries no force information. A viewer can see a hand close on an egg but cannot see how hard it squeezes (4.5).
Because the weaknesses fall in different places, serious teams combine several sources and assign each a specific job. One common division of labour is that video teaches what a task is and how scenes and objects behave, simulation makes skills reliable by letting the robot practise under many conditions, and a comparatively small amount of real robot data supplies the precise, body-specific control. Put more simply, human data teaches the robot how the world works, and a little robot data teaches its low-level reflexes. In training terms, the broad sources tend to dominate pre-training, the initial phase in which a model learns general patterns from large datasets, while robot data dominates fine-tuning, the later phase in which the model is adapted to a specific body and task.
Ranked by quality, teleoperation usually comes first, then wearable capture, then simulation and generated data, with head-camera video last. The lower-quality sources clearly help in early training, when the model is still learning general structure. Whether they improve final performance on a specific robot, once enough robot data is available, remains genuinely disputed.
Why it matters: A robotics company's data strategy now defines it as much as its model architecture does. The informative questions about any such company are which data sources it uses, at which stage of training, and why.
4.3 Teleoperation
In teleoperation, a person controls a robot remotely and the robot records what it senses and does. Three interfaces are common. A leader-follower rig gives the operator a second, unpowered or lightly powered copy of the arm (the leader); the operator moves it by hand, and the real robot (the follower) copies its joint angles. A VR headset tracks the operator's head and hands and maps their motion onto the robot, while the robot's camera view is streamed back to the operator. An exoskeleton is worn on the operator's arm or body and measures their joint motion directly. In each case the commanded human motion must be retargeted, meaning mapped from the human's geometry onto the robot's different limb lengths, joint layout and limits, which is simplest with a leader arm that shares the robot's kinematics.
Teleoperation has a practical advantage beyond data: a teleoperated robot can deliver a useful service before it can work autonomously, and every working session produces training data. Most labs have grown their manipulation datasets this way, and the recorded demonstrations are the standard input to imitation learning, in which a model learns to reproduce the actions a human demonstrator took in each situation.
Teleoperation has four limits.
- It cannot produce enough data for a foundation model on its own. Each demonstration takes a few minutes of a person's time and a robot's time. Even millions of operators would need an unrealistic amount of time to approach the volume of data on which language models are trained, so teleoperation is better suited to targeted, high-value data than to bulk pre-training.
- The robot can only be as good as the operator. A model trained purely to imitate human demonstrations learns, at best, the demonstrator's skill, and teleoperated skill is lower than the same person's skill with their own hands because the interface removes touch and adds delay. Some behaviours cannot be teleoperated at all. Balancing and running require reactions within milliseconds to forces the operator cannot feel, so legged locomotion is learned by other means, chiefly simulation (4.8).
- It is slow. People move more slowly and cautiously when controlling a robot remotely, because of latency, a limited camera view and the absence of touch. Robots trained on that data inherit the slowness. Teams either make the data faster, for example by better interfaces or by speeding up recorded trajectories, or train the robot to execute faster than the demonstrations it learned from.
- The work has to happen where the robot is. Each session needs a working robot, power, network and setup on site, so the data comes from only a few places and covers a narrow range of environments.
A further, technical limit is shared with all imitation learning. A policy trained only on successful demonstrations never sees what to do after a small mistake, so when it drifts slightly off the demonstrated path it enters states absent from its training data, makes larger errors, and drifts further. This compounding error, a consequence of covariate shift (the gap between the states seen in training and those met when the policy runs), is why practitioners deliberately collect demonstrations of recovering from errors as well as clean executions.
The debate: Teleoperation may be a temporary way to start or, for some tasks, a permanent business model, though in homes people are wary of a remote operator who can see and hear everything. Some teams use it as the final, high-quality layer on top of video and simulation, others mainly to verify that an arm is physically capable of a task, and few now rely on it alone.
4.4 Wearable capture
A team can dispense with the robot entirely, hand people an inexpensive capture device, and send them out into the world. One effort built thousands of these devices and collected about 500,000 hours of data, mostly of people performing everyday tasks in their own homes and workplaces across many countries (a reported result). The approach derives from an academic design called UMI (Universal Manipulation Interface), a hand-held gripper with a camera mounted on it. The camera records what the gripper sees, and the device's motion through space is reconstructed from the video, typically with visual tracking of the camera's position and orientation, so each recording yields both observations and a trajectory of gripper poses and openings that a robot can learn to reproduce.
The approach takes data collection out into the world instead of bringing the world to a robot. No robot, power supply or other equipment is needed on site, so the data covers far more homes, objects, lighting conditions and clutter than any lab could stage. Because the person holds the same kind of gripper the robot will use, the embodiment gap is narrower than with bare-hand video: the fingers and their contact with objects look the same, and only the arm and body behind the gripper differ. The lessons from that effort apply to anyone building a data operation.
- Design the device for mass use. It should be cheap, easy to manufacture in volume and simple for anyone to use without supervision, because the number of devices in the field sets the rate of collection.
- Make the device match the robot. The robot's grippers are built to look like the capture device, with the camera in the same position relative to the fingers, so that what the model sees in training matches what it sees on the robot. Once the data is varied enough, adapting the model to new types of robot hand also becomes easier.
- Check that the data works before collecting a lot of it. In the first year of that effort, robots were built solely to test whether models trained on the captured data transferred to real hardware, to avoid accumulating half a million hours of useless data. Transfer is the only meaningful test of capture data, and it is cheap to run early and ruinous to discover late.
- Running the operation is most of the work. That meant labour partners who trained their own workers, two different pay models, petabytes of data too large to upload over home internet connections, explaining an unusual job to confused workers, and being open with them that the data trains robots. Quality control also falls into this category: recordings with lost tracking, blocked cameras or incomplete tasks have to be detected and filtered at scale.
Why it matters: Data collection can itself be a company's core capability, and wearable capture is a third route to robot data alongside teleoperation and simulation, trading some closeness to the robot for far greater scale and diversity.
4.5 Learning from human video
There are trillions of videos of people doing things, and robots can learn from them because people are, mechanically, biological robots operating under the same physics. How a task works barely depends on who does it. There are only one or two sensible ways to pick up a cup, and a door opens the same way for everyone. Video therefore carries information about which objects matter, the order of steps, where to grasp and how things move when handled.
Extracting that information requires bridging the embodiment gap. Common techniques include estimating the 3D pose of human hands and bodies from the video and retargeting it onto the robot's joints, predicting how objects or points in the scene will move rather than predicting the human's exact motions, and learning a compact representation of visual change that the robot's own data later grounds in motor commands. Each technique discards something: hand-pose estimates are noisy, and none of them recovers the forces involved.
Watching alone is therefore not enough. If watching were sufficient, everyone who studied a world-class tennis player would play like one. Video shows what the task is, and practice and robot data supply how to perform it with a particular body (4.2).
In reported results, one model trained on head-camera video of people, plus less than an hour of robot data, transferred the skill to humanoid robots with hands. Another model was trained on human video before its robot had even been built, on the expectation that the robot would move like a person. New tasks then required a couple of hours of robot data, and sometimes tens of minutes, such as 45 minutes for chopping with a knife.
The second robot's hardware was designed around this strategy, with body proportions measured against real people and a waist that bends, because a robot that cannot move like the people in the videos can learn less from them (5.5). The closer the robot's kinematics are to a human's, the smaller the retargeting error and the more of the video's information survives the transfer.
Why it matters: Human video is the most plentiful training data in existence, so a robot built to move like a person can draw on far more of it than one with an unusual body.
4.6 Labelling, curation and programming via data
Collecting data is only half the work. Curation is the selection, filtering and weighting of data before training, and labelling (or annotation) is the attachment of descriptive information to each example, such as the task instruction, the step being performed, whether it succeeded and how well. Three lessons make data more useful.
- Label data in detail, including bad data. In one reported case, adding the lowest-quality demonstrations made a model worse, unless each example was labelled with how good and how fast it was. With those labels, the same poor data made the model better. The mechanism is conditioning: when quality is an input to the model during training, the model learns to distinguish good from bad execution instead of averaging the two, and at run time it can be asked for the high-quality behaviour. Poor demonstrations still teach useful things, such as how scenes look and which states occur, so labelling them is often better than discarding them.
- Use the model to pick the useful data. After a hundred hours of collection, much of the data will not help, because it duplicates situations the model already handles or contains errors. Signals from the model itself, such as its uncertainty on an example, its prediction error, or measured changes in performance when a subset is removed, can show which parts matter, and the team can then train on those or weight them more heavily.
- Fix a behaviour with a little targeted data. Teaching a robot to push a hose into a foam pad took about two hours of demonstrations of that one motion and an overnight training run, and the next day it worked. This is programming by data collection instead of by code: the engineer changes the robot's behaviour by deciding which demonstrations to record and adding them to the training mix, then retraining or fine-tuning.
Why it matters: Selecting and labelling data have therefore become core engineering skills, as central to a robot's performance as the model architecture.
4.7 Data flywheel
Self-driving car programmes demonstrated the approach first. A team builds a system useful enough to deploy, puts it to work, and lets it collect data as it operates, instead of trying to calculate in advance how much data it will need. The difficulty is circular: data is needed to build a robot that works on its own, and deployed robots are needed to gather that data. A data flywheel breaks the circle by letting each round of deployment produce the data for a better next version, which can then be deployed more widely and produce more data. Deployment is valuable not only for volume but because it surfaces the rare situations, often called the long tail, that no planned collection anticipates.
The robotics flywheel is harder to start than the self-driving one, for four reasons.
- Robots have to be useful first. Nobody drives a car to help develop self-driving, but people drive anyway, and instrumented cars collect data for free while a human does the driving. Nobody uses a robot that does not yet work, so robotics has to begin from other data until robots are good enough to deploy.
- More data is not automatically more useful. A fleet of a thousand robots moving a bin all day produces a great deal of data that is largely redundant. Useful data comes from deliberately varying the work and from tracking failures: logging each intervention, the moment a human has to step in, and feeding those cases back into training.
- Getting the data out, and getting customers to share it, both take effort. Many warehouses barely have internet connectivity. Customers also will not train a vendor's robot for free, so one answer is to design the product so that each customer sees its own robot improve whenever it shares data.
- Eventually robot data will be enormous. Once robots are useful there will be many of them, and data from deployed robots will exceed every other source by a wide margin. The shortage lasts only until robots are useful enough to deploy in numbers.
Self-driving took 17 years to reach a few thousand driverless cars, partly because cars are so specialised: the system must be nearly perfect at one high-risk task before it can be deployed at all. A child allowed to learn only driving would learn slowly. A general-purpose robot can begin with low-risk tasks, where occasional errors are tolerable, and build up to higher-risk ones, much as people do.
Training thus gains a third stage after pre-training and fine-tuning: learning from deployment data, which keeps accumulating and over time will outweigh everything else.
Why it matters: Each pilot should be planned as the start of a flywheel: it should prove the return on investment, produce the data the next capability needs, and let the customer see its own robot improve. The order of pilots matters, because each one supplies data for the next.
4.8 Simulation and sim-to-real
Simulation means training or testing a robot in a virtual world governed by a physics engine, software that computes how bodies move, collide and exert forces on one another at each small time step. It is used because real-world data is slow, expensive and sometimes dangerous to collect, while a simulator can run many robots in parallel, faster than real time, and reset instantly after a fall. The sim-to-real gap, also called the reality gap, is the difference between the simulator and reality, which means a skill learned in simulation may fail on the real robot. Its main sources are well known: contact and friction are hard to model accurately, motors and gears have delays, backlash and saturation that simulators simplify, rendered images differ from real camera images, and soft or deformable objects such as cloth and cables are expensive to simulate at all. A perfect simulator would amount to having roughly solved the universe, so the gap never fully closes.
Practitioners narrow the gap in two complementary ways. System identification measures the real robot's physical parameters, such as masses, friction coefficients and motor response, and tunes the simulator to match them. Domain randomisation does the opposite: it deliberately varies those parameters, along with lighting, textures, object shapes and sensor noise, across a wide range during training, so the policy learns to succeed under all of them and treats the real world as one more variation. Domain randomisation is the main reason simulated locomotion policies transfer to hardware without real-world training data.
Teams use simulation in three main ways.
- Practice in many conditions. To extend the tennis comparison, in simulation the ball can be made wet, strong wind can be added, and play can continue indefinitely. This is reinforcement learning at scale, and it makes a skill hold up under varied conditions, but it does not teach what the task is: someone still has to specify the task and its reward.
- Multiplying real data. Some teams take real recordings and use generative tools to create many varied copies with different lighting, backgrounds, scenes and conditions, a form of data augmentation that keeps the original actions while varying what the robot sees.
- Building simulations with AI. Frontier AI models can now reconstruct 3D scenes from real footage and write simulation code, which may make simulation worth revisiting for teams that dropped it as not ready in 2024. A large language model has already written the reward function, the rule that scores each simulated attempt, for training that taught a simulated hand to spin a pen. Writing reward functions by hand is notoriously difficult, because a policy will exploit any loophole in the scoring rather than perform the intended behaviour.
The big open debate: Humanoid walking and acrobatics rely heavily on simulation, often with no real-world data at all, while manipulation tasks such as folding and making coffee mostly use real data with large models, though some teams now bring simulation into manipulation too. For a company with limited capital, starting in simulation reduces risk before hardware is built or modified.
4.9 Simulation spectrum and world models
"World model" has at least 20 definitions in current use. It helps to treat them as points on a spectrum of methods for creating a virtual world, ordered from hand-built and explicit to learned and implicit.
- Classical physics simulation. People build 3D worlds by hand, with modelled geometry, materials and physics, much as film studios build computer-generated imagery. This is precise and controllable, and every quantity is available for measurement, but creating varied, realistic scenes is expensive in human effort.
- Gaussian splatting. Camera footage of a real place is converted into a 3D scene represented as a large collection of small, semi-transparent, coloured blobs (3D Gaussians) whose positions, shapes and colours are optimised until rendered views match the footage. A virtual camera can then move through the scene and render photorealistic views from new angles. The scene comes from reality but remains an explicit 3D model, and it captures appearance rather than physics, so it is typically combined with a physics engine for interaction.
- Neural or generative world models. A neural network directly generates video or sensor frames, with no explicit 3D scene, by predicting what comes next from what came before. Some are action-conditioned, meaning they take the robot's intended action as an input and generate the consequences of that action. Their realism comes from training on large amounts of video, and their characteristic failure is physical implausibility: objects that drift, merge or vanish over long rollouts.
A working definition: a world model is a simulation close enough to reality, and responsive enough, that a robot can act in it and the world reacts appropriately. A reactive (closed-loop) world responds to what the robot does, so a policy can be tested or trained inside it. An open-loop world simply replays a recording regardless of the robot's actions, which is useful for evaluating perception but cannot show the consequences of a different decision.
The current role: World models used to run only offline, for development and testing, and some now run inside the robot's live decisions. One recent vision-language-action model (VLA) uses a world model while it works, to picture where the robot should be a second or two ahead, and then chooses actions that move it toward that predicted state. World action models predict future video as part of choosing their actions (3.5). A robot has also been shown imagining training scenarios inside its own model, described as the robot dreaming, to extract more learning from less than an hour of real data; the underlying idea, training a policy on experience generated by a learned model rather than by the real world, is long established in model-based reinforcement learning. Two cautions apply. The imagined images help perhaps less than one might expect, because errors in the model become errors in the training data, and enthusiasm for world models can distract from the underlying constraint, which is still data.
Why it matters: The phrase "world model" should always prompt two questions: which kind is meant, and whether it runs offline to create training data and test the robot, online inside the robot's decisions, or, increasingly, both.
4.10 Evaluation
Robotics has nothing like the fixed, shared test sets of vision and language, where a model can be scored on the same examples by anyone, quickly and cheaply. A robot policy must be evaluated by running the physical robot many times, because its own actions change what it sees next, so a recorded dataset cannot stand in for a trial. Each trial takes real time and often a human to reset the scene, and results vary with lighting, object placement and wear, so many trials are needed before a difference between two models is statistically meaningful. Failures also compound over long tasks: a task made of many steps succeeds only if every step does, so evaluating a 20-minute task is more than 10x harder than evaluating a 2-minute one. The number of robots available caps how fast a team can test, and a team with few robots learns slowly. In academic robotics, teams mostly grade their own homework, choosing their own tasks, setups and success criteria, which makes results hard to compare across papers.
Simulated benchmarks offer repeatability and scale, but policies that perform well in a simulator do not reliably perform well on hardware, for the reasons given in 3.8, so they serve better for rapid iteration than for final judgement. Teams with commercial goals now use tougher tests.
- A three-stage check before deployment. First, set targets for accuracy and time (if one bus bar takes a day, no deployment wants it). Second, work through a list of disruptions, such as a box left in the way or the lights going off, to measure how the robot behaves outside the conditions it was trained in. Third, test safety rules, such as stopping when someone cuts the camera cable, to confirm that faults lead to a safe state.
- Benchmarks based on paid tasks. Use customers' real tasks and parts, so that passing the benchmark means the robot is commercially useful rather than merely competent on a laboratory proxy.
- Throughput, not only success rate. Measures such as boxes per hour combine success and speed in a single figure, and companion measures such as interventions per hour and mean time between failures (MTBF) capture how often a human must step in and how long the robot runs unaided.
- The customer's standard, not the researcher's. A restaurant grades a folded napkin on a scale where acceptable and unacceptable can differ by about an inch in a single fold, a distinction a binary research label of success or failure would miss.
Why it matters: Short tasks are cheaper to test and easier to get working, so they are the right place to start. Testing infrastructure deserves early investment, and progress should be measured against the customer's standard.
4.11 What the benchmarks show
Most robot results are measured on a team's own tasks, so two results are rarely comparable. A shared benchmark runs every model on the same tasks under the same scoring. One of the most complete in 2026 is RoboDojo: 42 manipulation tasks in simulation and 18 on real robots. Instead of one success rate, it scores five separate abilities:
- Generalization: new objects, layouts and conditions.
- Precision: tight-tolerance work such as inserting a key or fastening a screw.
- Long-horizon: many steps in a row, such as clearing and organizing a table.
- Memory: tasks where something seen earlier decides what to do later, such as remembering which cup hides a block.
- Open: instructions the model was not trained on, given in words, numbers or images.
Each ability gets a score out of 100 (partial credit for progress) and a success rate (full completions only). The overall score is the plain average of the five.
By October 2026 the board held 62 entries, and four things stood out:
- Manipulation is far from solved. The best purpose-built robot model averaged about 39 out of 100 and fully completed about a third of attempts.
- Progress is fast. Open robot models from 2024 and 2025 score between 0.2 and 3.5. On a related simulation suite, the best 2026 models average 84% success, against 14 to 31% for 2025 baselines.
- Open instructions are the weak spot. Most robot models score close to zero on the Open ability. They do what they were trained on and little else.
- The top of the board is hybrid. The highest scores, around 55, come from agent systems: a language model plans and checks the work, and learned robot skills carry out the motion.
A September 2026 evaluation tested what happens with no robot model at all. A frontier general-purpose language model, with no robot training, controlled the arms through a simple fixed interface with two commands: move the gripper to a point, or give up. In simulation it scored 29 overall with 22% success, above every purpose-built robot model on the board at that time (the best was 24.9). The previous generation of the same model scored about 1 under the same setup.
Behind that average, the model reached 50% success or better on ten tasks and zero on sixteen. It won where the task is mostly reasoning: reading numbers, following a rule given in words, or working out a geometric arrangement. It lost where the task is mostly contact: inserting, stacking under friction, pouring.
The study concluded that the model now understands what to do and where things are in 3D, but lacks physical commonsense: a working feel for contact, force, friction and how things move once touched. On real robots the gap was obvious. Testing was stopped for safety after the model repeatedly issued unreasonable or unsafe motions, some of which damaged hardware. Of 33 real trials kept for analysis, one fully succeeded.
Two further results from the same study matter for how robots will learn:
- Showing it a demonstration made it worse. Adding an example of the task to the prompt lowered success from 22.9% to 17.9% with images and 12.9% with text. The model already understood the goal, and an example from a different layout misled its precise motion.
- It adapted to changes on the fly. When the camera image was flipped, the motion axes were secretly reversed, or the head camera was removed, the model noticed that its moves did not land where intended and corrected itself within a few attempts. It succeeded in 6 of 8 layouts with a mirrored camera and 4 of 8 with reversed axes.
An October 2026 study gave nine frontier language models (fifteen configurations) direct control of a real two-arm robot on six tasks, with 20 trials per task and a cap of 40 moves per trial. The best models averaged 45 to 48% on a partial-credit score, close to a tie. The scores reflect steady partial progress on each task rather than frequent end-to-end success. They have also been rising quickly:
No single model was best on success, speed and cost at once. A common failure was closing the gripper on nothing: weaker models did it 59 to 64% of the time, stronger ones 32 to 39%. Some tasks stayed out of reach: standing bottles upright succeeded in 6 of 180 trials. The study lists its own limits: one robot, six tasks, a move cap, grading done by a language model watching the video, and trends fitted on few data points.
Why it matters: General-purpose models are getting better at robotics quickly, without any robot data, on the reasoning half of the problem. The contact half, which covers precision, force and dynamics, still depends on robot data and purpose-built models, and that is where most paid physical work sits (11.2). Public boards show the direction of progress, but deployment decisions still rest on the customer's own tasks (4.10).
Part 5. Hardware
Robot hardware has matured to the point where it is adequate for many commercial tasks, yet every major subsystem still carries characteristic weaknesses. This part examines those subsystems in turn: how a robot's body is described, what its joints are made of, why hands remain difficult, which senses it relies on, what shape it should take, where its computation runs, what it takes to certify it, and how it fails once deployed.
5.1 Embodiment and degrees of freedom
Embodiment refers to the specific physical robot a model controls: its geometry, its joints, its sensors and the limits of its motors. A policy trained on one embodiment does not automatically transfer to another, because the same intended motion requires different joint commands on a body with different link lengths or joint layouts.
Degrees of freedom (DoF) are the number of independent ways a joint or a whole system can move. An elbow hinge has 1, a human shoulder has 3, and a human hand has roughly 20 or more. A conventional industrial arm has six, which is the minimum needed to place a tool at any position and in any orientation within its reach; an arm with more joints than the task requires is called redundant, and the spare freedom lets it reach around obstacles or choose a more comfortable posture for the same tool position. One semi-humanoid robot adds four DoF in its waist alone.
Each additional degree of freedom brings dexterity at a price. It adds a motor, a gearbox, a sensor and wiring, all of which cost money and can fail. It also makes control harder: computing the joint angles that produce a desired hand position, the problem of inverse kinematics, has more solutions to choose between as DoF increase, and a learned policy must output a longer action vector and learn how the joints interact. The count is also not fixed for the life of the machine. When a joint fails in the field, a 32-DoF robot becomes a 30-DoF robot, and its software must either compensate or stop (3.10).
5.2 Actuators and component cost
An actuator is the robot's equivalent of a muscle and joint combined. It integrates an electric motor, a transmission (usually gears), one or more sensors and the control electronics that convert electrical power into precise motion. The bill of materials (BOM) lists the cost of every physical part in a product. Actuators make up 30 to 50 percent of a humanoid's BOM, and their basic design has barely changed in 50 years.
The central design choice in an actuator is the transmission. Electric motors spin fast with little torque, while joints need to move slowly with a lot of torque, so a gearbox trades speed for torque by its gear reduction (the ratio of motor turns to output turns). The common families are these:
- A harmonic drive (also called strain-wave gearing) uses a flexible toothed ring deformed by an elliptical cam inside a rigid outer ring. It achieves a high reduction in a compact, light package with almost no backlash (the free play between meshing teeth), which is why it is common in precise robot arms. Its drawbacks are limited tolerance to shock loads and high internal friction.
- A cycloidal drive uses a lobed disc that rolls eccentrically inside a ring of pins. It tolerates impacts better than a harmonic drive and also offers low backlash, at the cost of more weight and some vibration.
- A quasi-direct-drive (QDD) actuator pairs a large-diameter, high-torque motor with a low reduction, often a single planetary stage. Because there is little gearing between motor and load, the joint is backdrivable: an external force on the output can turn the motor, so the joint yields when it hits something. The motor current is then a reasonable estimate of joint torque, which allows force control without a separate torque sensor. Legged robots favour QDD for exactly this reason, since impacts with the ground are constant.
The trade-off runs along a single axis. High reduction gives precision and strength in a small motor but makes the joint stiff, frictional and hard to backdrive, so contact forces are hard to sense and collisions are hard to absorb. Low reduction gives compliance and good force sensing but needs a heavier motor and runs warmer. Designs that need both often add a dedicated joint torque sensor at the output, or a deliberate spring in series with the gearbox (a series elastic actuator), whose measured deflection gives the force directly.
Actuator choice also determines how faithfully the robot can be modelled and simulated. Friction, backlash and gear compliance are difficult to capture in a physics engine, so a simulation of a stiff, high-reduction joint may diverge from reality in exactly the contact-rich moments that matter. For this reason some teams design their own actuators specifically to be easy to model. That reasoning suggests a general rule for building versus buying. Buy any component with clear inputs and outputs and mature suppliers, such as lidar developed for self-driving cars or many cameras. Build any component that determines how well the whole system performs, because the hardware ultimately carries the intelligence: a model can only exploit the precision, compliance and sensing the body provides.
Why it matters: A component that accounts for up to half the cost of a humanoid and has barely changed in decades is a direct drag on margins, and for the same reason it is one of the clearest openings for investment and new design.
5.3 Hands and dexterity
Hands are the hardest part of robot hardware to build. A dexterous hand must pack many small, precise actuators, transmissions and sensors into a volume the size of a human hand, survive years of repeated loading and impact, and still perform delicate work. Few engineers have the combination of mechanical, electrical and manufacturing skills needed to build one.
Designers handle the space problem in a few standard ways. Some place motors in the forearm and route force to the fingers through tendons (cables running over pulleys), as the human hand does with forearm muscles; this keeps the fingers light but introduces cable stretch, friction and wear. Others put small motors directly in the fingers, which simplifies the mechanics but adds mass and heat at the fingertips. Many hands are underactuated, meaning they have fewer motors than joints and use linkages or springs so that the fingers curl and conform around an object passively. Underactuation trades individual joint control for simplicity and gentle adaptive grasps.
Human grasps remain the benchmark and are still only partly understood. They are strong yet compliant, stable under disturbance, and continuously adjusted using touch, in ways that current science cannot fully explain or reproduce. Heat is a practical limit as well: today's best dexterous robot hands overheat under sustained load, because small motors held at high torque dissipate heat in a tiny volume, and engineers have cooled them with bags of ice. Some teams therefore use simple parallel-jaw or suction grippers, because no available dexterous hand survives the duty cycle of a factory, and a hand that did would find buyers immediately.
The debate: One view holds that hands are an excuse for why general-purpose robots have not arrived, and that the intelligence controlling them is always the real limit; the other holds that dexterity is the most valuable problem as well as the hardest, because solving it unlocks an enormous range of tasks. Both can be true: hands are not yet durable enough for many deployments, and the model remains the main limit on what robots can do.
5.4 Sensors
Robots perceive the world through two broad classes of sensor. Proprioceptive sensors measure the robot's own state: joint encoders report angles, motor currents indicate effort, and an inertial measurement unit (IMU) combines accelerometers and gyroscopes to track orientation and acceleration. Exteroceptive sensors measure the environment: cameras, depth sensors, lidar, microphones, force/torque sensors (usually mounted at the wrist, measuring the forces and twisting moments the hand experiences) and tactile sensors (skins or fingertip pads that report pressure, contact location and sometimes slip).
One early research platform used three cameras, one on each wrist and one at the base, with no touch or force sensors at all. The choice was deliberate, based on the hypothesis that a good learning method can compensate for deficient sensing. Wrist cameras partly substitute for touch: when the gripper presses on an object, the camera sees it deform, slide or tilt, and a learned model can infer from those visual cues how hard it is squeezing.
Cheaper rigs push the same idea further. An omelette-making setup costing about $4,000, using inexpensive arms with a camera as the only sensor, cracks and whisks eggs after training on less than 10 hours of data, judging force from vision alone (a reported result). Results like this suggest that better intelligence could extract far more from existing sensors than current robots do.
The limits of vision are equally well understood. Cameras suffer occlusion (the hand itself blocks the view of the contact it is making), struggle with transparent and reflective objects, and degrade in poor lighting. Touch fills those gaps: it reports contact directly and early, before an object has visibly moved, which is when a slip can still be corrected. Tactile sensing has been slow to spread because skins are fragile, need dense wiring, and lack a standard data format that learning methods can easily consume.
The debate: Relying on cameras alone is a bet. Humans also hear, feel and smell, integrating all of it without noticing, and video is highly redundant, since neighbouring pixels and consecutive frames are nearly identical, while touch may carry far more task-relevant information per byte; by one estimate, a model using vision plus touch could be 100x more data efficient. Some teams already add infrared sensors, for example to judge how well food is cooked, and the practical course is to start with few sensors while watching for tasks where a missing sense caps performance.
5.5 Form factors: wheels, legs and humanoids
A robot's form factor is its overall body plan. Humanoids are not the right shape for every robot, and the choice involves trade-offs in stability, energy, cost and the range of tasks the body can reach:
- Wheels make sense for now. A wheeled base is statically stable: it stays upright with the motors switched off, because its centre of mass sits over its support. A legged robot is dynamically stable at best, meaning it stays up only while its controller keeps acting, so a software fault, a power loss or an unexpected push can bring it down. For that reason a first home robot should have wheels and two arms, so that it cannot fall and injure a pet or a child. Legs also consume more energy, since they must support the body's weight continuously even when standing still. Demonstrations in which a robot recovers from being pushed show good balance control, which says little about readiness for useful work.
- A human-like upper body on a wheeled base. One semi-humanoid robot mounts a human-shaped upper body on wheels, with a waist that has four joints. People bend, twist and lean constantly without noticing, a fact that becomes obvious only when a stiff neck takes the movement away, and human video is full of that motion. A flexible waist lets the robot reproduce postures seen in human demonstrations and extends the reach of its arms, from the floor to a high shelf, without legs.
- Humanoids are the likely long-term shape, reached in stages. Homes, factories and tools are built around human bodies, so a general-purpose robot that must use them will probably converge on a humanoid form; a human-like body also makes human video and teleoperation data easier to map onto the robot. Hardware decisions are slow and expensive to reverse, however, so some teams deploy a simpler body with customers now instead of spending years perfecting the final design.
- Fleets will mix robot types. No single robot type is likely to dominate. Fixed arms will stay in factories, four-legged robots will handle inspection on rough terrain, humanoids will take some jobs designed around people, and wheeled carts will move goods, with a single model potentially controlling all of them. Models trained across many embodiments are an active research direction precisely because of this mix.
- Take risk in only one place. One approach is to buy commercial robot arms, already shown to be capable enough through teleoperation, and build only the grippers and data-capture devices, so that all the engineering risk sits in the part that actually differentiates the product.
Why it matters: The right body is the simplest one that can perform the valuable task; every extra joint, leg or sensor adds cost and failure points that must be justified by work the robot could not otherwise do.
5.6 Onboard vs. offboard compute
Onboard compute means the model runs on processors inside the robot. Offboard compute means it runs in a remote data centre (the cloud) or a nearby server and exchanges observations and commands with the robot over a network. The choice affects cost, power consumption, latency and safety, and it is one of the most actively debated questions in the field.
Any approach must respect a layered timing structure. The lowest layer, the motor control loop that holds each joint at its commanded position or torque, runs on dedicated electronics at hundreds of hertz to a kilohertz and is always local. The question is where the higher layers, the learned policy and any reasoning model, should run. The main positions are these:
- Cloud with action chunking. The model predicts a short sequence of future actions at once, a technique called action chunking. It begins computing the next sequence while the robot is still executing the current one, called real-time chunking, and the new chunk is blended in smoothly when it arrives. Network delay is thereby hidden inside the buffer of actions the robot already holds, as long as the delay is shorter than the chunk. This avoids an expensive, power-hungry computer on the robot, and the robot's hardware does not become obsolete as models grow larger. Teams using this approach run it in production for laundry, coffee making and navigation.
- On the robot. Other teams, along with safety-critical driving systems, keep the model on the robot. A falling robot has no time to consult a server, and a robot holding a knife cannot freeze for three seconds without alarming the people around it. Networks cannot guarantee timing, because packet loss, congestion and outages are unpredictable, so these teams train a large model and then distill it: they train a smaller model to imitate the large model's outputs, producing a policy compact enough to run on the robot's own processors, usually with some loss of capability.
- Split by speed. Some teams run fast reflexes, operating on a timescale of milliseconds, on the robot, and run slower reasoning about what the robot sees in the cloud, about once a second. The slow layer chooses goals and sub-tasks; the fast layer turns them into motion and reacts to contact. This mirrors the general hierarchical design of robot control, in which each layer runs at a rate matched to how quickly its decisions go stale.
- Connectivity sets a ceiling. Many warehouses barely have internet access, with dead zones among metal racking, which limits how much any robot there can rely on the cloud.
Resolution: Decide by how sensitive the task is to delay, how much safety depends on the response, and how reliable the site's connection is. Fast or safety-critical control belongs on the robot, and slower planning can run elsewhere.
5.7 Redundancy, validation, and homologation
Removing the human operator completely, for example from a truck, requires hardware as well as software. The vehicle needs fully redundant steering and braking, meaning duplicate, independent systems so that the failure of one leaves the other able to stop or steer the vehicle safely, and each system must be independently tested and approved. This hardware is often not mass-produced, and getting it approved can take longer than building the software. Homologation is the regulatory approval a vehicle or component needs before it can be sold in a given market.
The same logic applies to any robot working near people, under the discipline of functional safety: the engineering practice of ensuring that a system responds correctly and predictably when something goes wrong. Its core method is to identify hazards, estimate their severity and likelihood, and assign each safety function a required level of reliability. Safety functions are typically implemented in certified, often duplicated hardware and simple software that sit beside the learned model and can override it. Common examples include an emergency stop wired directly to the motor drives, safe torque off (a hardware-enforced cutoff of motor power), limits on joint speed and force near people, and monitoring that slows or stops the robot when a person enters its workspace. A learned policy is not certified as safe in its own right; the system is made safe by bounding what the policy is permitted to do.
Validation is the evidence that the system meets those requirements: test campaigns, fault-injection trials in which components are deliberately disabled, and documentation that an independent assessor can audit. Because a neural network's behaviour cannot be exhaustively verified, this evidence leans on the surrounding safety layer and on extensive statistical testing.
Why it matters: For any pilot near people, safety engineering and hardware approval, more than the model, will usually consume most of the schedule, and visibly treating that work as a priority builds credibility with partners and investors.
5.8 Real-world failure modes
Deployed hardware fails in mundane, physical ways. Cameras and lidar overheat in enclosed spaces, sensors drift out of calibration as mounts loosen or temperatures change, so that a camera's estimate of where an object sits slowly diverges from where it actually is, and fog, dust, glare or steam confuse perception. Actuators wear: gear teeth develop backlash, bearings loosen, and motors run hotter as friction rises, which changes the dynamics a policy learned on a fresh machine.
Robots also lack a shared operating system of the kind phones and browsers rely on. Each platform has its own drivers, communication protocols and safety interfaces, so a company that runs on many kinds of machine must build and maintain that software layer itself. One autonomy software company runs on more than 50 platforms.
Problems can sit deep in the system and present themselves far from their cause. A model may appear to fail because of a training setting when the real fault is in the hardware: in one case the fingers left a small gap when commanded to close fully, because a connector had been attached incorrectly, and the policy's dropped objects looked like a learning problem. Diagnosing such faults requires tracing an observed behaviour back through software, firmware, wiring and mechanics. The most valuable trait in robotics engineers is therefore attention to detail across every level of the system, from tightening gripper screws to training models on thousands of GPUs.
Supply chains add a further layer of risk. The permanent magnets in electric motors depend on rare-earth materials sourced from very few countries, which exposes every actuator to concentrated geopolitical and price risk, and moving hardware production back to the US is much harder than it appears, because the specialised suppliers, tooling and manufacturing expertise have accumulated elsewhere over decades.
Part 6. From demo to reliable product
A robot that works most of the time makes a good video. A business needs one that works nearly every time, and closing that gap is the hardest problem in the field today. This part explains why the gap exists, and the methods practitioners use to narrow it: teaching robots to recover from their own errors, training models that measure progress, making reinforcement learning cheap enough to run on physical hardware, directing human effort at the cases where the robot is weakest, extending autonomy from single tasks to multi-hour workflows, and raising speed.
6.1 The gap between demos and reliability
Producing robot behaviour that looks impressive is no longer the main difficulty. Washing a pan, peeling vegetables and making a grilled cheese sandwich are all achievable with current learned models. What remains difficult is turning such a capability into a system that delivers value to someone who depends on it, shift after shift, without an engineer standing by.
The distinction is between success rate, the fraction of individual attempts at a task that end in success, and reliability, the probability that a system completes a long run of work without failing. A demonstration shows the first. A deployment depends on the second. General robot models typically succeed 80 to 90 percent of the time per attempt in real deployments, and because each attempt is a separate opportunity to fail, those probabilities multiply. At 90 percent per attempt, the chance of ten consecutive attempts without a failure is 90 percent multiplied by itself ten times, about 35 percent. At 80 percent it is about 11 percent. To complete a hundred actions in a row with reasonable confidence, the per-attempt success rate must be well above 99 percent, and 99 percent is a long way from 99.99 percent: each additional "nine" of reliability requires eliminating most of the failures that remained at the previous level, and those remaining failures are usually the rarest and strangest cases, which are the hardest to collect data for.
The multiplication assumes that attempts fail independently, which is a simplification. In practice, failures are often correlated: a robot that struggles with one unusual object tends to struggle with every object like it, and a change in lighting can depress performance for an entire shift. Correlation can make real-world reliability either better or worse than the simple calculation suggests, which is why practitioners measure it directly over long runs instead of extrapolating from short tests. Common operational metrics include mean time between failures (MTBF), the average duration a system runs before a failure occurs, and interventions per hour (or its inverse, mean time between interventions), which counts how often a human has to step in to correct, reset or rescue the robot. These numbers capture what a customer experiences far better than a per-attempt success rate does.
Classic specialist machines, such as a fixed industrial arm executing a hard-coded trajectory inside a guarded cell, can be extremely reliable because their environment is engineered to remove variation: parts arrive in the same place, in the same orientation, every time. The cost of that reliability is inflexibility. Reprogramming the machine for a new task, or tolerating a part that arrives slightly out of position, typically requires new fixtures and new engineering. Some startups aim to remove this trade-off by building learned models that can perform many tasks and are also reliable enough to sell.
Why it matters: When evaluating a demo, ask how long it ran, how many attempts were made, and how many failures were edited out. Robot videos typically show only the successful cases, so they report a best-case success rate and say nothing about reliability.
6.2 Recovering from mistakes
The compounding calculation in 6.1 assumes that one failed attempt ends the task. That assumption is often wrong, and the reason it is wrong is the most important lever on reliability. People are not especially accurate either. A warehouse worker makes far more than one wrong pick in a million, yet still finishes almost every order, because after a mispick the worker notices and picks again. Completion rate depends on per-attempt accuracy and on the ability to detect and correct errors, and the second factor can dominate. Robot models are beginning to show the same pattern: recovering from mistakes raises the task completion rate well above what per-attempt accuracy alone would predict.
A standard model trained purely by imitation shows the opposite behaviour. It performs well until it makes a small mistake, then typically goes out of distribution, meaning it faces a situation unlike anything in its training data, and from there it gets stuck or makes things worse. The mechanism is well understood. In imitation learning, also called behaviour cloning, a policy is trained to copy the actions in human demonstrations, and those demonstrations almost always show an expert doing the task correctly. The model therefore never sees what to do after an error. Once a small deviation occurs, the robot is in a state the demonstrations did not cover, its next action is less accurate, which pushes it further from familiar states, and the errors accumulate. This is called compounding error, and the mismatch between the states seen in training and the states the robot actually visits is called covariate shift. The standard remedy, associated with an algorithm known as dataset aggregation, is to let the robot run its own policy, record the states it drifts into, and have a human supply the correct action in those states, so that the training data comes to include the situations the robot itself creates.
In one reported case, the fix was to teach the robot explicitly how to recover, and that skill turned out to generalise beyond the specific recoveries it was shown. After enough training rounds, the robot knocked over a whole stack of napkins, separated them, and carried on with the task, even though it had never been shown how to handle that particular accident.
This broader ability can be called improvisational intelligence: producing sensible behaviour in situations nobody planned for. In older robotics, every behaviour was a hand-built controller, often a state machine with explicit branches for each anticipated condition, and it worked only in the situations its designers had foreseen. An unanticipated condition fell through every branch and the system halted or did something wrong. A learned model, shown something new, will usually do something reasonable, because it has learned general regularities about objects and actions instead of a list of cases. It does not always succeed, but it fails gracefully more often, and graceful failure is what makes recovery possible.
6.3 Progress meters: reward models and value functions
Once a robot succeeds around 90 percent of the time, paying people to watch every run in order to catch the rare failure becomes wasteful, and human attention becomes the bottleneck on further improvement. The solution, which two groups arrived at independently, is to train a separate model that watches the robot and scores how well it is doing. Such a model turns hours of unlabelled robot footage into a signal that says where things went right and where they went wrong.
- A reward model is a learned model that watches video of the robot and outputs a score from 0, meaning the task has just started, to 1, meaning it is done. When things go well, the score rises steadily. When the robot makes a mistake, the score drops, and those drops point directly to the moments worth examining and fixing. Because the score is computed automatically, it can be run over every episode the fleet records instead of a sample a person has time to review.
- A value function is the standard reinforcement learning term for a model that estimates how good a given situation is in terms of what will happen next, here expressed as how much time is left until the task succeeds. It learns, for example, that accidentally unfolding a shirt moves the task backward and that recovering moves it forward. The value function is typically trained on the outcomes of many recorded attempts: each moment is labelled with how the attempt actually turned out from that point, and the model learns to predict that outcome from the current observation. One general value function can work across many tasks, from folding clothes to fetching something from a fridge, because the underlying notion of "closer to done" transfers.
The value function is the more powerful of the two because it supports a precise judgement about individual actions. If the estimated time remaining falls after an action, the action helped. If it rises, the action hurt. The difference between what an action achieved and what the value function expected is called the advantage, and it is the quantity most reinforcement learning methods use to decide which behaviours to strengthen and which to suppress. Both kinds of model share a known weakness: a learned scorer can be wrong in ways the policy learns to exploit, scoring an attempt highly because it superficially resembles success. Practitioners guard against this by checking the scorer against human judgement on held-out episodes and by retraining it as the robot's behaviour changes.
Why it matters: Progress meters replace continuous human supervision with automatic scoring, which makes it affordable to keep improving robots after deployment.
6.4 Making real-world RL affordable
Reinforcement learning (RL) is the family of methods in which an agent improves by trying actions, observing the results, and adjusting its policy toward actions that led to better outcomes. Labs improve language models with RL over millions of attempts, which is cheap because each attempt is only computation. For robots, every attempt happens in physical time on physical hardware. One million attempts at a one-minute task takes about 700 robot-days, and the hardware wears, objects must be reset between attempts, and some failures damage things. Language-model RL also tries each prompt 10 to 50 times simply to estimate which answers are better than average. On real hardware, that sampling is far too slow.
The property at stake is sample efficiency: how much improvement a method extracts per attempt. Two broad strategies raise it. Offline RL learns from a fixed dataset of previously recorded experience, including other policies' attempts and human demonstrations, without collecting anything new. Online RL collects fresh experience with the current policy and learns from it as it goes, which corrects the policy's actual mistakes but costs robot time. Practical robot systems usually combine the two, extracting as much as possible from existing data and spending real-world attempts only where they add information. One process makes this practical:
- Train a foundation model on varied data.
- Deploy it, with a person stepping in to stop it from wasting time on dead ends. For example, if the robot picks up two cardboard boxes stuck together and keeps trying to fold them as one, a person slides a finger between them and lets the robot carry on. These interventions are cheap, keep each attempt productive, and themselves become training data showing the right correction at the moment it was needed.
- Train a general value function on that experience.
- Use the value function to improve the robot's policy, meaning the way it decides what to do, by favouring actions the value function judges to have moved the task forward and disfavouring those that set it back.
- Repeat.
Each cycle produces a better policy, whose deployment produces better data, which trains a better value function. Reported results include roughly double the throughput from the RL stage alone, over 90 percent success at making espresso, a latte service that ran for 13 hours straight, and a real box-assembly job at a customer's factory, a job the workers there disliked. These results are still not 99.99 percent reliable, and the robots are still slower than people.
One unusual theory addresses why RL works at all in very large models. A research paper found that in such models, random changes to the weights often still produce good results, so RL may work partly because it adds random noise that explores nearby behaviours, not because each update does anything especially smart. This remains a hypothesis about mechanism and does not change the practical recipe above.
6.5 Human-in-the-loop active learning
A third route to reliability sits between collecting more demonstrations and letting the robot practise alone. The robot runs on the real task, the reward model shows where it struggles, people record demonstrations of how to recover in exactly those situations, the model is retrained, and the cycle repeats. The model's own weaknesses decide what data to collect next, which is what active learning means: instead of gathering data uniformly, the learner directs collection toward the examples it is most uncertain about or most likely to get wrong. Uniform collection spends most of its effort on situations the model already handles, while targeted collection spends it on the failure modes that limit reliability.
On restaurant napkin folding, this raised success from about 80 percent with standard training to 99.4 percent over a 24-hour run (a reported result), and the robot was not thrown off by morning light changing through the windows. Gradual lighting change is a classic source of covariate shift for vision-based policies, so holding up through it indicates that the corrective data covered real variation in the environment. The process resembles reinforcement learning from human feedback (RLHF), the method used to tune chatbots, in which people rate a model's outputs and those ratings are used to train it. The robot tries, people grade the attempts, and both the grades and the new demonstrations train the model.
Why it matters: For a company deploying robots today, this is the most practical way to improve reliability, because it runs on the customer's actual task and spends human effort only on the cases that are failing.
6.6 Workflows that run for hours
Most robot autonomy is still measured in tasks of 30 seconds to a minute, while the commercial value lies in workflows that run unsupervised for hours. A long workflow is a sequence of many tasks, so the compounding arithmetic of 5.1 applies with full force, and it adds problems a single task does not have: deciding what to do next, waiting on other machines, and remembering what has already been done.
Getting there takes more than a better low-level model:
- A layer that sequences skills. One design wraps the action model in a higher-level reasoning system that decides which skill to use and when, an arrangement often called hierarchical control, with a planner on top issuing goals and a policy underneath executing them. While the washer runs, the robot does something else. When the washer finishes, it moves the towels to the dryer right away so the washer is not left waiting. It is the same step language models took from autocomplete to assistants that finish whole jobs. This layer is called an agentic harness. One laundry demo built this way runs a full cycle of about an hour in a real hotel laundry room, including noticing a dropped towel and putting it in the dirty basket (a reported result).
- Memory for jobs with many steps (3.11), so the system can track which items are finished, which machine holds what, and what was interrupted, instead of relying only on what the cameras show at the current moment.
- Treating it as a whole-system problem. Long jobs are a system problem as much as a modelling one. They need a capable reasoning layer, careful engineering around it (monitoring, error detection, safe fallback behaviour when something goes wrong, and a clear way to hand control to a person), a good model, and hardware that keeps working over hours of continuous use without overheating or drifting out of calibration.
6.7 Speed
Robots are still slower than people, partly because teleoperated demonstrations are slow. A person controlling a robot remotely sees through cameras, lacks a full sense of touch, and moves cautiously, and a policy trained by imitation reproduces that caution, including the pauses. RL helps remove such hesitation. A person may pause to line up a cable plug, for instance, but a policy optimised for completing the task quickly can learn that the pause is unnecessary and skip it. One later model ran faster than human teleoperation, and one napkin-folding robot reached human speed without slowing from fatigue (both reported results).
The usual measure is throughput, the number of units of work completed per hour, which combines speed and reliability: a fast robot that needs frequent resets can have lower throughput than a slower one that never stops.
Why it matters: Customers pay for output per hour. A reliable robot that works at half a person's speed often does not pay for itself.
Part 7. Deployment and go-to-market
Robots reach customers one site at a time. Before they can work entirely alone, they earn revenue through mixed autonomy, in which a human corrects the robot when it fails, and every deployment generates data that makes the next one easier. This part treats deployment as an engineering discipline in its own right and then examines the business logic that follows from it: where to deploy, how to stage adoption, what the unit economics must look like, and where lasting advantage comes from.
7.1 Deployment is a technical problem
Software reaches millions of users through a download. A robot reaches one customer at a time, by truck, and must then be installed, commissioned, tested for safety at that particular site and integrated with the people and equipment already there. Every site differs in lighting, floor surfaces, clutter, network coverage, object variety and the habits of the people who work there, so each deployment exposes failure modes that never appeared in the lab. A grasp that succeeds reliably on a test bench may fail under the glare of a skylight or on a slightly different batch of packaging. Much of the real engineering work of a robotics company is therefore finding, diagnosing and fixing these site-specific failures, and the people who do it (field engineers, remote operators, integration staff) are as central to the product as the model.
This is why the strongest teams treat deployment as a priority from the first day rather than as a final step after the research is done. A data flywheel is the loop in which deployed robots collect experience, that experience improves the model, and the improved model justifies more deployments that collect more experience. The loop is slow to start: robots must be built, shipped, installed and trusted before they produce useful data, and the data pipeline (logging, upload, labelling, training, validation, rollout) must exist before any of it can be used. A team that waits until its model is "ready" before deploying starts this loop late and never recovers the lost time.
Estimates of the timeline differ. One view holds that robots will be technically capable within a few years and will then spread gradually, because manufacturing, installation, certification and customer adoption all move at physical rather than digital speed. A faster estimate puts the change at a year or two, on the reasoning that once people see robots working around them, trust builds quickly and demand accelerates. The two views agree that capability alone does not determine the pace; the rate at which robots can be built, deployed and accepted does.
7.2 Levels of autonomy and mixed autonomy
Self-driving cars have a formal taxonomy of autonomy levels, and three of its terms carry over usefully to robots:
- L2++ describes highly capable driver assistance in which a human remains legally and practically responsible, such as supervised driving-assist systems that steer, brake and change lanes while the driver watches and is ready to intervene.
- Driverless operation, or true autonomy, means no human is in control at any moment and the onboard computer handles everything, including the rare and dangerous situations.
- Mixed autonomy means the robot works on its own most of the time, but a person takes over and corrects it when it makes a mistake. This intermediate stage is what lets a deployment earn money before full autonomy exists.
In practice, mixed autonomy usually takes the form of teleoperation, in which a human controls the robot remotely through a camera feed and an input device, or lighter remote assistance, in which the human does not drive the robot directly but answers a question such as which object to pick or which path to take. The commercial value of the arrangement depends on the ratio of robots to operators. If every robot needs a dedicated human watching it, the robot saves little labour; as reliability improves, one operator can supervise a growing fleet and intervene only when called, and the economics shift accordingly.
A disengagement (in robotics often called an intervention) is an event in which a person has to take back control. Miles per disengagement is the standard self-driving metric, and robotics uses analogous measures such as interventions per hour of operation or the number of tasks completed per intervention. These metrics matter because they convert reliability directly into staffing cost and because their trend over time shows whether the system is actually learning. They have known weaknesses: the number depends heavily on how hard the operating conditions were and on how cautious the operators are, so two figures are comparable only when measured under similar conditions and intervention policies.
The data a deployed robot generates follows the same progression described in 2.7. It begins with human demonstrations, which teach the basic skill. Next come human corrections gathered during mixed autonomy, which are exactly the data that pushes a model to high performance: each one is recorded in a state the robot itself reached and shows how to recover from a mistake the robot itself made, which is the information pure demonstrations lack. Finally the robot learns from its own experience working alone, with successes and failures labelled by outcome.
7.3 Where to deploy first
The usual order is businesses first and homes last. Several distinct arguments point the same way.
- Start with the most structured settings. A common sequence begins with structured factories and warehouses, where layouts are fixed, objects are known and the space can be adapted to the robot. Next come partly structured service settings such as hospitals, grocery stores, hotels and airports, which have predictable routines but also members of the public moving through them. Homes, the least structured environment, come last. Each stage supplies data and operating experience for the next.
- Start where people are used to new technology. Computers went first to professionals and companies and entered homes once people had become familiar with them at work. A robot in the home would feel strange to someone who had never seen a robot anywhere else, while a person who has worked beside one in a warehouse brings some trust with them.
- Homes are hard for three reasons. The home is the most chaotic environment, with endless variation in layout, objects and clutter. The safety bar is the highest, since a robot can never be allowed to fall on a baby, and households contain children, pets and elderly people who cannot be trained or expected to follow procedures. And households pay less than a business whose robot works all day and whose value can be measured against a wage.
- Businesses are easier customers. They pay more, are more patient with early limitations and let the company improve the robot on site, often with a human operator in the loop. A consumer tries a robot for two days and gives up.
- Look for variability. Learned AI is only needed where there is randomness: where the objects, positions or conditions change from one attempt to the next. A task that is identical every time is better served by conventional, preprogrammed automation, which is cheaper and more predictable. Current targets for learned robots include delivery, inspection and security, warehouse and factory work alongside people rather than inside safety cages, and data centers.
Laundry is an example of how location determines value. A laundry-folding robot is possible this year, but few people want a machine that sits in a corner of the home and does only that. The same skill installed in a commercial laundry or factory running every day would have buyers waiting, because there it replaces a measurable, continuous cost. The capability is identical; the business case depends entirely on where the robot works.
7.4 ODD and geofencing
The operational design domain (ODD) is the explicit specification of the conditions a system is designed to work in. It can include geography (this campus, these four buildings), environment (indoor lighting only, dry floors, this kind of weather), the objects the robot will encounter, the speeds it will move at and the presence or absence of people. Geofencing means restricting operation to a defined, mapped area, so that the robot stops or refuses to proceed when it reaches the boundary.
Defining the ODD is a core step for both safety and commercial scoping. For safety, it bounds the hazard analysis: engineers can only assess the risks of conditions they have specified, and a system operating outside its ODD has no validated safety case. For scoping, it makes the contract with the customer concrete, stating what the robot will and will not do, and it gives the company a clear test of whether a new site is within reach of the current system. Expansion then proceeds by deliberately widening the ODD, one condition at a time, with validation at each step.
The way a system perceives its world determines how fast its ODD can grow. Systems that depend on detailed maps made in advance can only expand as fast as new maps are made and kept up to date, since a moved shelf or a renovated corridor can invalidate them. End-to-end systems that interpret the scene directly from sensor data without detailed prior maps do not have that limit, although they still need validation in each new kind of environment before they can be trusted there.
7.5 Vertical-robotics recipe
A vertical robotics company is one that builds a complete solution for a specific industry or task rather than a general-purpose platform for others to adapt. A workable recipe for such a business today runs as follows:
1. Learn the customer's current workflow in depth, including the informal steps no one writes down. The robot has to fit the workflow, not the other way round, because a customer will rarely reorganize a working operation around an unproven machine.
2. Find the point in the workflow where a robot adds the most value: usually the step that is most repetitive, most physically demanding, hardest to staff or most often a bottleneck for everything downstream.
3. Keep hardware and data costs low. Modern models that react continuously to what they see can make up for cheaper, less precise hardware, because a closed-loop policy that observes its own errors can correct a position that an open-loop, preprogrammed robot would miss.
4. Deploy with mixed autonomy, so the robot earns revenue while its remaining failures are covered by human operators and turned into training data.
5. Make each robot at least break even.
6. Only then increase the number of robots.
Two further points refine the recipe:
- Choose a first task that teaches something and pays. Restaurant napkin folding is one example. It is hard enough to push research, because the robot has to take a single napkin from a stack (separating thin, deformable layers that cling together), handle endless fabric shapes and meet a strict quality standard, and customers asked for it. Deformable objects like fabric are especially demanding because their shape cannot be described by a few fixed parameters and changes with every touch. Each deployment then works as a test of the general system rather than as a separate business, and the lessons transfer to the next task.
- Deploy one task at a time. A language model had to be good at nearly everything before it was useful, because a weak one produces fluent nonsense that no one can rely on. A robot company is in a different position: it can deploy the few skills it has mastered while it keeps building the general model behind them. That is how a company can build a general model while selling into a narrow market, using revenue and field data from each narrow task to fund and train the broader capability.
7.6 Staged adoption
Business deployments usually progress from a proof of concept, which shows the robot can do the task at all, often in controlled conditions, to a pilot, which runs the robot in the customer's real operation for a limited period against agreed success criteria, and finally to full deployment. Each stage should have measurable exit criteria agreed in advance (throughput, quality, uptime, intervention rate), because pilots without them tend to drift on indefinitely without a decision, a pattern often called pilot purgatory. The hardware used in the pilot must go straight into production without being ripped out and replaced; a pilot built on bespoke or prototype equipment proves little about the system the customer would actually buy and doubles the integration effort.
Standardize a small, repeatable set of uses before customizing anything, because heavy customization at the start is what kills adoption. Every custom feature adds engineering work that does not transfer to the next customer, and a company that says yes to every request ends up running a consulting business with robots attached. Adoption also depends on change management: the work of preparing the people at the site for the new system, including training staff to work beside the robot, adjusting procedures, assigning responsibility for clearing faults and addressing workers' concerns about their jobs. Deployments with sound technology still fail when the people around them were never brought along.
The hard milestone is a customer believing the robot creates enough value to deploy it across many of its sites, which takes far more than one location or a pilot. One restaurant customer folds thousands of napkins a day across many locations (a reported result). A shared model helps at this stage. Every site runs the same model, so the work is consistent from one location to the next, and all robots receive improvements at the same time through over-the-air updates, software delivered over the network without a technician visit. Well-run fleets stage these updates, releasing a new model first to a few robots and expanding only after its performance is confirmed, because a regression shipped to every site at once can halt the customer's operations everywhere.
The ideal is a robot that starts working as soon as it arrives at a site, with no new data collected there, a property called zero-shot deployment. A version of this has been shown by a robot folding T-shirts for three days at a robotics conference while visitors deliberately got in the way, an environment the system had not been trained in and could not have been prepared for.
7.7 Unit economics
Unit economics describes the revenue and cost attached to a single deployed robot, independent of the company's overall overhead.
- Payback period is how long a deployed robot takes to earn back its cost, calculated as the upfront investment divided by the net savings or revenue it generates per period. Industrial buyers typically judge automation projects by this figure and reject those whose payback is too long. In the past, this is what killed robotics companies as they tried to grow: the numbers never worked.
- Total cost of ownership (TCO) is the full cost of a robot over its working life, which includes the purchase price, integration and installation, maintenance and spare parts, energy, software, downtime and the human supervision needed in mixed autonomy. A low sticker price can hide a high TCO, and customers increasingly compare robots to labour on this basis. Some companies sell robots as a subscription (often called robots as a service), which turns the upfront cost into an operating expense and shifts maintenance risk to the vendor.
- Break-even per unit is the test to pass before scaling up. A company that loses money on every robot cannot grow its way out of the loss, since each new deployment deepens it.
- Integration cost was the historical barrier. Programming and integrating a robot for each site often cost more than the robot itself, and a caged work cell can cost several times as much (1.3), once fencing, fixtures, part feeders, safety interlocks and specialist programming are included. Robots that adapt on their own remove most of that cost, and that is what makes wider adoption possible. Robots working on existing factory lines next to people, without cages, demonstrate the point, since any place with people in it is messy and cannot be engineered to the precision a preprogrammed robot needs.
A robot costing about $50,000 works out to roughly $2 an hour over its life, compared with $35 to $40 an hour for fully loaded US labour, meaning wages plus benefits, payroll taxes and other employment costs. That is competitive even with offshore labour. The advantage compounds with utilization: a person works one shift, while a robot can work several, so one robot can do the work of about three people. The hourly figure covers only the hardware, however, and the comparison holds only if integration, maintenance, downtime and supervision stay small relative to it, which is why reducing intervention rates and integration effort matters as much as reducing hardware cost.
7.8 Labor, safety and demand
The public debate about jobs usually concerns robots taking them. The opposite pressure, a shortage of workers, is arguably stronger today. Labour figures vary by country, sector and source and change quickly, so they should be verified before being quoted.
- Many jobs go unfilled. Large numbers of open jobs exist alongside unemployment, because the work is dangerous, physically punishing, poorly located or hard to keep staffed, and turnover in such roles is high. The gap will likely grow as populations age and fewer young workers enter manual trades. Demand for robots then arrives on its own, because companies already cannot hire for these roles.
- Safety risks are both sudden and long-term. Some work puts lives at risk through falls, crushing or exposure to hazardous materials. Much more wears bodies down slowly through musculoskeletal injury, for example from bending in one position all day or repeating the same lifting motion many times a shift. Robots that take over these tasks reduce injury costs as well as labour costs. The robots themselves must also be safe: industrial and collaborative robots are designed and installed under international safety standards that require a documented risk assessment for each application, and robots sharing space with people rely on measures such as speed and separation monitoring and limits on contact force and power.
- The social question is real. If robots become very capable, society has to ensure the benefits are broadly shared rather than held by a few, through policy, retraining and the way the gains are distributed.
Start where customers have a painful, urgent labour shortage, because those customers adopt fastest: the alternative to the robot is not a cheaper worker but an unfilled shift.
7.9 Defensibility
Where a robotics company's lasting advantage, or moat, comes from is one of the field's clearest disagreements.
- The deployer's view. If a few labs build general models and share or license them, models become a commodity, interchangeable and priced near cost. A company's advantage then comes from knowing customer workflows, choosing where robots add the most value, owning the data from its deployments, running the mixed-autonomy loop well and meeting the physical requirements of each site. The practical consequence is architectural: keep the model and the robot separated by a clean interface, so that a better model from any source can be swapped in without rebuilding the product.
- The brain-maker's view. In language, the training data is largely public, so a result can be copied within months. In robotics the data is not public; it has to be collected by physical machines in the world. The best model wins the most deployments, which produce the most data, which makes the model better still, a self-reinforcing loop that competitors cannot shortcut by downloading anything. On this view the brain is very hard to commoditize, while robot bodies are becoming commodities, the way phones did once most of them ran the same operating system.
- What both sides agree on. Advantages in technique are temporary, because methods spread through publications and staff movement, and even model builders count deployments as the lasting advantage in robotics. Both camps agree that deployment data is what accumulates over time. They disagree about who will end up owning it.
Why it matters: If models become commodities, the winning position is to own the workflow and the customer relationship; if the model's advantage keeps growing, a company should partner with the leading model maker or build its own. In either case, deployments are the main asset.
Part 8. Markets and capital
8.1 Horizontal vs. vertical
Industries organise themselves along two broad lines. A vertically integrated company owns most or all of the layers that make up a product within one area: a car company that designs its own driverless system, writes its own software and assembles its own vehicles is an example. A horizontal company does the opposite. It specialises in one layer, such as a chip, an operating system or a piece of autonomy software, and sells that layer to many builders who assemble the rest. The two models make money differently. A vertical company earns its margin on the finished product and controls its whole experience; a horizontal company earns a smaller slice from each customer but across a much larger base, and its strength grows with the number of products that depend on it.
One autonomy software company deliberately models itself on a chip supplier. Rather than building the finished machine, it aims to be engineered into many manufacturers' products, where, once validated and certified, it becomes expensive and slow to remove. In the chip industry, securing a place in a customer's product design in this way is called a design-win. A design-win is valuable because the customer's engineering, testing and supply planning are built around the chosen part, so switching to a rival would mean repeating that work. The supplier then earns revenue for the life of the product, often for years.
Robotics will probably split into layers the way computing did. In the 1950s and 1960s, a business bought a room-sized computer from one manufacturer, whose technicians installed it, maintained it and often wrote its software. Humanoid companies operate much like this today, supplying the machine, the software and the on-site support as a single package. Computing later separated into chips, operating systems and applications, each supplied by different firms, and software companies stopped making their own processors. Separation of this kind happens when the interfaces between layers become stable and well defined, so that a part made by one firm reliably works with parts made by others. The robotics equivalent would be a wide variety of robot bodies built on a shared layer of robot intelligence. One possible form is a single dominant robot model used by many hardware makers, in the way many phone manufacturers share one operating system, with chips as the one layer that ends up concentrated among a few suppliers.
There is nevertheless a strong case for going vertical early. Buying each component from a specialist supplier looks sensible when every sourcing decision is judged on its own, but the decisions add up to strategic exposure. Suppliers can move up the value chain: a chassis maker that sells to robot companies can start selling complete robots of its own, turning its former customers into resellers of a competitor's product. A long chain of suppliers is also slow to change, because every design revision must be negotiated across several firms with their own schedules and priorities, and this matters when the product changes every few months. Each interface between suppliers is, in addition, an opening for a newcomer to insert a better part and capture margin. The electric-car industry followed this pattern. Chinese carmakers that built most of their parts in-house went from marginal players to dominant ones, and one leading maker produces about 75% of its parts itself, which lets it redesign a vehicle and its components together and cut cost at every step. Robotics shows early signs of the same dynamic: about 87% of humanoids shipped in 2025 were Chinese, and one leading humanoid maker produces 86% of its components in-house (figures from the article cited in Sources).
Integration does not have to be permanent. Industries tend to integrate while the product is changing quickly, because the cost of coordinating across company boundaries is highest when designs are still moving, and to specialise once the product settles and interfaces can be fixed in standards. Owning the stack also does not mean making every part. Commodity components such as fasteners, bearings or standard connectors can still be bought (2.2), while the elements that determine how fast the product improves, namely the robot's design, its software and model, and its deployment, stay in-house. The aim is control over the rate of improvement of the whole system, not self-sufficiency for its own sake.
8.2 Who owns the customer
The company that holds the customer relationship decides, to a large extent, what the business can do and what it will want to do. The customer determines the requirements, the sales cycle, the regulatory obligations and the kind of evidence needed to close a deal. A company selling to the military must meet procurement rules, security clearances and long qualification programmes, and it becomes organised around them; a company selling to consumers lives on brand, price, distribution and support at scale. Over time the two become very different organisations, even if they started with similar technology.
Owning the customer is not always the best position. A new technology company often lacks the brand, service network, financing arrangements and trust that an established manufacturer has built over decades. In those cases it can be better to let the incumbent keep its brand and its customer relationship while the newcomer supplies the technology underneath. One autonomy company, for example, runs self-driving trucks under another manufacturer's brand: the manufacturer sells and services the vehicle, and the autonomy company provides the driving system. The trade-off is that the supplier gives up some margin and direct customer insight in exchange for faster distribution and lower selling costs.
8.3 Market size for general robot models
Language models offer a useful comparison. In 2016 it would have been hard to estimate the market for general language models, because AI was then sold as narrow products, one for translation, another for speech recognition, another for spam filtering, each trained for a single task. Once a single general model could perform many such tasks, the addressable market turned out to be most of digital knowledge work, wherever text is read, written or analysed.
The equivalent market for general robot models is physical labour. Today's robots cost six or seven figures partly because each one is engineered and programmed for a single job: integrators design custom fixtures, write task-specific motion programs and tune the system on site, and that engineering is a large share of the total cost. When the task changes, much of that work must be redone. General software that runs on many kinds of hardware, and that can be taught a new task with demonstrations rather than reprogramming, could reduce this integration cost sharply and make automation viable for jobs that currently do not justify it. This is a projection, and it depends on the technology working reliably in real settings.
8.4 Timing
Timing errors are punished in both directions. A company that arrives two years early typically runs out of money before the market exists; one that arrives two years late finds the market already crowded. Being early earns nothing, because there is no consolation prize for having been right first. One driverless-car company that was technically ahead of many rivals was shut down anyway after a serious public incident led regulators to withdraw its permission to operate, which showed that technical lead alone does not secure survival.
Enabling technologies can also exist long before they become useful. Wearable capture, in which a person wears sensors or cameras that record their movements while performing a task, was known to work by 2018. The data it produced had no good use, however, until training techniques developed for language models, such as large transformer architectures able to learn from varied demonstrations, matured in 2024. A company built around wearable capture in 2018 would have had the right data source and no way to turn it into a capable robot.
8.5 Why small teams can compete
In language modelling, the training data already existed in the form of the public internet, so the main constraint was computing power, and whoever spent most on computing tended to win. Robotics is more open. Nobody yet holds a large body of robot interaction data, because such data must be physically collected, through teleoperation, wearable capture or deployed robots, at real cost and real time. A large budget therefore cannot buy a head start in the way it could with internet text.
Large companies still have real advantages: more money, more experienced staff, more computing capacity, and the ability to collect data quickly once they commit to a direction. A small team's opening is focus. By going deep on specific paid work, it can collect data from deployments it already runs, which yields examples drawn from exactly the conditions its robots must handle, along with revenue and customer feedback. This creates a data flywheel, in which deployments generate data, data improves the model, and a better model wins more deployments. Competition should be expected to arrive fast. Once any company shows early results in a niche, others are capable of following quickly.
8.6 Value capture and geopolitics
- The US is unlikely to out-manufacture China, but it can lead in the AI that controls robots. Most of the value is expected to sit in that layer, because a robot body without capable control software can perform little useful work, while the same software can run across many bodies and improves with every deployment.
- Leading in manufacturing does not guarantee capturing the value. China's crowded and heavily subsidised markets drive prices down until profit margins are thin, and the rapid movement of engineers and suppliers between competing firms spreads technical know-how, making it hard for any single company to keep an advantage.
- Chinese manufacturers lead on cost and on iteration speed, helped by dense local supplier networks that can produce new parts within days. US companies typically compete on software and on integration with business customers, including the ability to fit robots into existing enterprise systems, workflows and compliance requirements.
- Sovereign AI refers to the desire of countries to have physical AI built and controlled within their own borders, because it touches defence, critical infrastructure and sensitive data. Restrictions on foreign robots for government-funded buyers are likely, because robots carry cameras and microphones that can see and hear everything around them and contain communications hardware that could transmit what they record. A US company with a supply chain outside China gains an advantage under these policies, since it can sell to buyers who are barred from using components of certain origins.
8.7 Capital structures
Robotics companies are typically financed in stages. Early rounds, often called seed and Series A, fund the team and the first prototypes on the strength of the founders and the idea; later rounds fund manufacturing, deployment and scaling on the strength of measured traction. Each round sells a share of the company at a price, and existing holders are diluted, meaning their percentage ownership falls as new shares are issued. Because robotics is capital-intensive (hardware must be designed, built, tested and deployed before revenue scales), founders and investors in the field encounter a wider range of financing structures than in pure software.
- SPV (special purpose vehicle): a legal entity created to pool several investors so that they appear as a single line on a company's cap table (the record of who owns which shares). SPVs are often used to gain access to a popular private funding round, where the company wants few direct shareholders. A deal offered through several stacked SPVs, one investing in another, can be a real opportunity, but it can also be a warning sign. Each layer typically charges its own fees and share of profits, which erode returns, and deals that reach investors through long chains are sometimes those that better-informed investors passed on. Information rights and the ability to verify what the SPV actually holds also weaken with each layer.
- Closed-end fund: a fund that issues a fixed number of shares, which then trade on a stock exchange like those of a company. Investors who want out sell their shares to other investors rather than redeeming them from the fund. This is why, unlike an ETF (exchange-traded fund), which must be able to create and redeem shares continuously and so holds mostly liquid securities, a closed-end fund can hold large stakes in illiquid private companies. A publicly traded closed-end fund generally cannot charge carry, the share of investment profits that private fund managers usually take.
- NAV (net asset value), premium, and discount: NAV is the value of what a fund holds, divided across its shares. Because the shares trade on an exchange, their market price is set by supply and demand and can sit above the NAV (a premium) or below it (a discount). Premiums arise when investors value access to assets they cannot buy elsewhere; discounts arise when investors doubt the reported valuations or the managers.
- Premium crystallization: if a fund trades above its NAV, issuing new shares at the market price raises the value backing each existing share instead of simply diluting it, because each new share brings in more assets than its proportionate claim. For example, take $100 of assets trading at 3x NAV and issue 10 percent new shares. The new buyers pay three times the NAV of what they receive, so the fund grows to $130, and value per share rises about 18 percent after dilution. Some acquisition-driven companies and companies holding treasury assets have grown value this way. The mechanism works only while the premium lasts; if the market price falls to or below NAV, issuing shares no longer adds value per share.
- Why it matters: Robotics will need tens of billions of dollars over time, more than private venture capital alone can supply, so choices about which investors, which structures, public markets and government money deserve attention early. Funding is also concentrating, with the largest reported rounds for robot-model companies now in the billions.
8.8 Jevons paradox
Jevons paradox is the observation that when a resource becomes cheaper or more efficient to use, total consumption of it often rises rather than falls. It was first noticed with coal: more efficient steam engines used less coal per unit of work, which might have been expected to reduce coal demand, but the lower cost of power made steam engines worthwhile in many more industries, and total coal consumption grew. The effect occurs when demand is highly sensitive to price, so that a fall in cost unlocks enough new uses to outweigh the savings on existing ones.
Applied to robotics, cheaper robot labour will do more than substitute for existing labour spending. It will also expand the market by making it economic to automate tasks that were never worth automating at previous costs, such as low-volume, highly varied or intermittent work for which a custom-engineered robot could never pay back its integration cost. The size of the eventual market therefore cannot be estimated simply from current spending on the labour that robots would replace.
Part 9. Compute and supply chain
Chips and supply chains set the floor on the cost and risk of everything else a robotics company does. Inference hardware is a recurring cost on every robot, training compute is a large fixed cost on every model, and the availability of motors, magnets and sensors determines whether robots can be built at all. A founder therefore needs the basics even without being a chip analyst.
- Training and inference. Training is the process of building a model: adjusting its parameters over very large datasets, which requires large clusters of chips running for days to months. It is a large cost, incurred once per model version. Inference is the process of running a trained model to produce outputs, such as the next motor commands for a robot. Each inference step is cheap, but it is repeated continuously for as long as the model is in use. A fleet of deployed robots runs inference all day, so inference cost scales with the number of robots and hours worked, while training cost scales with the number of models built. The two workloads stress hardware differently: training needs enormous memory bandwidth and fast links between many chips, while inference on a robot needs low latency (the delay between an observation and the resulting action) within tight power and heat limits. Chips are increasingly specialised along the same line, with some designed for training and others for inference.
- GPUs, ASICs, and stranded investment. GPUs (graphics processing units) are general-purpose parallel processors that have become the standard chips for AI. One maker dominates the market, largely because of its software ecosystem: the programming tools and optimised libraries that developers already know make switching to another vendor costly even when competing hardware is comparable. ASICs (application-specific integrated circuits) are custom chips designed for one workload. Because they omit circuitry that the target workload does not need, they cost less per unit of work and use less power, but they carry a specific risk: if model architectures change, the chip may no longer run the new models efficiently, and the money spent designing and buying it is stranded. The same reasoning is one argument for keeping expensive computing off the robot, where it is fixed for the robot's lifetime, and placing it in servers that can be upgraded or repurposed, at the cost of depending on a network connection.
- Distillation. Distillation is a technique in which a large, capable model (the teacher) is trained first, and a smaller model (the student) is then trained to reproduce the teacher's outputs. The student learns from the teacher's full output distributions or actions rather than only from the original labels, which carries more information per example and lets a much smaller model retain much of the teacher's capability. The student is fast and cheap enough to run on the robot. Robotics is likely to rely on distillation widely, because a robot's control loop must produce actions within strict time limits that a very large model cannot meet on onboard hardware.
- Where the gains come from. Most of the improvement in intelligence per watt in recent years came from better models and algorithms, such as more efficient architectures, better training methods and lower-precision arithmetic, not from hardware alone, though one hardware generation did deliver about 30x. Algorithmic progress spreads quickly through publications and open models, so a focused startup is better placed to build on other companies' model progress than to try to outspend them on it.
- Not yet a bubble. Demand for valuable AI work is still growing faster than computing capacity can be built, which is why computing prices are rising rather than falling. A bubble would show the opposite pattern: capacity outrunning paying demand, and prices falling as a result.
- Co-design. Co-design means designing hardware, models and system software together, so that each is shaped around the others. Gains then multiply rather than add, because an improvement in one layer can unlock improvements in the others that would not be possible in isolation. A model designed to suit a chip's memory layout, for example, may allow a compiler optimisation that would otherwise be impossible. Three 2x improvements designed together can approach 100x instead of 8x. This does not conflict with building a product that can switch models (Part 10), because the two operate at different layers: co-design happens within the model and inference stack, while model-agnosticism is a property of the product architecture above it.
- Supply chain. Actuators, the permanent magnets inside their motors, and many sensors depend on production concentrated in a few countries, with rare-earth magnet materials an especially concentrated case. Moving production back home is slower and harder than it looks (5.8), because it requires not only factories but also the surrounding network of material processors, specialised suppliers and experienced workers that took decades to build elsewhere.
Why it matters: Two of these factors, inference cost per robot and dependence on Chinese supply chains, show up directly in a robotics company's margins and risk.
Part 10. What the robotics market rewards
Lessons from AI-native software and robotics companies apply directly to how a robotics company should be run. The common thread is that robotics combines the iteration speed of machine learning with the unforgiving physical reality of hardware, and the practices below are ways of keeping both honest.
- Measure results, not effort. "Be customer obsessed" describes an effort, and a team can practise it diligently while still producing a poor outcome. "Create obsessed customers" describes a result, and it predicts whether customers renew, expand and recommend the product. Goals stated as results can be checked; goals stated as effort cannot.
- Track the company with data, not gut feeling. No one trains a model on intuition, yet many companies set budgets and priorities that way. The same discipline that governs model development should govern the company: each new capability should be checked against a real metric, such as task success rate, interventions per hour or customer usage, to confirm that it actually moved something.
- Stay model-agnostic. A model-agnostic product is built so that the underlying model can be replaced without rewriting the rest of the system, usually by keeping a clean interface between the model and the surrounding software. This protects the company technically, since the best available model changes frequently, and business customers trust it because it reduces their dependence on any single model provider.
- Do not trust public benchmarks. A benchmark is a standard test used to compare models. Any public benchmark that matters commercially gets gamed ("benchmaxed") within months, as developers tune models to the specific test, sometimes with test examples leaking into training data, until scores rise without matching gains in real capability. Internal benchmarks built from real tasks, ideally paid ones, are harder to game because they reflect the conditions the robot will actually face.
- Do research and deployment together. Research shows what is technically possible and therefore what can be sold; deployments show which problems actually matter to customers. Without deployments, research priorities are guesses, and a team can build something impressive that nobody wants to buy. The history of robotics includes many companies that built very shiny, very cool robots and failed for lack of a paying use.
- Design tasks to test capabilities. Effective evaluation starts from the capability required, such as following spoken instructions or fine dexterity, and then designs a concrete task that exercises it. Fetching a named drink from a fridge, for example, tests language understanding, object recognition among similar items, navigation and manipulation of a door and a container in one sequence, and failures can be traced to the capability that broke.
- Expect a large hardware team. At one robotics startup, 30 to 40 percent of staff work on hardware and only a small group works on models. Hardware absorbs effort because every design change must be built, tested and maintained physically. Data collection is also a skill in its own right: people with no robotics background can become highly skilled teleoperators and data collectors, and the quality of their demonstrations directly shapes the quality of the trained model.
- Hire for attention to detail and curiosity about the whole system. Robots fail at the boundaries between subsystems, where a mechanical tolerance, a software timing issue and a model's behaviour interact. The strongest candidates combine deep respect for colleagues' expertise with a strong desire to understand everything else. Mindset matters more than credentials, and the hard part of hiring is designing a process in which candidates can demonstrate it.
- Use the open models. Capable robot foundation models, meaning large models pretrained on broad robot data and adaptable to many tasks, are now available as open source. Small teams should start from such a general model and fine-tune it (continue training it on their own task data) from the first day, rather than building a separate model for each site. A shared base model lets data from every deployment improve performance everywhere.
- Hire people committed to the mission before things get hard. Resilience cannot be screened for once a crisis has started, because by then the team is already fixed. Commitment has to be assessed at hiring, when the cost of a wrong choice is lowest.
- Watch market timing. Being too early costs as much as being wrong (8.4), so the readiness of the market, in customer budgets, regulatory acceptance and the maturity of supporting technology, needs to be rechecked continuously.
Part 11. Strategy while models, compute and hardware get cheaper
Three inputs to robotics are falling in price at once. Models grow more capable with each release, computation becomes cheaper per operation with each hardware generation, and the physical parts of a robot, from arms, motors and cameras up to complete humanoids, are becoming commodities: interchangeable goods sold mainly on price, where no single supplier can charge much more than another. In parallel, the largest companies in the world are entering the field. Frontier AI labs, chip makers, large social media and search companies, and car and humanoid makers building their own robots all now treat robotics as a market worth owning.
For a smaller company this raises two strategic questions. The first is what it can own that keeps its value while everything beneath it gets cheaper. The second is how it can keep growing for five to ten years without being crushed by a competitor with far greater resources. The answers below draw on the evidence assembled in earlier parts. They are an interpretation of that evidence, and reasonable people disagree with parts of it.
11.1 What each trend makes cheap
- Models. One general model now matches its maker's fine-tuned specialists with no extra training (a reported result). A fine-tuned specialist is a copy of a model given additional training on one narrow task; when a single general model performs as well without that step, the specialist's extra engineering stops being an advantage. Strong robot foundation models, large models pretrained on broad robot and internet data and then adapted to particular robots and tasks, are already released as open source. Techniques also spread quickly: once a group publishes early results, the method, the architecture and often the training recipe can be reproduced, and others catch up within months. Any lead built on technique alone is therefore temporary. General robot intelligence appears to be following the path general language intelligence took, improving every year and becoming available to everyone.
- Compute. Each new generation of accelerator chips performs more operations per unit of cost and energy, and each algorithmic improvement (more efficient architectures, lower-precision arithmetic, better training schedules) reduces the computation a given capability needs. Together these lower the cost of running a model of fixed capability year after year. Distillation, training a small "student" model to imitate the outputs of a large "teacher", makes it practical to train a large model in a data centre and then run a compact version on the robot's onboard computer, where power, heat and latency limits are strict. Total demand for computation is still rising faster than supply, so the price of the most advanced capacity stays high, but the cost of any particular level of capability keeps falling.
- Components. Fourteen manufacturers sell arms under $10,000, a dozen humanoids are available to buy or lease, and leader-follower rigs cost under $2,000 (RCSV figures). A leader-follower rig is a teleoperation setup in which a person moves a lightweight "leader" arm by hand and a powered "follower" arm copies its joint positions, which is the standard way to record demonstration data cheaply. Chinese manufacturing, with its dense supplier networks for motors, gearboxes, sensors and electronics, pushes these prices down further.
The general pattern is that anything a large company can buy, download or train by spending more money will become cheaper for everyone. The model, the computing power and the robot body all fall into this category, and a company whose only asset is one of them has no lasting advantage, because a better-funded competitor can match it by writing a larger cheque. What keeps its value is whatever takes time in the physical world: experience accumulated at real sites, relationships built over years, data that exists only where robots have actually worked. None of this can be downloaded, and a larger budget does not make it happen faster.
11.2 What does not commoditize
Six assets hold their value as the layers beneath them get cheaper:
- Data from real customer sites. This is the only data that matches the exact conditions the robot will face: the real lighting, the real clutter, the real variation in parts and packaging, the real mistakes people make. Models trained on data from other settings suffer from distribution shift, a drop in performance when the inputs at deployment differ statistically from the inputs seen in training, and site data is the most direct remedy. Nobody can scrape it from the internet, and as fleets grow it will likely outgrow every other source (4.7). Even those who argue that the model itself is the lasting advantage (7.9) agree that deployment data is what would make it so.
- Knowledge of customer workflows. This means knowing where in a customer's process a robot helps most, what success looks like in measurable terms (the napkin grade, the throughput target), and what the unusual cases are: the damaged carton, the part that arrives upside down, the shift change when the line stops. Such knowledge is rarely written down. It is learned on site, one customer at a time, and it determines whether a technically capable robot actually earns its keep.
- Trust and a track record. Industrial buyers judge suppliers on safety records, uptime history and how problems were handled when they occurred. Above all, a customer choosing to roll out across many of its sites requires far more than a successful pilot (7.6), because a pilot proves that the robot can work while a multi-site rollout requires confidence that it will keep working, be supported and be maintained. A newcomer cannot buy that history, however much capital it raises.
- Being built into customers' operations. A horizontal supplier, one that sells a component or capability used across many industries and many other companies' machines, tends to stay in place once integrated, because removing it means redesigning, retesting and requalifying the systems that depend on it. Economists call this a switching cost. The same logic applies to a team that knows how to deploy, support and update hundreds of robots in the field, since that capability is organisational and takes years to rebuild elsewhere.
- Data operations. A data collection pipeline includes the capture devices, the labour partners who perform demonstrations, the logistics of moving equipment and people to sites, quality control on the recordings, and the consent and legal arrangements that make the data usable. Building one can take a whole first year. The result is hard to copy quickly because each piece depends on relationships and procedures, not on money alone.
- How fast the company learns. One useful definition of intelligence is the speed at which a new skill is acquired, and the same definition applies to companies. A team that can turn a new customer task into a reliable deployment in days, by collecting the right few hours of data (4.6) and knowing exactly how to fine-tune and validate on it, will grow faster than a team that needs months for the same step. This speed comes from accumulated tooling, process and experience, and it improves with every task completed.
Why it matters: Each of these assets becomes more valuable as models, computing and hardware improve, because better and cheaper tools let a company deploy more robots faster, and every deployment adds to the assets.
11.3 Choosing a layer that gains when others improve
Robotics is splitting into layers, much as computing did after the mainframe era (8.1), when a single vendor supplying the whole machine gave way to separate markets for chips, operating systems and applications. The emerging robotics layers are chips, simulation and development tools, the general model, the robot body, and the application layer, meaning the customer's actual workflow and the robot's job within it. Large companies are concentrating on the lower layers. Chip makers sell computing power and simulation tools. Frontier labs and large technology companies build general models. Car and humanoid makers build complete robots, often using their own factories as the first customer.
The strongest position for a smaller company is a layer that improves every time the layers below it improve. Economists describe such layers as complements: goods whose value rises when the price of a related good falls. When a lab releases a better model, an application company that can switch models (Part 10) immediately has a better product without having paid for the research. When hardware gets cheaper, each robot earns a better return on its cost. When computing gets cheaper, the cost of serving each customer falls and margins grow. Every advance by a large company then helps the smaller company instead of threatening it. Software companies prospered for decades in exactly this way while chip makers competed fiercely with one another, because every faster chip made existing software more capable and new software possible.
The weakest position is competing directly with a large company at the layer it is paid to improve. A company whose only advantage is a slightly better general model is racing labs with far more computing power, more researchers and more data, and its lead will last only until their next release. A company whose only advantage is a slightly cheaper arm is competing with manufacturers whose scale gives them lower unit costs, better supplier terms and the ability to absorb losses longer.
The debate: The opposing case holds that the model layer itself will accumulate an advantage, because the best model wins the most deployments and therefore gathers the most data (7.9), and the shared intelligence layer predicted in 7.1 is a bet on the same outcome. If that case is right, a few well-funded labs will dominate the model layer, and for everyone else the advice holds either way: be the best partner to the leading model makers, and own the deployments they need.
11.4 Loops that compound
A loop compounds when each round makes the next round easier or cheaper, so that progress accelerates over time instead of growing in a straight line. Five such loops matter in robotics, and the strongest companies run several at once:
1. The data loop. Deploy robots, collect data on the customer's real task, use it to improve the robot there, and win more sites on the strength of the result. Each new site contributes different objects, layouts and conditions, and that variety improves the model's ability to generalise, which helps every other site (3.12). The loop is slow to start, since the first sites must be won with little data, and fast once running.
2. The customer-visible loop. Customers will not train a supplier's robot for free, and they are often reluctant to share operational data at all. They do share it when they can see their own robot improve after each round of data (4.7), because the exchange is then direct: their data buys them a better robot. Making that improvement visible, through measured success rates and throughput on their own task, is part of the product.
3. The learning-speed loop. As the base model and the company's own data improve, each new task needs less fresh data to reach reliable performance, because more of the required skill is already present in the model and fine-tuning only has to supply what is specific to the task. In one reported case, a factory task went from needing more than a month of data to under a day. Cheaper new tasks mean more tasks per customer, which produces more data and further shortens the next task.
4. The cost loop. Historically the real barrier to automation was not the price of the robot but the cost of integrating it: custom fixtures, programming every motion by hand, and engineering time at each site (7.7). General models lower that setup cost. Cheaper deployment makes it worthwhile to serve customers who were previously too small to justify the integration effort, which enlarges the market. This is an instance of the Jevons paradox, the observation that making a resource cheaper to use often increases total consumption of it instead of reducing it (8.8).
5. The fleet loop. When every robot runs one shared model, an improvement can be pushed over the network to every robot at every site at the same time (7.6), in the way software updates reach phones. A failure case found and fixed at one customer upgrades all of them, so the fleet as a whole learns from the experience of each unit. The loop requires disciplined testing before each release, since a faulty update reaches every site just as quickly.
Why it matters: A company running these loops becomes harder to catch every month, however large the competitor, because a new entrant starts with no deployments, no site data and no track record, and money cannot compress the time needed to acquire them.
11.5 Staying relevant for five to ten years
Nine rules for a smaller company that intends still to matter in a decade:
1. Do not compete on what large companies are paid to improve. General models, computing power and standard parts will get better and cheaper whether or not a small company invests in them. The sensible course is to use them and to build so that each can be replaced when a better one appears.
2. Own the last step to the customer. That means the customer's specific task, site, data, quality standard and uptime commitment. This is where general capability turns into revenue, and it is slow, local, physical work: site surveys, integration, operator training, maintenance visits. Large companies typically do this poorly at scale because it does not reward the standardisation their organisations are built around.
3. Take risk in only one place. Buy everything that is already proven good enough, and concentrate engineering risk on the part that creates the value (5.5). A project that is simultaneously inventing new hardware, a new model and a new market multiplies its chances of failure.
4. Keep the model and the robot separate. A clean interface between the two, for example a defined set of observations going into the model and a defined action format coming out, lets a company adopt the best available model and the cheapest suitable body as each improves, instead of being locked into one choice. In software engineering this is called modularity: components that interact only through well-specified interfaces can be changed independently.
5. Deploy one task at a time. A perfect general robot is not a prerequisite for a business. Deploy the skills that have been mastered, earn revenue from them, and start the flywheel turning (7.5); each deployed task funds and informs the next.
6. Measure progress on paid tasks. A metric tied to work customers pay for keeps the team from optimising for demonstrations (6.1), which can be staged, filmed after many attempts, or performed under conditions that never occur on a real site.
7. Use the weaknesses of large companies. Physical deployment is slow even for the largest companies (7.1), because it depends on site access, safety approval and customer schedules that capital cannot accelerate. Operations do not scale like software, since each additional site needs people and equipment. And robotics is a fairer contest than language was, because nobody has the data yet (8.5). Speed of decision, focus on a narrow problem and closeness to customers are real advantages in the early years.
8. Use structural advantages where they are real. Rules favouring domestic AI and a trusted supply chain outside China can give lasting advantages with government-funded and security-conscious buyers (8.6), who may be required to verify the origin of components and software. These advantages are worth pursuing only where the rules actually apply to the intended customers.
9. Get the timing right. Being too early earns nothing (8.4): a company that tries to sell a capability before it is reliable spends its capital on pilots that do not convert. The discipline is to watch for the moment a capability becomes reliable enough for a paid task, measured in success rate and interventions on the real job, and then move fast.
11.6 How a large company could still win, and a moat test
Four developments could still favour a large company:
- A model that works anywhere without new data. If general models become good enough to perform well at any site with no new data, which is the ideal described in 6.6 and is known in machine learning as zero-shot performance, then knowledge about individual sites loses much of its value. Advantage would shift further toward distribution, trust and owning the customer relationship, areas where established companies with large sales forces are already strong.
- Large companies with their own demand. A car or electronics maker with its own factories can deploy thousands of robots internally. It controls the sites, needs no sales cycle and can tolerate early failures, so it generates data and scale that no startup can match in that setting. The constraint is that internal deployments cover the owner's own processes, which may not represent the wider market.
- Bundling. A platform company can sell models, simulation and chips together, pricing the bundle so that any one piece appears nearly free and squeezing competitors who sell only one of them.
- Hardware price wars. Subsidised manufacturing can push hardware prices below cost and wipe out hardware profit margins, eliminating any company whose business depends on earning them.
A moat, in strategy, is an advantage that protects a business from competitors over time, as a moat protects a castle. To test any claimed moat, ask: "If a large company released this capability for free tomorrow, would we still win?" If the answer is no, the advantage is only a feature, something that improves the product today but that a competitor can neutralise by matching it. Deployment data, trust across many sites, workflow knowledge and a fast learning loop usually pass the test, because releasing a model does not transfer any of them. A slightly better model or a slightly cheaper part usually does not.
Why it matters: Put together, the evidence points to one position. Begin with specific, paid work in places that already exist, where the variable tasks around current equipment are what hold production back. Build the minimum robot each job needs from standard parts (2.2), and train one shared model across every configuration, so the variety of robot bodies becomes variety in the data (3.10). Start from the best open models, fine-tune them on demonstrations from real work, and let each deployment improve through imitation, correction, and practice (3.7). Sell the work the robot does, and measure progress only by tasks customers pay for.
Part 12. Glossary
- Action chunking: predicting a short sequence of future actions at once instead of one at a time. It lets a slower or remote model keep a robot moving smoothly.
- Active learning: letting the model's own weaknesses decide which data to collect next.
- Actuator: a motor, gears, sensors, and control electronics at a joint; the robot's equivalent of a muscle. Actuators make up 30 to 50 percent of a humanoid's parts cost.
- Agentic harness: a higher-level reasoning system that strings a robot's skills together into long jobs.
- ASIC: a custom chip built for one workload. It is efficient, but may become useless if model designs change.
- Behavioral cloning: see imitation learning.
- Benchmark: a fixed set of tasks and a scoring rule that every model is run on, so results can be compared. Useful for direction, easy to over-fit.
- Bitter lesson: the observation that simple, general methods given more data and computing power beat hand-designed methods.
- BOM (bill of materials): the list of every component in a product and its cost.
- Chain-of-thought: the robot works out a plan in steps, in words or pictures, before acting, so it can use general knowledge.
- Closed-loop (reactive) simulation: a virtual world that responds to the robot's actions, unlike an open-loop replay of a recording.
- Cobot: a collaborative robot arm designed to work safely near people.
- Compositional generalization: combining known skills or concepts in a new way never seen in training.
- Cross-embodiment: one model controlling many different robot bodies.
- Degrees of freedom (DoF): the number of independent ways a joint or system can move. A human hand has roughly 20 or more.
- Design-win: a lasting contract to have your part built into a customer's product (a chip-industry term).
- Diffusion / flow matching: methods that produce a precise output by refining random noise over a few steps; used to produce smooth robot motion.
- Disengagement: a person taking back control from an autonomous system; the standard autonomy metric.
- Distillation: training a small model to imitate a large one, so it can run on the robot.
- Embodiment: a specific physical robot, with its own body, sensors, motors, and proportions.
- End-to-end model: one learned system that goes from camera pixels to motor targets, instead of a chain of separate perception, planning, and control modules.
- Foundation model: one broadly trained model adapted to many tasks; in robotics, one model for many robot bodies and tasks.
- Geofencing: limiting operation to a mapped or bounded area.
- Homologation: regulatory approval to sell a vehicle or component in a market.
- Imitation learning (behavioral cloning): training a model to copy human demonstrations. It tops out near human level and copies human fumbles.
- In-context learning: adapting from the last few examples or moments of experience, without retraining.
- Inference: running a trained model, which is an ongoing cost, as opposed to training, which is a one-time cost.
- Jevons paradox: when something becomes cheaper, total use of it goes up.
- Mixed autonomy: the robot works on its own and a person corrects its mistakes; the stage that lets deployments earn money before full autonomy.
- Moravec's paradox: what is easy for humans is hard for robots, and the reverse.
- NAV (net asset value): the value of what a fund holds. The fund's shares can trade above (premium) or below (discount) that value.
- ODD (operational design domain): the set of conditions a system is designed to operate in.
- Open X-Embodiment: the project that pooled data from about 10 robot types and showed a general model beat specialist models by 50 percent.
- Out of distribution: a situation unlike anything in the training data.
- PD controller: a simple classical feedback controller that drives a motor toward a target position.
- Physical commonsense: a working feel for contact, force, friction and how objects move once touched. The part of manipulation general-purpose AI models still lack.
- Policy: the model that decides what the robot does based on what it sees.
- Premium crystallization: issuing new fund shares above NAV to raise the value of each existing share.
- Proprioception: the robot's sense of its own body, such as its joint angles.
- Real-time chunking: computing the next sequence of actions while the robot carries out the current one, to hide delay.
- Reinforcement learning (RL): the robot practices and improves from a success signal; it can exceed human performance.
- Reward model / value function: a model that watches the robot and scores its progress; drops in the score flag mistakes.
- Scaling laws: the pattern of performance improving predictably with more data, computing power, and model size.
- Sensorimotor common sense: the skill of continuously seeing the world and adjusting the body to it.
- Sim-to-real gap: the difference between simulation and reality that can make a skill learned in simulation fail on a real robot.
- Sovereign AI: countries wanting AI built and controlled at home.
- SPV (special purpose vehicle): a legal entity that groups several investors into one line on a company's ownership table.
- Teleoperation: a person controlling the robot remotely; it delivers a service and produces training data.
- Throughput: useful output per hour, combining success rate and speed.
- UMI (Universal Manipulation Interface): a hand-held gripper with a camera, used to collect demonstrations without a robot.
- VLA (vision-language-action) model: a language model adapted to output robot actions from camera images and an instruction.
- Wearable capture: collecting demonstrations from people wearing or holding capture devices, with no robot involved.
- World action model: a model that predicts future video and the robot's actions together.
- World model: a responsive simulation close enough to reality to train, test, or imagine in; increasingly used inside the robot's decisions, not only offline.
Further learning
- How to judge a robot demo. Ask how long it ran and how many failures were cut out. Check whether the robot is reacting to the world, as on stairs or when grasping, or replaying a planned motion, as in dancing or backflips. Ask how much error the task allows, whether this exact place and object were in the training data, and whether the robot is in a pilot or deployed across many sites.
- The most reliable information is in research papers, which are written for specialists. Public demos show the best a robot can do, not what it usually does.
- Judge progress by whether a robot does an ordinary task reliably in a place it has never seen, which tells you more than a stunt in a controlled setting.
- Follow the open debates as new evidence comes in: which family of models wins (3.5), whether world models run inside robots (4.9), whether compute runs on the robot or in the cloud (5.6), and where lasting advantage comes from (7.9).
Sources
The industry figures in Part 2 come from RCSV Research, State of Robotics 2026. Other published work referred to in the text:
- Open X-Embodiment Collaboration (2023). Open X-Embodiment: Robotic Learning Datasets and RT-X Models.
- Chi et al. (2024). Universal Manipulation Interface: In-The-Wild Robot Teaching Without In-The-Wild Robots.
- Unbeaten Path (2026). Robotics companies should go full-stack. unbeatenpath.substack.com, Issue 38, August 25, 2026.
- RoboDojo Team (2026). RoboDojo: A Unified Sim-and-Real Benchmark for Comprehensive Evaluation of Generalist Robot Manipulation Policies. Leaderboard, arXiv 2607.04434.
- Zhang, Wang, Ouyang et al. (2026). An Unexpected Robot Policy (RoboDojo evaluation report). Report, arXiv 2609.24170.
- The Bitter Lesson Comes to Robotics (2026), arXiv preprint. arXiv 2610.25287.