A robot arm keeps missing a handle, and nobody wants to code every motion. In imitation learning robotics, a person shows the task and the robot learns a control policy from those demonstrations; the main limit is states the demonstrations never covered.
This explainer sits within artificial intelligence in robotics. It focuses on how demonstrations are collected, where copying a human goes wrong, and how the approach differs from reinforcement learning.
What imitation learning does
A policy, not a recording
A common misunderstanding is that the robot replays what it was shown. Imitation learning, also called learning from demonstration, has the robot learn a control policy by analysing demonstrations performed by an algorithmic or human supervisor, as the Berkeley AI Research DART post puts it. A policy maps what the robot observes to what it should do next.
The early driving work shows the difference. In ALVINN, a neural network was trained on driving demonstrations and learned a policy that mapped images of the road to steering. The network produced steering for whatever the camera showed, so it was generalising from examples rather than playing back a fixed path.
Behavior cloning and DAgger side by side
Two terms matter most for the rest of this page. Behavior cloning is the off-policy variant, where demonstrations are given independent of the robot’s policy. DAgger is an on-policy method, where the robot’s own policy is executed and a supervisor is queried for feedback on the states the robot visited.
| Aspect | Behavior cloning | DAgger |
|---|---|---|
| Policy type | Off-policy | On-policy |
| How demonstrations arrive | Given independent of the robot | Supervisor gives corrective feedback while the robot’s policy runs |
| Known weakness | Cannot entirely solve imitation learning because of covariate shift | Can be difficult for human supervisors, potentially dangerous, and computationally expensive |
| Planning use | Suits a first dataset when supervisor time or safety is the constraint | Suits correcting a policy that drifts, if a supervisor can respond safely |
The definitions and limitations in this table come from the Berkeley AI Research DART post.
Collecting demonstrations
Teleoperation as the input
Teaching a robot by demonstration usually means a human teleoperating the robot through the task so that the examples are recorded. The Berkeley post describes exactly this: a human tele-operates a robot to perform the task and provide examples. The recorded observations and actions together form the demonstration dataset that the policy is trained on.
Choosing when the supervisor gets involved
The collection method shapes the data. With behavior cloning, the supervisor demonstrates independently of the robot, so the data can be gathered without the learned policy ever moving the hardware. With DAgger-style collection, the supervisor must respond to states the robot actually reaches. The Berkeley authors note that such on-policy methods can be difficult for human supervisors, potentially dangerous and computationally expensive.
The practical judgement follows from that. Start with independent demonstrations when supervisor effort or safety is the bottleneck. Add corrective feedback on the robot’s own runs when the learned policy keeps failing in situations the demonstrations did not show.
Where copying a human fails
Covariate shift and compounding error
The compounding error problem is the best-known weakness of behavior cloning. The Berkeley post states that imitation learning cannot be entirely solved with behavior cloning, because of covariate shift: the states encountered during training differ from the states encountered during testing. Once a small mistake moves the robot into a state the human never demonstrated, the policy has no example to follow and the next mistake is more likely.
ALVINN illustrates it. Behavior cloning was applied to teach cars to drive in the late 1980s, and after learning, the policy was unstable. The demonstrations never covered the road edges, so a car that drifted to the side had no recovery examples to draw on.
How DAgger responds
DAgger targets the gap directly. Because the supervisor is queried about the states the robot actually visited, the dataset grows to include the off-course situations that plain demonstrations skip. The price is the supervisor burden described above.
How many demonstrations a robot needs
No single number applies across tasks, so published counts are best read as examples rather than targets. In a planar grasping study described in the Berkeley DART post, the policy was trained with DAgger on 160 expert demonstrations and reached a 90% success rate. That figure belongs to that task and method.
The useful lesson is about coverage rather than volume. Demonstrations that include the robot’s own mistakes and recoveries address the failure described in the previous section, while more copies of clean demonstrations do not.
Imitation learning versus reinforcement learning
The sharpest contrast is in what the robot learns from. Imitation learning starts from a supervisor’s demonstrations. Reinforcement learning for robots is the separate approach where the policy is improved through the robot’s own experience rather than from a human example.
- Training input. Imitation learning relies on demonstrations from an algorithmic or human supervisor, so a person’s skill is the ceiling and the starting point.
- Typical failure. Behavior cloning fails when the robot leaves the states covered by the demonstrations, which is covariate shift.
- Human effort. Demonstrating a task is a different job from specifying one in code, which is why teaching by demonstration appeals to non-programmers.
- Combination. The two are not rivals. GAIL, a variant in the inverse reinforcement learning style, uses reinforcement learning to reduce covariate shift, as the Berkeley post notes.
The choice therefore depends on what you already have. If a person can perform the task and the robot can be driven by teleoperation, imitation learning is a direct starting point. If the main risk is drifting into unseen states, bringing reinforcement learning in, as GAIL does, is one documented response.
Newer methods and neighbouring approaches
Recent work keeps the demonstration idea and changes how actions are predicted. A preprint reviewing the field, available on arXiv, notes that leveraging expert demonstration data lets robots acquire complex control strategies autonomously. It reports that Action Chunking with Transformers improved task performance by predicting joint positions for future k time steps instead of a single step. It also reports that the SRIL method ran 100% to 200% faster than state-of-the-art policies on dual-arm tasks, which is the preprint’s own result for those tasks.
A learned policy is one way to produce motion. Optimisation-based controllers are another route, covered in model predictive control robotics, and the two are often compared when choosing how a robot should act.
Frequently asked questions
| Question | Answer |
|---|---|
| Can I teach a robot by demonstration without coding? | Largely, yes. The approach uses expert demonstration data so robots can acquire control strategies without manual programming, according to the arXiv preprint. In practice a human teleoperates the robot to provide examples. |
| Does DAgger always beat behavior cloning? | No. In a human study with 10 participants described in the Berkeley DART post, behavior cloning outperformed DAgger. Supervisor difficulty is one reason to expect that. |
| Who published the DART post and when? | Michael Laskey, Jonathan Lee and Ken Goldberg published it on Oct 26, 2017. It accompanied a paper to be presented at the 1st Conference on Robot Learning. |
