Tutorial
From MLPs to vision-language-action models for driving: concepts, runnable PyTorch code, quizzes and papers.
Draft for review
Paper identifiers were checked against arXiv, and Alpamayo-R1 statements against the text of its PDF. The five inline exercises and forty-five PyTorch reference implementations are run in CI, and the toy results are quoted over several seeds. The explanations have not yet had a second reader, so treat them as a draft until one has been through them.
Who this is for
You know how to train an MLP or a CNN with gradient descent and what a loss, a batch and an embedding are. You have not necessarily worked through transformers, variational autoencoders, diffusion, flow matching or reinforcement learning. A robotics background (joints, trajectories, controllers) helps for the imitation, VLA, driving and world-model modules. By the end you should be able to read the Alpamayo-R1 paper, NVIDIA's open driving VLA, and explain each block.
We start with autonomous driving because that paper describes the whole recipe in the open and releases weights and code. Manipulation (SO-100 arm) examples from our own experiments are used as a second lens where the ideas carry over.
Modules
| Group | Module | You can afterwards |
|---|---|---|
| Foundations | Before you start | Check you have the prerequisites |
| Transformers | Explain self-attention and why it replaced recurrence | |
| Tokens, ViT and VLMs | Describe how an image and a sentence become one sequence | |
| Variational autoencoders | Explain a latent space, the ELBO, and where VAEs appear in robot learning | |
| Discrete tokenizers: VQ-VAE and FSQ | Explain vector quantization, dead codes and finite scalar quantization | |
| Diffusion | Explain denoising training and sampling | |
| Flow matching | Train a flow model on a toy problem and say why few steps work | |
| Sampling, guidance and diffusion policy | Use DDIM and classifier-free guidance, and explain a diffusion policy over action chunks | |
| Policies and reasoning | Imitation learning and action chunking | Explain compounding error, chunking, and why generative policies help |
| Reinforcement learning from scratch | Explain rewards, returns, values, the Bellman equation, Q-learning, exploration and baselines | |
| Reinforcement learning | Explain policy gradients, PPO and GRPO, and reward design | |
| RL for real robots: sim-to-real | Explain the reality gap, domain randomization and adaptation | |
| Preference learning and offline RL | Explain DPO and why offline RL needs conservatism | |
| RL for alignment: RLHF and its successors | Explain reward models, the KL-regularised objective, PPO in RLHF, over-optimisation, DPO, RLAIF and verifiable rewards | |
| Reasoning and chain of thought | Explain chain of thought, self-consistency, how reasoning is trained and why a fluent trace is not proof | |
| VLA models | Compare RT-2, OpenVLA, pi0 and SmolVLA by how they output actions | |
| Scaling and deployment | Scaling laws and data recipes | Fit a scaling ladder, size a model for a compute budget, and choose between up-weighting and mining the long tail |
| Co-training and catastrophic forgetting | Reproduce forgetting on a toy, and compare replay, EWC, smaller steps and adapters at keeping the old task | |
| Training at scale | Budget training memory, choose between data, ZeRO, tensor and pipeline parallelism, and estimate run time | |
| Distillation | Distil a large or slow teacher into a small or one-step student, including rule-based and on-policy teachers | |
| Efficient fine-tuning and inference | Explain LoRA, quantization, the KV cache and speculative decoding | |
| Deploying a driving VLA | Break an inference into a latency budget and choose the technique that cuts the stage that dominates | |
| Driving and world models | From modular stacks to end-to-end | Map perception, prediction and planning modules onto learned components and explain when end-to-end beats them |
| Sensor encoders for driving | Explain how cameras, lidar, radar, maps and vehicle state become features, how they are fused, and why occupancy beats a flat BEV | |
| Camera geometry and object queries | Project a 3D point into an image, move points between frames, and explain DETR-style matching | |
| State estimation and tracking | Build a Kalman filter, check its noise settings with the innovation test, and explain what it does when detections go missing | |
| Driving data and labels | Name the main driving datasets and benchmarks, explain how labels are made and checked, compute agreement beyond chance with its uncertainty, size an audit, and mine the long tail | |
| Planning and vehicle models | Check a trajectory against a bicycle model, simulate the IDM, and explain trajectory vocabularies and rule-based baselines | |
| Predicting other drivers | Explain why prediction is multimodal and why per-agent predictions miss collisions between agents | |
| Route conditioning | Explain why a policy needs the route at a fork, and the forms a route can take | |
| Evaluating driving models | Compute ADE, minADE, collision and PDM-style scores, and explain why open-loop numbers mislead | |
| Closed-loop simulation and the data engine | Explain why replayed agents mislead, and compare targeted and random data in a failure-driven loop | |
| World models | Explain what a world model is for and name the two families | |
| Uncertainty and runtime safety | Use ensembles and conformal prediction, and say when a guarantee does not apply | |
| Safety engineering for learned drivers | Wrap a policy in an RSS envelope, size the evidence for a failure rate, and explain why redundancy needs independence | |
| Driving VLA literature | Place Alpamayo-R1 in the line of work that led to it |
Each module ends with self-check questions, a quiz and a short list of sources, and most embed a tested PyTorch reference implementation. Plan on one to two hours per module including the sources. The foundation modules, reinforcement learning, preferences, reasoning and efficiency are general machine learning; the rest are robotics and driving.
Beyond the modules
| Page | What it gives you |
|---|---|
| Driving VLA concepts | The idea, the architecture, the training recipe, and the limits, using Alpamayo-R1 |
| Reading list | Papers in a sensible order, each with what to look for |
| Practices | Exercises: reading questions, running released code, and mapping onto our SO-100 results |
| Glossary | Terms in one or two sentences |
Suggested paths
Pick the path closest to your background; every module still stands alone.
| If you are | Read in this order |
|---|---|
| New to machine learning for robots | Before you start, Transformers, Tokens, ViT and VLMs, Flow matching, Imitation learning, Reinforcement learning from scratch, VLA models, then Driving VLA concepts |
| An engineer from a modular driving stack | The End-to-end driving course: eight sections from the modular stack to Alpamayo-R1 |
| Interested in training and alignment | Reinforcement learning from scratch, RL for alignment, Preference learning and offline RL, Scaling laws, Training at scale, Distillation, Efficient fine-tuning |
| Preparing for a safety or deployment review | Evaluating driving models, Uncertainty and runtime safety, Deploying a driving VLA, RL for real robots |
How to read it
- Start with Before you start and its self-test.
- Work through the modules in order, or skip the ones you know. Each runs its code exercise and embeds the tested reference implementation.
- Read the concepts page, then the Alpamayo-R1 paper with the reading-list questions beside you.
- Do the practices that fit your hardware.
Conventions
- Formulas are typeset with KaTeX; code exercises use NumPy and PyTorch. In this repository run the
reference code with
uv run --group tutorial python -m libs.tutorial.<name>(CPU is enough). - "The paper reports" is a claim by the authors. "We measured" is a run in this repository. Keep them apart: the first is evidence about their setup, the second only about ours.