Capstone: ship a car-following driver
The course capstone: design a teacher that respects a safety envelope, train a driver by imitation, put the RSS filter in front of it, evaluate it on held-out scenarios and write the safety case.
The brief
You are shipping the longitudinal control of a learned driver: follow the car ahead, keep up with traffic, never hit it. A safety filter will sit between your network and the actuators, and a reviewer will read your one-page safety case before the driver goes anywhere near a road.
The scenarios are the car-following ones from Closed-loop simulation and the data engine: a lead car that usually cruises and, in some scenarios, brakes at 5 m/s² for up to 3 s. The filter is the RSS rule from Safety engineering for learned drivers, with the car assumed able to brake at m/s².
The capstone uses most of the course: imitation learning, closed-loop evaluation, the safety envelope and the evidence arithmetic. Plan on two to three hours.
What you write
The lab file is libs/tutorial/labs/capstone.py. The given parts are complete: drive runs any
action function in closed loop and reports the metrics below, and held_out_scenarios returns the
600 test scenarios (30 percent with a hard-braking lead). Do not train on them. You write four
functions:
| Function | What it must do |
|---|---|
cautious_expert(gap, speed, lead) | A teacher: an acceleration rule whose following gap stays clear of the RSS gap, while keeping up with the lead |
train_driver(seed) | A small network that imitates your teacher, trained only on scenarios you generate yourself |
guarded_action(driver) | The driver with the RSS filter in front of it, returning the acceleration and where the filter overrode |
evidence(collisions, scenarios) | The 95 percent upper bound on the collision rate that a test result supports |
How it is graded
Run uv run --group tutorial python scripts/tutorial_labs.py check capstone. The tests train your
driver once and drive it, guarded, through the held-out scenarios.
| Test | Passes when |
|---|---|
| The teacher respects the envelope | At the RSS gap it backs off; well beyond it, it closes in |
| The guard works | At an unsafe gap it brakes at and reports an override; at a safe gap it does not |
| No collisions | Zero collisions in the 600 held-out scenarios |
| The filter rarely steps in | Fewer than 2 percent of time steps overridden |
| The car keeps moving | Mean speed above 9 m/s |
| Evidence | The rule-of-three bound with no collisions, a larger bound with one, an error for zero scenarios |
Each test closes a loophole. A driver that always brakes never collides and never trips the filter, and fails on speed (0.66 m/s in our run). A driver that imitates the time-gap expert from the closed-loop module never collides either, because the filter catches it, but the filter overrides it in 14.6 percent of steps: the network is driving closer than the envelope allows and relying on the guard. Our reference solution has no collisions, overrides 0.03 percent of steps and averages 9.92 m/s.
Hints
- Design the teacher around the envelope. The cheapest way to make the filter quiet is a teacher
whose wanted gap is the RSS gap plus a margin (
rss_min_gapfrom the safety module). Then check that it still keeps up. - Collect the states your driver will see. Imitation fails where the demonstrations never went (Imitation learning). If your driver collides or trips the filter on scenarios you generate, use the data loop from the closed-loop module: drive with the learned driver, keep the failing scenarios, let the teacher label the visited states, refit.
- Use
driveon your own scenarios first.make_scenarios(n, hard_share, seed)fromlibs.tutorial.closed_loopmakes as many as you like; give them a seed different from 2026.
The safety case (written, one page)
The tests check the numbers. The safety case is the argument around them, and it is graded by a reviewer against this rubric.
| Section | A good answer |
|---|---|
| Operating domain | States what was tested: one lane, one lead car, the speed range and braking the scenarios contain |
| Hazards | Names the hazards the driver addresses (rear-ending the lead) and those it does not (cut-ins, pedestrians, a lead braking harder than 5 m/s²) |
| The envelope | States the RSS parameters, what they assume about the car and the lead, and why is credible for this car |
| Evidence | Reports the held-out result and the bound it supports (for 0 collisions in 600 scenarios, about 0.5 percent per scenario), and says plainly that this is far from a road-ready claim |
| What would change your mind | Lists the tests that could break the argument: harder braking, sensing delay, wet roads, a lead that brakes twice |
| Next evidence | Says what you would run next and how many failure-free scenarios a stronger claim needs |
A safety case that claims more than the evidence supports is a fail, however good the numbers.
Check yourself
- Why does a driver that relies on the filter count as a failure even with zero collisions?
- Zero collisions in 600 held-out scenarios supports what bound at 95 percent confidence?
- Why must the 600 test scenarios stay out of training, and what would training on them do to the result?
- Name two hazards your driver was never tested against, and how you would add them.
Answers
- The filter is the last line of defence, built for a worst case; if it acts in 15 percent of steps, the driver's behaviour is the filter's braking, which is abrupt and assumes the car can always brake at b_min. A good driver should leave the envelope alone almost always.
- About 3 / 600: −ln(0.05) / 600 ≈ 0.005, half a percent per scenario.
- They are the evidence. A driver trained on them can fit their specifics, so its result says less about scenarios it has not seen, and the bound computed from them is no longer honest.
- For example a lead braking harder than 5 m/s² (add scenarios with a stronger brake) and a delay between sensing and acting (add a lag to the gap the driver sees). Cut-ins and a second braking event are others.
What to hand in
libs/tutorial/labs/capstone.py, withcheck capstonepassing.- The one-page safety case, with the numbers from your own run.
Toolkit: rewards and policy gradients
The part of reinforcement learning a driving engineer needs: rewards and returns, the policy gradient, why a baseline matters, the group baseline GRPO uses, and why the policy is kept close to a reference.
Before you start
What the tutorial assumes, with a refresher on each piece: tensors and shapes, MLPs, losses and stable cross-entropy, backpropagation by hand, optimisers, overfitting and regularisation, embeddings, and entropy, cross-entropy and KL divergence.