v2 Report

v2 changes three things: the car (a Tesla Model 3), the courses (two new layouts on a lot that none of the fine-tuned models trained on) and who drives. Next to eight frontier vision-language models (VLMs) and a human, six robotics models, fine-tuned on 94 minutes of our own driving, drive the same car through the same controller. This report covers what is new. For the basics (the tools, the protocol, safety), see the v1 report.

00What's new

  • The car. A 2021 Tesla Model 3 instead of the 2022 Corolla. Its steering is about 2.5× faster and has nearly twice the range, so it can turn on a 6 m radius instead of 16 m.
  • Two courses. Both are on a lot that none of the fine-tuned models ever saw. Course A is about 141 m: a left, a long right sweep, a left hairpin and a sharp right into the finish. Course B is shorter (about 92 m) and tighter: a left straight into a tight right, then two more rights into the finish.
  • A second kind of driver. Five vision-language-action models (VLAs) and one world-action model (WAM): FLUX 3 Action, π0.7, π0.5, Dream-1 (the WAM), GR00T N1.7 and Xiaomi Robotics-1. All six were fine-tuned on our driving dataset and drove five independent episodes per course. They output timed chunks of curvature and speed instead of calling tools.
  • More frontier models, more trials. GPT-6 Astra, GPT-6.1 Sol, GPT-5.6 Sol, Claude Opus 5.5, Claude Fable 5.2, Claude Fable 5.1, Gemini 4 Carbon and Grok 4.6 each ran three independent trials per course.
  • A human baseline. One of us drove both courses through exactly the three tools the VLMs get.

01The Tesla

satelliteWi-FiCAN · angle, 50 Hzfrontier modelOpenAI · Anthropic · Google · xAIthinking, s to minStarlinklaptopchat appMCP servergatewayCodex · Claude Code · Antigravity · Cursorcomma fourtwo road camerasopenpilot+ our controllerharness at the Autopilot computerbehind the gloveboxwheel turns ≤ 250°/shuman driver, brake always readyTesla Model 3time left on the command
A chat model drives the Tesla. The loop is v1's. An observation (teal) goes from the comma to the model, and the model's command (blue) comes back and replaces whatever the car was doing. Two things changed: the comma joins the car at the Autopilot computer, and a Starlink dish is the one network for the laptop, the comma and the models' servers.

On the Corolla, the comma four plugged into a camera connector behind the mirror and asked the steering motor for torque: openpilot pushed until the measured wheel angle matched (v1 report). On a Model 3 with Autopilot hardware 3, comma's harness goes in line with the Autopilot computer, behind the glovebox. openpilot then sends the car the steering-angle and acceleration requests that Autopilot itself would send, and the car's steering rack turns the wheel to that angle. Installation took more work than the Corolla's. The harness sits deep behind the glovebox, our car's current firmware wasn't recognized automatically, and openpilot's own speed control on Teslas is still an alpha setting we had to turn on.

It was worth it. At our speeds, openpilot lets the Tesla's wheel move up to 250° a second (5° every 20 ms frame), about 2.5× the Corolla's 100°/s. The wheel can also go almost all the way to lock: 100% steering is now 340° of wheel, where the Corolla's 180° scale delivered only about 130° in practice. The tightest circle drops from about 16 m to about 6 m, and a 90° turn at 1 m/s from about 25 s to 9 s.

r ≈ 16 mToyota Corollav1wheel 130° at 100%up to 100°/s90° turn in ≈ 25 sr ≈ 5.8 mTesla Model 3v2wheel 340° at 100%up to 250°/s90° turn in ≈ 9 sgrid 5 m · 4× speed
The same command on both cars: 100% left at 1 m/s. The Corolla's numbers are measured in the v1 trials; the Tesla's come from opendbc's Model 3 geometry (12:1 steering ratio, 2.875 m wheelbase).

This changes what a course can ask for. v1's turns had to stay gentle enough for the Corolla, and a late correction often couldn't be recovered. Course A's two tightest turns are about 7 m in radius, and Course B's sharpest right is about 5 m on the course line, close to the Tesla's limit. The safety layers and the safety driver are as in v1 (v1 report).

02Two kinds of driver

VLMs drive as in v1: in their own agent apps (Codex, Claude Code, Antigravity, Cursor), through observe, set_motion and stop_now. A trial is one conversation of up to three attempts, with a reflection prompt after each failed one. It stops early once an attempt reaches the finish and is scored by its best attempt. Only the prompt's notes on the car changed. v1's turning example was too conservative (v1 report); v2's gives the Tesla's rough radius at 30%, 60% and 100% steering and warns that 100% swings the car around almost on the spot.

VLAs and WAMs have no tools and no conversation. A runner on the laptop sends the model the latest camera frames and the car's measured curvature and speed. The model returns a chunk: timed setpoints of curvature (1/m, left positive) and speed. The comma runs the chunk through the same controller, steering scale and speed ceiling as set_motion, until the next chunk replaces it. Each episode starts fresh, with nothing carried over from the last.

satelliteWi-FiCAN · angle, 50 Hzfine-tuned VLA or WAMserved in the cloud≈ 2–10 chunks/sStarlinklaptoprunnergatewayframe → views; chunk → /chunkcomma fourtwo road camerasopenpilot+ our controllerharness at the Autopilot computerbehind the gloveboxruns the row due nowhuman driver, brake always readyTesla Model 3time left on the command
A fine-tuned VLA or WAM drives the Tesla. Same car, comma and network, but much faster. Five of the six models sent about two chunks a second (1–3 across episodes), and Dream-1 ten. The VLMs sent a command every 11 s (Claude Opus 5.5) to over two minutes (Gemini 4 Carbon) on average. Each chunk is timestamped with the moment its frame was captured, and the comma runs whichever row is due now, so time spent in inference and transit can never shift the plan.

Latency matters more here than it looks. By the time a chunk reaches the car, its first rows are already in the past. Over every chunk the six models sent, a median 0.5 s of plan (0.4–0.6 s for the middle half) was due before it arrived: the frame's age, inference, transit, and the 0.2 s the steering takes to respond. Only the rows still ahead get driven, until the next chunk replaces them. Our first runner skipped rows using the laptop's clock alone and forgot the transit time, so the car ran rows that were already stale. Timestamping each chunk with its frame's capture time, and letting the comma pick the row, fixed that.

0.00+0.05+0.10+0.150 s1 s2 s3 s4 s5 s6 scurvature 1/m · left ↑

chunk 12 of 12 · 3 rows already past · first row due +0.128/m at 1.1 m/s

Recorded chunks, replayed. Each dotted line is one chunk's curvature plan, drawn at the times its rows were due. Faded rows were already past when the chunk arrived, solid rows were driven, and hollow rows were replaced by the next chunk. The teal line is the curvature the car measured. On Course A, FLUX 3 Action and π0.5 turn into the hairpin. On Course B, FLUX 3 Action holds a left turn until its episode ends, short of the course's tight right.

03The fine-tuning dataset

DrivingBench Driving Dataset 1 is 108 minutes of a person driving the Tesla by hand through cone courses: 62 passes over 4 parking lots and 6 layouts, recorded at 10 Hz with both road cameras, the car's curvature and speed, and its pose. Collecting it was as simple as driving. openpilot logs both cameras and the full CAN log whenever the car is on, so a pass is one labeled session in our operator UI: start it, drive the course, stop in the finish area, end it. Each pass is trimmed from 1 s before the car moves to 3 s after it stops, so a model learns to stop but not to idle. 54 passes (94 minutes) are for training and 8 are held out.

Day · lot DDay · lot D
Day · lot DDay · lot D
Dusk · lot BDusk · lot B
Dusk · lot DDusk · lot D
Night · lot BNight · lot B
Night · lot CNight · lot C
Front-camera frames from the dataset, before cropping.
  • Day to night. We drove over two afternoons into the evening: 31 passes in daylight (half of them in low sun), 18 at dusk and 13 at night under the car's headlights and the lots' lamps.
  • Both directions. Every layout was driven both ways (31 passes clockwise, 31 counterclockwise), which gives a second, different course for free: what was a left turn becomes a right turn approached from the other side.
  • Slow and turny. The median speed is 1.6 m/s (3.6 mph), and 97% of moving time is under 2.5 m/s. A quarter of moving time is spent in turns tighter than a 12 m radius, and the median pass turns through 436° in total.
  • Mostly clean, some recoveries. Most passes were driven as well as we could. In some, we drove the way a model might go wrong (drifting toward one side, or turning in late) and then recovered, with a hard turn or by braking and then turning. Pure imitation of perfect driving never shows a model how to get back on course.

The dataset is tiny by robotics standards: about 10 km of driving, under 2 hours. None of it comes from the evaluation lot; the nearest training lot is half a kilometre away.

drivencollected

driven the other waycollected

mirroredtraining only

mirrored, other waytraining only

One layout becomes four. We drove each layout in both directions; at training time, every pass was also mirrored left to right.

04Fine-tuning the models

All six models were fine-tuned on the same 54 training passes. We fine-tuned the four open-weight models (π0.5, FLUX 3 Action, GR00T N1.7 and Xiaomi Robotics-1) on the Dreamscale Labs platform. π0.7 and Dream-1 were fine-tuned through their makers' services, on the same dataset.

The models differ a lot, so for each one we tried to choose the settings that suit it best, to give each a fair shot. We think that is fairer than forcing one frame rate, batch size or step count on very different models. Most of them shared a few data choices:

  • Two views instead of one frame, for models that take small square images. Shrinking the whole 672×380 front frame into a small square leaves the next gate's cones a few pixels wide. The training data instead drops the sky (everything above row 140) and splits the rest down the middle into two near-square views, with the cones twice as large.
  • Mirroring. Every training pass was also flipped left to right, with both cameras mirrored and curvature, yaw and lateral position negated: 108 episodes from 54 passes. The data was already balanced between left and right turns; mirroring doubles how many turns, approaches and recoveries a model sees.
  • One look per episode. Many training episodes got one mild colour shift for their whole length (±0.3 EV exposure, contrast, saturation and white balance), never per frame, so video models see no flicker.
  • Moving off. From a standstill, models predicted speeds below the car's minimum commanded speed of 0.5 m/s, so at first they never moved. The executor treats 0.15–0.5 m/s as a 0.5 m/s crawl, and anything slower as a stop, for every model.

05Results

The leaderboard has every attempt, with traces, video and transcripts. Progress is how far along a course's reference line the car got, from the end of its starting zone (10 m on A, 3 m on B) to the finish. It comes from GPS, good to 1–2 m; the GPS tracks and the scoring script are published with the traces on Hugging Face, in v2-evaluation/. A VLM's score is the mean over its three trials of each trial's best attempt; a VLA's or WAM's is the mean of its five episodes. Both-courses scores average a model's two course scores.

Human baselineGemini 4 CarbonGPT-6 AstraClaude Opus 5.5Claude Fable 5.2GPT-6.1 SolClaude Fable 5.1Grok 4.6GPT-5.6 Solπ0.7Dream-1Xiaomi Robotics-1FLUX 3 Actionπ0.5GR00T N1.7turnsCourse A · 141 m0%50%100%1234100%100%33.8%38.2%32.6%18.3%15.8%10.7%10.9%100%100%84.1%100%68.5%43.1%Course B · 92 m0%50%100%123489.9%79%36.7%22.8%10.1%23.6%9%10.9%7.9%41.3%35.3%39.6%21.8%22.5%21.6%
Every attempt on each course. Each dot is one attempt (VLMs and the human, one line per trial) or one independent episode (VLAs and WAMs). Solid dots count toward the score: a trial's best attempt, or every episode. Ringed dots reached the finish. The bar and the number are the leaderboard's score, its whisker ±1 standard error over the course's trials (none where a course has a single trial); the ticks below mark each course's four turns.
  • Course A was within the fine-tuned models' reach; Course B was not. On A, FLUX 3 Action, π0.7 and Dream-1 finished all five of their episodes, and 19 of the 30 episodes finished. On B, none of the 30 did. The best score on B was π0.7's 41.3%, and FLUX 3 Action fell from 100% to 21.8%. Across both courses π0.7 leads the fine-tuned models, at 70.7%.
  • One VLM drove both. Gemini 4 Carbon finished all three of its trials on A and two of three on B, scoring 100% and 79%. No other VLM finished a trial. The next best scored 38.2% on A (Claude Opus 5.5) and 36.7% on B (GPT-6 Astra). Even Gemini's first attempts on B averaged 35.2%, level with the best three fine-tuned models there (35.3–41.3%).
  • But slowly. Gemini's five finishes took 26 to 49 minutes each, at about one command every two and a half minutes. The fine-tuned models' 19 finishes on A took 83 to 97 seconds.
  • The human baseline finished A, not B. On A it went 72.5%, 90.1%, then 100%, finishing in 2 min 56 s. On B it went 31.5%, 9.0%, then 89.9%. Its both-courses score of 95.0% is the only one above Gemini 4 Carbon's 89.5%, though it had one trial per course to Gemini's three.

Course A

STARTFINISHN10 m1234≈ 6 mReference route ≈ 141 m1 Left, r ≈ 10 m2 Right sweep, r ≈ 9 m3 Left hairpin, r ≈ 7 m4 Right into the finish, r ≈ 7 m

Course B

STARTFINISHN10 m1234≈ 5 mReference route ≈ 92 m1 Left, r ≈ 9 m2 Right, r ≈ 5 m3 Right onto the back straight, r ≈ 9 m4 Right hairpin into the finish, r ≈ 6 m
  • π0.7
  • Dream-1
  • Xiaomi Robotics-1
  • FLUX 3 Action
  • π0.5
  • GR00T N1.7
Where the VLAs and WAMs went. Every episode's path, and a car where each one ended short of the finish, pointing the way it was going. On Course B, they bunch up on both sides of turn 2.
  • The fine-tuned models failed together on B. 29 of the 30 episodes ended before turn 3. Fifteen ended in or just after turn 1, nearly all still steering left, short of the tight right. The other fourteen made turn 2 but were still steering hard right after it, where the course eases.
  • VLMs mostly ended early. 63 of 70 VLM attempts on A, and 63 of 71 on B, ended before turn 2. GPT-5.6 Sol and Grok 4.6 ended all 18 of their Course A attempts at or just past the first left, nearly all well to its outside.
  • The best fine-tuned models repeat themselves; VLMs vary. On B, π0.7's five episodes all ended between 38.2% and 43.8%, and GR00T N1.7's between 19.1% and 23.6%. VLM trials swing more: Claude Fable 5.2's three Course A trials scored 67.2%, 5.3% and 25.2%.
  • In-context learning is weak. In the 46 VLM trials that reached a third attempt, the third beat the first in 29 (26 of 42 without Gemini 4 Carbon), and the median change was 3 points. Gemini is the exception: four of its five finishes came on a second or third attempt, after a failed one.

06Limitations

Three trials per VLM per course, five episodes per fine-tuned model and one human trial per course are enough to see large differences, not small ones. The two kinds of driver answer different questions: a VLM gets up to three attempts with reflection in one conversation and is scored on its best, while a VLA or WAM gets five independent episodes and is scored on their mean. The fine-tuned models were not trained alike: each got the settings we thought suited it, on its own platform, so a gap between two of them reflects their fine-tuning as well as the model. Courses A and B were driven on different days, so light and conditions differ along with the layout. And as in v1, the forward cameras can't see the ground right beside or under the car.

07Conclusion

On Course A, 94 minutes of driving was enough: three fine-tuned robotics models drove an unseen course cleanly every time, in about a minute and a half. On the tighter Course B, all six failed, almost all around one pair of turns, most still holding the turn they were in when the course changed. General-purpose models still drive by thinking: slowly, with high variance and little gain across attempts. One of them, Gemini 4 Carbon, finished both courses, at over two minutes a command. Neither kind is a good driver yet. The policies need to carry what they learned to new layouts, and the VLMs need to get faster. A fast policy for control under a slow model that can notice when it is about to make the same mistake again still looks like the interesting combination.

Acknowledgements

Thanks to Dreamscale Labs, on whose platform we fine-tuned π0.5, FLUX 3 Action, GR00T N1.7 and Xiaomi Robotics-1; to Physical Intelligence for π0.5 and π0.7, Black Forest Labs for FLUX 3 Action, Dream Labs for Dream-1, NVIDIA for GR00T N1.7 and Xiaomi for Xiaomi Robotics-1; and to comma.ai for the comma four and openpilot.

Citation

Please cite this work as:

@misc{drivingbench2026v11,
  title  = {DrivingBench v2: Frontier models and fine-tuned robot policies drive a real car},
  author = {Aditya Ramabadran and Simon Mahns and Tobias Gessler},
  year   = {2026},
  url    = {https://drivingbench.com/report/v2/},
}