Reading the Model Before the Crash

Autonomous-driving systems are increasingly being built as end-to-end models: a single learned system maps sensor streams to driving behavior, often without a clean boundary between perception, prediction, and planning. That design promises better generalization across cities and scenarios, but it also removes many of the intermediate quantities that engineers traditionally test.

Mechanistic interpretability and activation-based analysis offer a different route into these systems. Instead of asking only whether a vehicle merged, yielded, or avoided a collision, researchers can inspect the internal representations that appear before the action: clusters associated with following distance, attention to road users, uncertainty, or safety-critical scene structure. Wayve’s scenario-intelligence work, for example, treats emergent representations as a way to find coverage gaps and rare cases before they become on-road failures. (wayve.ai)

But an interpretable signal is not automatically a reliable warning signal. A representation may correlate with a failure without causing it; a saliency map may describe attention without revealing the decision mechanism; a simulator may reproduce an intervention while still misrepresenting how other road users react. Recent work on closed-loop neural simulation, controllable world models, and safety evaluation makes the problem more concrete—and more difficult. (wayve.ai)

This seminar follows one question from representation to deployment: can internal signals provide trustworthy, generalizing evidence of unsafe behavior before the vehicle acts, and what would it take to validate that claim in the real world?