Competing-risks survival analysis / Waymo Open Motion Dataset
Pooled into a single model, both effects cancel and the answer is “no difference.” They only appear once conflicts are separated by type—which is the entire argument for modeling them as competing risks.
01 / PROBLEM
A robotaxi fleet drives millions of miles between serious collisions. Waiting for crashes to accumulate is not a safety program—by the time the statistics are significant, the software has been replaced repeatedly.
The project measures near-conflicts instead: moments where two road users came close enough that a small perturbation would have produced contact. They occur thousands of times per hour of driving and are observable in trajectory data every AV already records.
The catch is that a near-conflict is not a crash. Extracting one means deciding, from geometry alone, whether two vehicles were actually on intersecting paths—and that is where many pipelines quietly go wrong.
02 / APPROACH
Each vehicle entering a scene becomes a subject. The clock starts when it is first observed; the event is its first traffic conflict. Vehicles reaching the end of the recording without one are censored, preserving their time at risk. Rear-end, crossing, head-on, and lane-change conflicts are modeled as competing causes, with severity as a second partition of the same events.
Conflict type comes from the angle between the road users’ directions of motion—not their reported headings, which are unreliable for pedestrians and stationary vehicles. Thresholds follow the FHWA SSAM convention; the head-on split above 150° is project-defined and flagged wherever the numbers appear.
03 / MEASUREMENT ENGINEERING
Time-to-collision and post-encroachment time were designed for simpler traffic settings. Applied to a two-dimensional, multi-agent urban scene, each fails in a specific, reproducible way. Finding and correcting those failures was most of the work.
The classical formula divides center-to-center distance by closing speed. In two dimensions, it flags vehicles that merely pass nearby. I replaced it with exact rotated-bounding-box path intersection solved through an interval Separating Axis Theorem.
8 of 10 sampled flags did not intersectBehind another vehicle, PET is simply time headway, so a comfortable 1.2-second following distance registered as a conflict. Restricting PET to genuine crossing geometry removed those false events.
62% reduced to 1.4%An axis-aligned box inflates a vehicle at 45° to more than twice its footprint, making adjacent-lane vehicles appear to overlap. Exact rotated-box membership at 0.5-meter resolution corrected the geometry.
More than 2× footprint inflation corrected04 / RESULT
Adjusted for time at risk, traffic density, road geometry, and 52 other covariates, with standard errors clustered by scene. Two effects reach significance—and point in opposite directions, which is why the pooled model finds nothing.
Adjusted AV-versus-human hazard ratios with 95% confidence intervals, scene-clustered and unpenalised. Left of the dashed line means lower hazard. Filled markers are significant; the open severe marker denotes a large but unresolved effect. Rear-end represents 62% of all conflicts. Results were replicated independently in R.
Rear-end HR 0.854 (0.751–0.970, p = 0.016) and lane-change HR 1.598 (1.078–2.369, p = 0.020). Pooled together, the model reports HR 1.013, p = 0.79—two real effects in opposite directions, averaged into silence.
Posted speed limit raises rear-end conflict hazard and lowers lane-change hazard. Pooling those outcomes cancels that covariate effect too; one aggregate model cannot represent a predictor whose direction reverses across outcomes.
05 / SEVERITY
AVs show a 25–38% lower severe-conflict hazard across every specification tested. The point estimate is stable; statistical significance is not.
| Specification | Severe HR | p |
|---|---|---|
| Primary — full covariate set | 0.720 | 0.061 |
| Environmental covariates only | 0.747 | 0.093 |
| Including prevalent cases | 0.623 | 0.002 |
| Fine–Gray subdistribution | 0.722 | 0.062 |
Detecting HR 0.72 at 15% prevalence requires about 564 events at 80% power—the cohort contains exactly 564 severe conflicts. Because the study sits on that threshold, the effect is reported as unresolved, not as a finding. Roughly 845 severe events would be needed for 90% power.
06 / VALIDATION
All four were found by building diagnostics before trusting the results. Two were my own mistakes.
35.8% of human comparison vehicles were parked, versus 0.6% of AV egos. Parked cars manufacture severe conflicts because relative speed becomes the passing vehicle’s full speed. A baseline-only driving rule corrected eligibility without immortal-time bias.
A small ridge penalty retained from an earlier cohort shrank coefficients while understating uncertainty. One apparent finding at p = 9e-06 was honestly p = 0.11. Every reported model is now unpenalised.
Temporary Protocol Buffer wrappers reused memory addresses, handing one vehicle another’s trajectory. Negative survival times—including an event 14.9 seconds before observation—exposed contamination in roughly 45% of scenes. The dataset was rebuilt.
Including vehicles already in conflict at their first frame made proximity nearly deterministic, causing coefficients to diverge. This was a tautology, not a numerical nuisance, and the comparison was refitted without those proximity covariates.
Every hazard ratio was refitted independently in R. Python lifelines and R coxph agree to three decimal places across all seven models. Fine–Gray estimates from cmprsk::crr agree in direction, magnitude, and significance for every outcome.
07 / RESEARCH ROADMAP
The extraction pipeline and classical benchmarks are complete and cross-validated. Next are three multi-head neural survival architectures, external deep-survival benchmarks, a conflict-type × severity risk matrix, and SHAP attribution.
| Component | Purpose | Status |
|---|---|---|
| Cause-specific Cox | Interpretable hazard ratios by conflict type | Complete |
| Fine–Gray | Cumulative incidence under competing risks | Complete |
| SH- / MH- / CS-MNN | Multi-head neural survival architectures | Building |
| DeepHit · DeepSurv · DSM | External deep-survival benchmarks | Queued |
| Threshold sensitivity | Refit at 1.5 s and 2.0 s definitions | Queued |
| Risk matrix | Conflict type × severity probabilities | Queued |
| SHAP attribution | Actionable drivers by conflict type | Queued |
08 / SAFETY APPLICATION
The work sits under ISO 21448 (SOTIF): characterizing hazardous situations in which nothing malfunctioned. A near-conflict in crash-free data is exactly that.
Proximity alone is a poor trigger: two vehicles can sit meters apart at a red light without risk. A model trained on conditions provides an escalation criterion that survives review.
“The vehicle handles following distance better” becomes a hazard ratio with a confidence interval, adjusted for exposure and traffic, rather than a claim that waits years for crash counts.
Speed limit drives rear-end risk; junction proximity and crosswalk density drive crossing risk; lane alignment drives lane-change risk. Different failure modes require different mitigations.
Matched-condition comparisons can reveal a release that lowers rear-end conflicts while raising lane-change conflicts—a tradeoff that a pooled count cannot identify.
09 / TECHNICAL STACK
From raw Protocol Buffer trajectories through validated conflict geometry to fitted survival models. Extraction now runs in 4.5 seconds per 20-second scene—a 10× improvement—and the complete corpus rebuilds in under three hours.
