Applied ML Systems · Capstone brief
DS-09 · Crash severity, and the limits of prediction
Given crash circumstances — road type, lighting, weather, restraint use — how well can severity be predicted, and what happens to the model's usefulness when you remove the variables a policymaker cannot change?
The question
Given crash circumstances — road type, lighting, weather, restraint use — how well can severity be predicted, and what happens to the model's usefulness when you remove the variables a policymaker cannot change?
Data
NHTSA Fatality Analysis Reporting System (https://www.nhtsa.gov/file-downloads?p=nhtsa/downloads/FARS/). Optionally Virginia DOT traffic volume counts for exposure (https://www.vdot.virginia.gov/doing-business/technical-guidance-and-support/traffic-operations/traffic-counts/).
Content note. FARS records fatal crashes. It is real people. Set expectations before assigning, and let a student choose a different brief without explanation.
Expected failure modes
FARS contains only fatal crashes, so "predicting severity" inside it is a truncated problem and students routinely forget this. Without exposure data, counts measure traffic volume, not risk. Restraint use is strongly predictive and completely uninteresting as a finding — the assignment is what remains after it.
Done looks like
A severity or outcome model built on FARS with the selection bias stated up front, a version excluding non-actionable variables, and a short list of road-design factors that survive.
Five C's
CT: reasoning about a truncated sample. CR: constructing an exposure denominator from a separate source. CO: peer challenge on the selection-bias framing. CM: writing about fatality data without sensationalizing. CZ: recommendations here affect road funding and the people who use those roads.
Mentor role
A traffic engineer, transportation planner, or public-health epidemiologist reviews the exposure approach at Checkpoint 1 and the recommendations at Checkpoint 2. School-supervised.
Rubric calibration
R1: the truncation is acknowledged in the question. R2: exposure denominator sourced or its absence stated. R3: a baseline using only restraint and speed. R4: performance reported on actionable variables alone. R5: sober, non-sensational writing. R6: refuses to rank roads without exposure data.
Two ways this goes wrong
(a) The student reports the deadliest intersections and has found the busiest ones. (b) The student builds a 94%-accurate model that is mostly "was the person wearing a seatbelt" and presents it as a road-safety finding.
Source moved or something unclear? Send feedback on this brief. Mentors are advisory; the school supervises. These briefs do not produce verified computer-science credit.