Algorithmic Policy & Governance · Capstone brief

POL-01 · Take apart a risk score

Using only public documentation, reverse-engineer how a scoring or ranking system claims to work — then list what you still cannot know. Mystery is a finding.

The question

Using only public documentation, reverse-engineer how a scoring or ranking system claims to work — then list what you still cannot know. Mystery is a finding.

Policy / materials

One publicly documented system (agency model card, procurement filing, published methodology, or vendor docs the teacher approves). A worksheet with fields for inputs, outputs, human override, and unknown.

Expected failure modes

Treating marketing as methodology. Inferring protected attributes without evidence. Declaring the system fair or unfair without a pre-stated standard.

Done looks like

A teardown memo: claimed mechanism; evidence for each claim; unknown list; one paragraph on what a public body should demand before buying or expanding it.

Five C's

CT: separating claim from evidence. CR: choosing a system with a paper trail. CO: peer tries to find a doc you missed. CM: board-ready unknown list. CZ: people scored by the system.

Mentor role

Government teacher, debate coach, or policy mentor reviews the unknown list. Standing instruction: reject invented internals. School-supervised.

Rubric calibration

R1: one system. R2: every claim cited. R3: comparator is stated policy goals. R4: unknowns listed. R5: memo is readable. R6: refuses omniscience.

Two ways this goes wrong

(a) A confident essay with no citations. (b) A rant that never explains the mechanism.

Credit lane fit

Lane A immediately (gov, IR, leadership, AP Research). No verified credit claim.