Adversarial ML & AI Security · Capstone brief
SEC-07 · Adversarial examples that matter
Using a teacher-provisioned robustness lab (tiny vision or audio classifier + an approved educational evaluation tool), do small allowed perturbations change the decision — and is that threat realistic for a target you name (school camera, lab microphone, homework-scanner fantasy)?
School-approved sandbox only. No production targets. Showcase materials paraphrase class behavior — do not publish payload strings.
The question
Using a teacher-provisioned robustness lab (tiny vision or audio classifier + an approved educational evaluation tool), do small allowed perturbations change the decision — and is that threat realistic for a target you name (school camera, lab microphone, homework-scanner fantasy)? You may conclude no.
Lab / materials
Teacher harness only (e.g. a digits/CIFAR-toy notebook the school already runs). Students do not hunt production cameras. Public reference for the idea of robustness evaluation: NIST AI RMF (https://www.nist.gov/itl/ai-risk-management-framework). No attack-code pastes in the public write-up.
Expected failure modes
Pixel-noise theater with no threat model. Declaring school security cameras “defeated.” Copying robustness-attack libraries into a public repo as the deliverable.
Done looks like
Lab measurement (clean vs perturbed accuracy under the harness’s allowed budget), a threat-realism paragraph (physical access, lighting, attacker goal), and a yes / no / only-if on whether this matters for the named target. “No” with a good argument scores well.
Five C's
CT: lab success ≠ operational threat. CR: a realistic attacker sketch — or a reasoned rejection of one. CO: peer argues the opposite realism claim. CM: a design-review slide. CZ: fear-mongering about cameras hurts students.
Mentor role
Computer-vision or physical-security person reviews realism at Checkpoint 2. School-supervised.
Rubric calibration
R1: named target + harness. R2: numbers from the lab, not a blog. R3: clean accuracy baseline. R4: realism gate is explicit. R5: a reviewer can disagree with your “no.” R6: no production systems, no cookbook.
Two ways this goes wrong
(a) Impressive lab plot, zero words on whether a hallway camera is in play. (b) A TikTok-threat slide deck with no measurement.
Source moved or something unclear? Send feedback on this brief. Mentors are advisory; the school supervises. These briefs do not produce verified computer-science credit.