Adversarial ML & AI Security · Capstone brief
SEC-09 · Red team tabletop
For a fictional (or teacher-approved hypothetical) AI feature launch at a school — attendance chatbot, tutoring copilot, or media-center search — can you run a tabletop, score the response, and produce an after-action brief a director could use?
School-approved sandbox only. No production targets. Showcase materials paraphrase class behavior — do not publish payload strings.
The question
For a fictional (or teacher-approved hypothetical) AI feature launch at a school — attendance chatbot, tutoring copilot, or media-center search — can you run a tabletop, score the response, and produce an after-action brief a director could use?
Lab / materials
No code required. Inject a single incident the teacher signs off on (e.g. “assistant gave harmful instructions to a student” or “RAG surfaced another student’s dummy record”). Use a simple inject timeline (T+0, T+30m, T+1d). Map the scenario to one NIST AI RMF function in plain language (https://www.nist.gov/itl/ai-risk-management-framework).
Expected failure modes
Fan-fiction crisis with no decisions. Scoring “who panicked best.” Inventing a real school’s incident.
Done looks like
Facilitator guide (roles: teacher, admin, IT, family liaison), inject list, scoring rubric for response quality (containment, comms, evidence, follow-up), and a two-page after-action with three fixes. Refusal log: no impersonation of real staff in outreach.
Five C's
CT: scoring decisions, not vibes. CR: injects that force a tradeoff (privacy vs parent notification). CO: classmates play roles; you facilitate. CM: after-action a director can forward. CZ: a real child’s name never appears.
Mentor role
School admin or incident-response person observes or reads the AAR at Checkpoint 2. School-supervised.
Rubric calibration
R1: one feature, one incident class. R2: injects are written so another class could rerun. R3: “we would just turn it off” is a baseline, not an automatic A. R4: gaps named. R5: AAR is short. R6: fictionalized.
Two ways this goes wrong
(a) A 20-page movie script. (b) A write-up of a rumor about a real district tool.
Source moved or something unclear? Send feedback on this brief. Mentors are advisory; the school supervises. These briefs do not produce verified computer-science credit.