ACC-08 · Caption quality

Score auto-captions for a classroom video set; define an acceptance bar before publishing.

Classroom video set (school-owned or teacher-approved). Auto-caption output from one tool. Scoring rubric fixed before watching (accuracy, speaker ID, timing, critical-word errors). Acceptance bar written before scoring. Deaf/hard-of-hearing partner input if consented; otherwise cite published caption-quality guidance via WAI (https://www.w3.org/WAI/).

Setting the bar after seeing the scores. Counting only word error and ignoring timing. Publishing “good enough” without a bar. Using a single funny fail as the whole study.

Pre-declared rubric and bar; score sheet; publish/no-publish decision with rationale; remediation steps (human edit budget estimate).

CT: quality as a bar, not a vibe. CR: pre-registration of the bar. CO: peer double-scores one video. CM: publish decision memo. CZ: caption-dependent viewers.

Media or special-ed teacher reviews the bar before scoring. School-supervised.

R1: one tool, one video set. R2: rubric applied consistently. R3: human-caption or no-caption comparator. R4: critical-word errors called out. R5: decision memo clear. R6: refuses publishing below the bar.

(a) Bar moved after scoring. (b) “Mostly fine” with no numbers.

Lane A immediately. No verified credit claim.