AI projects · Understand · weekend
Does more training data always help?
These projects are not aligned to any state standards or curriculum. They are generally about learning, understanding, using, and evaluating AI.
Ages 11-14
Overview
People often say "just add more data." You will train the same tiny classifier with three training-set sizes and graph held-out accuracy. Sometimes more data helps; sometimes it plateaus; sometimes messy extra data hurts. You measure which happens in your setup.
Age band
Ages 11–14 (middle school).
Time needed
Weekend (about 4–6 hours), or two short weekday sessions.
AI goal
Understand — see how training-set size relates to test accuracy with a controlled experiment.
Materials
Free tools only; no paid accounts required.
- Google Teachable Machine Image project, or another free browser trainer an adult approves.
- One binary classification problem with plenty of photos (for example blue mug vs white mug), at least 40 images per class available.
- Paper or spreadsheet for three runs: n = 5, n = 15, n = 30 training images per class.
- A fixed held-out test set of 10 images per class that never enters training.
Steps
- Fix the question. "For this mug classifier and this held-out set, how does accuracy change when training size per class goes from 5 → 15 → 30?"
- Build the pool. Collect or photograph ≥40 images per class. Randomly assign 10 per class to the permanent test set. The rest are the training pool.
- Run A (small). Train with 5 images per class drawn from the training pool. Test on the fixed held-out 20 images. Record accuracy = correct/20.
- Run B (medium). Train with 15 per class (include the earlier 5 plus 10 new). Same held-out test. Record accuracy.
- Run C (large). Train with 30 per class. Same held-out test. Record accuracy.
- Graph. Draw a simple line chart: training size on the x-axis, accuracy on the y-axis (0 to 1).
- Interpret carefully. Write two sentences: (1) what happened in your graph; (2) why this still does not prove the slogan "more data always helps" for every AI system. Mention at least one reason accuracy might stall (easy test set, near-duplicates, or the model family being too simple).
Optional stress: add 10 badly lit extra images to Run C's training set and see if accuracy drops — that shows "more" is not the same as "better."
Ethics and safety note
Object photos only — no faces. Delete browser camera permission afterward. Do not claim your graph applies to huge industrial models; say what you measured.
What done looks like
| Criterion | Not yet | Done |
|---|---|---|
| Controls | Different test images each run | One fixed held-out set for all runs |
| Three sizes | Only one training size | n = 5, 15, 30 per class with accuracies |
| Graph | Table only, no visual | Line graph with axes labeled |
| Interpretation | "More data is always better" | Describes your curve + a limit on the claim |
Adaptations
Classroom
Groups share one photo pool; each group runs one size and the class builds the graph together.
Homeschool
Use printed photos taped to a board if camera upload is awkward; same counting rules.
Weekend
Skip the optional messy-data stress test; keep three clean runs.
Research
Repeat Run B three times with different random draws of 15 images; report the range of accuracies.
Science fair
Show the graph large; include the photo of the fixed test set so judges see the control.