AI projects · Understand · weekend

Does more training data always help?

These projects are not aligned to any state standards or curriculum. They are generally about learning, understanding, using, and evaluating AI.

Ages 11-14

Overview

People often say "just add more data." You will train the same tiny classifier with three training-set sizes and graph held-out accuracy. Sometimes more data helps; sometimes it plateaus; sometimes messy extra data hurts. You measure which happens in your setup.

Age band

Ages 11–14 (middle school).

Time needed

Weekend (about 4–6 hours), or two short weekday sessions.

AI goal

Understand — see how training-set size relates to test accuracy with a controlled experiment.

Materials

Free tools only; no paid accounts required.

  • Google Teachable Machine Image project, or another free browser trainer an adult approves.
  • One binary classification problem with plenty of photos (for example blue mug vs white mug), at least 40 images per class available.
  • Paper or spreadsheet for three runs: n = 5, n = 15, n = 30 training images per class.
  • A fixed held-out test set of 10 images per class that never enters training.

Steps

  1. Fix the question. "For this mug classifier and this held-out set, how does accuracy change when training size per class goes from 5 → 15 → 30?"
  2. Build the pool. Collect or photograph ≥40 images per class. Randomly assign 10 per class to the permanent test set. The rest are the training pool.
  3. Run A (small). Train with 5 images per class drawn from the training pool. Test on the fixed held-out 20 images. Record accuracy = correct/20.
  4. Run B (medium). Train with 15 per class (include the earlier 5 plus 10 new). Same held-out test. Record accuracy.
  5. Run C (large). Train with 30 per class. Same held-out test. Record accuracy.
  6. Graph. Draw a simple line chart: training size on the x-axis, accuracy on the y-axis (0 to 1).
  7. Interpret carefully. Write two sentences: (1) what happened in your graph; (2) why this still does not prove the slogan "more data always helps" for every AI system. Mention at least one reason accuracy might stall (easy test set, near-duplicates, or the model family being too simple).

Optional stress: add 10 badly lit extra images to Run C's training set and see if accuracy drops — that shows "more" is not the same as "better."

Ethics and safety note

Object photos only — no faces. Delete browser camera permission afterward. Do not claim your graph applies to huge industrial models; say what you measured.

What done looks like

CriterionNot yetDone
ControlsDifferent test images each runOne fixed held-out set for all runs
Three sizesOnly one training sizen = 5, 15, 30 per class with accuracies
GraphTable only, no visualLine graph with axes labeled
Interpretation"More data is always better"Describes your curve + a limit on the claim

Adaptations

Classroom

Groups share one photo pool; each group runs one size and the class builds the graph together.

Homeschool

Use printed photos taped to a board if camera upload is awkward; same counting rules.

Weekend

Skip the optional messy-data stress test; keep three clean runs.

Research

Repeat Run B three times with different random draws of 15 images; report the range of accuracies.

Science fair

Show the graph large; include the photo of the fixed test set so judges see the control.

All AI projects

Does more training data always help? · Schoolhouse