CIFAR-10

CIFAR-10 is the standard step up from MNIST: 60,000 color images at 32×32, in 10 classes, split into 50,000 for training and 10,000 for testing. The images are small enough to train on a laptop but real enough that a linear model is clearly not enough. It is the right place to learn normalization, data augmentation, and the difference between a validation split and a test set.

The idea

Three differences from MNIST drive almost all CIFAR-10 practice:

• Color — three channels instead of one, so inputs have shape [32, 32, 3].
• Texture, not shape — the classes are animals and vehicles, so the signal lives in edges and object parts, which favors deeper convolutional stacks.
• Small images, limited data — 32×32 is coarse and 50,000 images is few, so overfitting arrives quickly.

Normalize per channel: subtract the training-set mean and divide by the training-set standard deviation, computed from the training split only. Using the test set for those statistics leaks information. Keep all 10,000 test images untouched until the end, and carve a validation slice out of the training data.

Worked example

Splitting and normalizing:

Augmentation: pad the 32×32 image to 40×40, take a random 32×32 crop, and flip horizontally with probability 0.5. That is the standard CIFAR recipe and it costs almost nothing.

In code

import tensorflow as tf

(x_train, y_train), (x_test, y_test) = tf.keras.datasets.cifar10.load_data()

mean = x_train.mean(axis=(0, 1, 2))
std = x_train.std(axis=(0, 1, 2))
x_train = (x_train - mean) / std
x_test = (x_test - mean) / std

augment = tf.keras.Sequential([
    tf.keras.layers.RandomCrop(32, 32),      # pads by 4 pixels first
    tf.keras.layers.RandomFlip("horizontal"),
])

model = tf.keras.Sequential([
    tf.keras.layers.Input(shape=(32, 32, 3)),
    augment,
    tf.keras.layers.Conv2D(32, 3, padding="same", activation="relu"),
    tf.keras.layers.MaxPool2D(2),
    tf.keras.layers.Conv2D(64, 3, padding="same", activation="relu"),
    tf.keras.layers.MaxPool2D(2),
    tf.keras.layers.GlobalAveragePooling2D(),
    tf.keras.layers.Dense(10),
])
model.compile(optimizer="adam",
              loss=tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True),
              metrics=["accuracy"])
model.fit(x_train, y_train, epochs=20, batch_size=64, validation_split=0.1)
model.evaluate(x_test, y_test)

RandomCrop(32, 32) pads by 4 pixels on each side before cropping, which is "pad to 40 then crop" expressed as one layer. Keras preprocessing layers are no-ops when training=False, so predict and evaluate see the original geometry.

Check yourself

1. Why must the per-channel mean and standard deviation come from the training split only?
2. What does RandomCrop(32, 32) do to the input before cropping, and why does that help?
3. Why is a validation_split of the training data a poor substitute for the held-out test set?

Key takeaways

• CIFAR-10 is [32, 32, 3] color data in 10 classes; normalize per channel from training statistics.
• Augmentation through pad-crop and flip is the cheapest way to close the overfitting gap.
• Keep the 10,000 test images untouched; tune on a validation slice of the training data.