Visual Analysis for MNIST

MNIST is small enough — 28×28 grayscale, 60,000 training images, 10 classes — that you can visualize a whole model without a GPU. It is the best place to build the habit of looking at activations before trusting a number. The techniques here scale unchanged to larger images.

The idea

An MNIST CNN has two spatial scales worth inspecting:

• Conv filters — each 3×3 kernel can be drawn directly as a tiny image. Early filters tend to look like edge and stroke detectors.
• Feature maps — after feeding one digit, the (26, 26, 32) activation stack shows which filters fired where.

Dense layers are harder: a 128-unit layer has no geometry. Inspect it indirectly by reshaping each weight row to 28×28 and plotting it as the class template it supports.

A useful diagnostic is the activation histogram. If most activations sit at exactly 0, the ReLU is dead for that filter and it receives no gradient. Sorting filters by mean activation over a batch puts the alive ones first.

Worked example

Feed a handwritten 7 through Conv2D(32, 3) → MaxPool(2) → Conv2D(64, 3) → Flatten → Dense(128) → Dense(10).

• conv2d output: (1, 26, 26, 32).
• max_pooling2d output: (1, 13, 13, 32).
• conv2d_1 output: (1, 11, 11, 64).
• dense output: (1, 128).
• dense_1 output: (1, 10) — argmax should be 7.

Plot a 4×8 grid of the conv2d maps. The cells that light up trace the diagonal stroke of the 7. Now feed a misclassified example and plot the same grid beside the correct digit — you can often see why the model hesitated.

In code

import numpy as np, tensorflow as tf

(x_train, y_train), _ = tf.keras.datasets.mnist.load_data()
x_train = (x_train[..., None] / 255.0).astype("float32")

names = ["conv2d", "max_pooling2d", "conv2d_1"]
model = tf.keras.models.load_model("mnist_cnn.keras")
probe = tf.keras.Model(model.inputs, [model.get_layer(n).output for n in names])

for n, m in zip(names, probe(x_train[:1])):
    dead = int(tf.reduce_sum(tf.cast(m == 0, tf.int32)))
    print(n, m.shape, float(tf.reduce_mean(m)), dead)

The zero count tells you how many cells are dead for this digit. Repeating across digits reveals filters that never fire for any input — candidates to prune.

Check yourself

1. Why can a 3×3 conv kernel be plotted as an image while a 128-unit dense layer cannot?
2. What does a feature map of all zeros tell you about that filter and its gradients?
3. The Flatten output has 7,744 values — where does that number come from?

Key takeaways

• MNIST lets you render kernels, feature maps, and dense templates without a GPU.
• Dead ReLUs show up as all-zero feature maps; activation histograms find them.
• Always pair a numeric metric with one visual check before trusting a model.