Inception Model

The Inception architecture (Going Deeper with Convolutions, 2014) replaced the question "how big should this filter be?" with "why choose?" Each module runs several filter sizes in parallel and concatenates the results. The 1×1 convolutions that make this affordable are the idea that outlived the paper.

The idea

An Inception module is a small parallel circuit with four branches:

• a 1×1 convolution,
• a 1×1 convolution followed by a 3×3 convolution,
• a 1×1 convolution followed by a 5×5 convolution,
• a 3×3 max pool followed by a 1×1 convolution,

and the four branch outputs are concatenated along the channel axis. The 1×1 convolutions before the 3×3 and 5×5 are bottlenecks: they reduce channel depth so the expensive spatial convolutions run on fewer channels. That is what makes a 5×5 affordable at all.

The paper's own numbers for one module: the 1×1 branch outputs 64 channels, the 3×3 branch reduces to 96 and outputs 128, the 5×5 branch reduces to 16 and outputs 32, and the pooling branch outputs 32. The concatenated output is 64 + 128 + 32 + 32 = 256 channels.

Worked example

Compare a direct 5×5 convolution on 192 input channels producing 32 outputs against the bottleneck version:

The bottleneck is roughly 10× cheaper for the same output channel count. It also inserts a nonlinearity in the middle of the path, which is a side benefit rather than a cost.

In code

Building the branches by hand is the clearest way to see the structure:

import tensorflow as tf

def inception_module(x, f1, r3, f3, r5, f5, pool_proj):
    b1 = tf.keras.layers.Conv2D(f1, 1, padding="same", activation="relu")(x)

    b2 = tf.keras.layers.Conv2D(r3, 1, padding="same", activation="relu")(x)
    b2 = tf.keras.layers.Conv2D(f3, 3, padding="same", activation="relu")(b2)

    b3 = tf.keras.layers.Conv2D(r5, 1, padding="same", activation="relu")(x)
    b3 = tf.keras.layers.Conv2D(f5, 5, padding="same", activation="relu")(b3)

    b4 = tf.keras.layers.MaxPool2D(3, strides=1, padding="same")(x)
    b4 = tf.keras.layers.Conv2D(pool_proj, 1, padding="same", activation="relu")(b4)

    return tf.keras.layers.Concatenate()([b1, b2, b3, b4])

inp = tf.keras.Input(shape=(224, 224, 3))
y = inception_module(inp, 64, 96, 128, 16, 32, 32)   # [None, 224, 224, 256]

padding="same" with strides=1 on the pooling branch keeps all four branches at the same height and width, which Concatenate requires. For real work, do not build this by hand:

base = tf.keras.applications.InceptionV3(
    weights="imagenet", include_top=False, input_shape=(299, 299, 3))

Note the input size: InceptionV3 takes 299×299, not the 224×224 that VGG uses.

Check yourself

1. Why does a 1×1 bottleneck make a 5×5 convolution affordable?
2. What must be true of all four branches for Concatenate to work?
3. Why does include_top=False matter when you want to attach your own classifier?

Key takeaways

• Inception runs multiple filter sizes in parallel and concatenates their channels.
• 1×1 convolutions act as bottlenecks that cut compute by roughly an order of magnitude.
• tf.keras.applications.InceptionV3 is the supported way to use it, and it expects 299×299 input.