Deep Dream

DeepDream is activation maximization applied to a photograph instead of random noise. The model hallucinates the patterns its filters were already detecting, turning a cloudy sky into eyes and feathers. It is the clearest way to see what a pretrained network considers a feature.

The idea

Two choices define a DeepDream:

• Which layer you maximize. Shallow layers strengthen edges and textures; deep layers strengthen whole objects.
• Octaves. The image is processed at several scales, resizing up between passes so large features can emerge, then blending the result back down.

The loss is the sum of a chosen layer's activation tensor:

Gradients flow to the input, not to the weights. The critical difference from training is scale: the update applies to a single image with a small step, and the image is blurred after each step so structures stay coherent. Without blur, the optimizer finds the same adversarial speckle as plain activation maximization.

Worked example

Use InceptionV3 with include_top=False, accepting (None, None, 3) inputs.

• A 512×512 image enters at scale 1.0.
• Choose mixed4, a mid-level layer roughly 32×32 spatially.
• Loss is tf.reduce_mean(layer_output).
• Gradient ascent on the image with steps=100 and step_size=0.01.
• Each step: update, clip to [-1, 1], blur with a Gaussian.
• Between octaves, resize the image by 1.4 and re-run the same loop.

The interesting part is not the final pixels but the set of layers. Maximizing mixed4 paints repeated mid-level motifs; maximizing mixed10 produces larger, object-like blobs.

In code

import tensorflow as tf

base = tf.keras.applications.InceptionV3(include_top=False, weights="imagenet")
dream = tf.keras.Model(base.inputs, base.get_layer("mixed4").output)

raw = tf.io.decode_image(tf.io.read_file("sky.jpg"))
img = tf.Variable(tf.image.resize(raw, (512, 512))[None])
img.assign(tf.keras.applications.inception_v3.preprocess_input(img))

for step in range(100):
    with tf.GradientTape() as tape:
        loss = tf.reduce_mean(dream(img))
    grad = tape.gradient(loss, img)
    img.assign_add(0.01 * grad / (tf.reduce_mean(tf.abs(grad)) + 1e-8))
    img.assign(tf.clip_by_value(img, -1.0, 1.0))
    img.assign(tf.nn.avg_pool2d(img, 3, 1, "SAME"))

The normalizer keeps each step's magnitude fixed regardless of how large the raw gradient happens to be.

Check yourself

1. Why does DeepDream amplify existing features instead of inventing objects from nothing?
2. What does the octave loop buy you that a single scale does not?
3. What happens to the image if you remove the blur step?

Key takeaways

• DeepDream is activation maximization on an existing image with a frozen network.
• Layer choice controls whether you get textures or objects; octaves control scale.
• Blurring and gradient normalization are what keep the result coherent.