Adversarial Examples

An adversarial example is an input that looks unchanged to a human but makes a classifier confidently wrong. The trick is to use the model's gradient with respect to the input instead of the weights. Adversarial examples are the cleanest demonstration that a network's decision surface is not what its accuracy number suggests.

The idea

Optimization normally updates parameters to reduce loss. An adversarial attack inverts the roles: hold the weights fixed and update the input to increase the loss for the true label. The fast gradient sign method (FGSM) takes one step in that direction:

Then clip the result back into the valid pixel range. Three properties make this both useful and unsettling:

• One step is enough. A single gradient evaluation can flip a prediction.
• The change is tiny. With inputs scaled to [0, 1], $\epsilon = 0.03$ moves each pixel by at most 3% of full range.
• It transfers. A perturbation computed on one model often fools another, which is why black-box attacks work at all.

A targeted attack replaces the true-label loss with the loss for a chosen wrong label and moves down that gradient instead.

Worked example

Sign-gradient on a two-pixel toy. Suppose $x = [0.5, 0.5]$, the true label is 3, and the input gradient is $\nabla_x J = [0.8, -0.6]$. With $\epsilon = 0.1$:

The first-order change in loss is $\nabla_x J \cdot (x' - x) = 0.8(0.1) + (-0.6)(-0.1) = 0.14$, an increase of about 0.14 for a move of 0.1 per pixel. Repeat that logic at MNIST scale and the same procedure pushes a confidently correct image across a decision boundary.

In code

import tensorflow as tf

loss_fn = tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True)

@tf.function
def fgsm(model, x, y, epsilon=0.03):
    x = tf.cast(x, tf.float32)
    with tf.GradientTape() as tape:
        tape.watch(x)
        logits = model(x, training=False)
        loss = loss_fn(y, logits)
    grad = tape.gradient(loss, x)
    x_adv = x + epsilon * tf.sign(grad)
    return tf.clip_by_value(x_adv, 0.0, 1.0)

adv = fgsm(model, x_test[:32], y_test[:32])
metrics = model.evaluate(adv, y_test[:32], verbose=0)

Note what is missing: tape.gradient(loss, x) differentiates with respect to the input, so x must be watched because it is a tensor rather than a variable. If your model contains a preprocessing layer that rescales inputs, attack the preprocessed space, or the perturbation will be partly undone when the model reads it back.

Adversarial training — retraining on adversarial examples — is the best-known defense, but it is not free: you generate attacks inside the training loop, and robustness to one $\epsilon$ does not transfer to another.

Check yourself

1. Why does FGSM use sign instead of the raw gradient, and what does $\epsilon$ control?
2. How would you change fgsm to make a targeted attack against class 7?
3. Why does adversarial training not simply fix the problem for all perturbations?

Key takeaways

• Adversarial examples exploit gradients with respect to the input, not the weights.
• FGSM is one signed step of size $\epsilon$, followed by clipping to the valid pixel range.
• Perturbations transfer between models, so defenses have to be built into training.