An Introduction to Adversarial Examples in Deep Learning

An ELI5 Introduction to Adversarial Examples

Figure 1: To our eyes, the figures are identical, but to an image classifier, they are not the same. This is an example of an adversarial example.

Table of Contents

What Are Adversarial Examples?

We start by building our favorite network architecture.
We define the hyperparameters like the number of epochs, batch size, optimizer, etc.
We then feed batches of preprocessed images as tensors to our network.
We measure the loss of the network with the ground-truth labels and the predicted labels.
We then calculate the gradients of the loss function with respect to the network’s learnable parameters.
We backpropagate these gradients to update the learnable parameters of our network such that the loss is minimized.

Figure 2: Pictorial overview of the training process in deep learning (source).

We then calculate the gradients of the loss function with respect to the network’s learnable parameters.
We backpropagate these gradients to update the learnable parameters of our network such that the loss is minimized.

We do not calculate the gradients for the input data.
We perform the parameter updates such that the loss (aka training objective) is minimized.

What if we calculated the gradients with respect to the input data?
What if the input data were updated (with the gradients) to maximize the loss?

We need to update the input data so that its semantic characteristics do not change much.
We are still able to maximize the loss.

Why care about gradients only? Because this quantity will inform us how to nudge the input data to affect the loss function.

Figure 3: Pictorial overview of the FGSM method (source).

Types of Adversarial Examples

Natural Adversarial Examples (!)

Figure 4: On the left-most corner of each image, we see the ground-truth label, and on the right-most corner, we see the predictions of a standard CNN-based classifier (source).

[…] DenseNet-121 obtains around 2% accuracy, an accuracy drop of approximately 90% […]

For multi-class classification problems, the classifiers are trained to associate an entire image to a discrete class label, lake terrier, for example. Due to this, the background objects present in an image also get associated with the ground-truth class label, leading to catastrophic failure modes, as shown above.
Humans are more used to using shape as an image descriptor to classify images. However, it does not seem to be the case for the classifiers - they heavily rely on attributes like color and texture. This point becomes even clearer with examples presented in the top-center of Figure 4.

Geirhos et al. studied this phenomenon in their paper ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustness. They first stylize the ImageNet-1K training images to impose bias toward shape in the CNN-based classifiers. Unfortunately, this idea does not transfer well in the case of the ImageNet-A dataset.

Taming the Input Data Points as Attacks (Synthetic)

What if we calculated the gradients with respect to the input data? What if the input data were updated (with the gradients) to maximize the loss?

# Load ResNet50
resnet50 = tf.keras.applications.ResNet50(weights="imagenet")

# Run inference
preds = resnet50.predict(preprocessed_image)

# Process the predictions
...

Figure 5: A sample image for running inference with an image classifier.

cce = tf.keras.losses.SparseCategoricalCrossentropy()
loss = cce(
    tf.convert_to_tensor([341]),
    tf.convert_to_tensor(preds)
)
print(loss.numpy()) 
# 0.0004157156

scc_loss = tf.keras.losses.SparseCategoricalCrossentropy()
    
for t in range(50):
    with tf.GradientTape() as tape:
        tape.watch(delta)
        inp = preprocess_input(image_tensor + delta)
        predictions = resnet50(inp, training=False)
        loss = - scc_loss(
                tf.convert_to_tensor([341]),
                predictions
            )
        if t % 5 == 0:
            print(t, loss.numpy())

    # Get the gradients
    gradients = tape.gradient(loss, delta)
    
    # Update the weights
    optimizer.apply_gradients([(gradients, delta)])

    # Clip so that the delta values are within [0,1]
    delta.assign_add(clip_eps(delta))

# Clipping utility so that the pixel values stay within [0,1]
EPS = 2./255

def clip_eps(delta_tensor):
    return tf.clip_by_value(delta_tensor, clip_value_min=-EPS, clip_value_max=EPS)

This form of gradient descent-based optimization is also referred to as Projected Gradient Descent.

Figure 6: A synthetically generated adversarial example.

# Generate prediction
perturbed_image = preprocess_input(image_tensor + delta)
preds = resnet50.predict(perturbed_image)
print('Predicted:', tf.keras.applications.resnet50.decode_predictions(preds, top=3)[0])

loss = - scc_loss(
    tf.convert_to_tensor([341]),
    predictions
)

loss = (- scc_loss(tf.convert_to_tensor([true_index]), predictions) + 
    scc_loss(tf.convert_to_tensor([target_index]), predictions))

Figure 7: A synthetically generated adversarial example (targeted this time).

Figure 8: The learned delta vector.

A Note On Optimizer Susceptibility to the Synthetic Attacks

Grad-CAM on Perturbed Images

Conclusion

References

Adversarial Robustness - Theory and Practice (NeurIPS 2018 Tutorial)
Read
Watch
Chapter 10 (Adversarial examples), GANs in Action
Introduction to Adversarial Machine Learning
Adversarial example using FGSM