Skip to main content
W&B Fully Connected
>
Articles

How to Implement Deep Convolutional Generative Adversarial Networks (DCGAN) in Tensorflow

In this short tutorial, we explore how to implement Deep Convolutional Generative Adversarial Networks in Tensorflow, with a Colab to help you follow along.
Saurav Maheshkar
Created on April 5|Last edited on January 18
In this article, we'll walk through how you can implement a Deep Convolutional Generative Adversarial Network using the Tensorflow Framework.
First introduced in the paper "Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks" by Alec Radford, Luke Metz and Soumith Chintala, it was a simple extension of the original GAN paper which exclusively used Convolutional Blocks as the core components of the Discriminator and the Generator.
This Report is a mirror of another report written to implement DCGANs in PyTorch. You can find that Report here.
💡
This post assumes a basic understanding of Generative Adversarial Networks.
💡
Source: "Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks" by Alec Radford, Luke Metz, and Soumith Chintala
Lastly, before jumping in, if you'd like to follow along with this piece in a Colab with executable code, you can find that right here:


Here's what we'll cover:

Table of Contents

ImplementationResultsSummaryRecommended Reading


Let's dive in!

Implementation

The strategy to train any GAN (at a high level) is quite simple; we first have the Generator (in this case, a Convolutional Neural Network (CNN)) create some images based, calculate the generator's loss and perform backpropagation; then we feed these generated images and real images to the discriminator (again a CNN), combine the losses (from the real images of the dataset and the generated images from the generator) and then perform backpropagation.
This way we train two networks, one to generate images (the generator) and another which learns to generate between real images and those generated by our other network. Both networks have their own optimizers and are usually evaluated using the same loss function.
In this case we use the Cross Entropy Loss for both the Generator and the Discriminator. For more details, refer to this post.
💡
Let's see an example training loop in Tensorflow:
# Initialize generator and discriminator
generator = make_generator_model()
discriminator = make_discriminator_model()

# Optimizers
generator_optimizer = tf.keras.optimizers.Adam(1e-4)
discriminator_optimizer = tf.keras.optimizers.Adam(1e-4)

def train_step(images):
    # Generate Random Noise to feed to the Generator
    noise = tf.random.normal([config.BATCH_SIZE, config.noise_dim])

    with tf.GradientTape() as gen_tape, tf.GradientTape() as disc_tape:
      generated_images = generator(noise, training=True)

      real_output = discriminator(images, training=True)
      fake_output = discriminator(generated_images, training=True)

      gen_loss = generator_loss(fake_output)
      disc_loss = discriminator_loss(real_output, fake_output)

      wandb.log({"Discriminator Loss": disc_loss.numpy(), "Generator Loss": gen_loss.numpy()})

    # Calculate Gradients
    gradients_of_generator = gen_tape.gradient(gen_loss, generator.trainable_variables)
    gradients_of_discriminator = disc_tape.gradient(disc_loss, discriminator.trainable_variables)
 
    # Update Optimizers
    generator_optimizer.apply_gradients(zip(gradients_of_generator, generator.trainable_variables))
    discriminator_optimizer.apply_gradients(zip(gradients_of_discriminator, discriminator.trainable_variables))

def train(dataset, epochs):
  for epoch in range(epochs):
    for image_batch in dataset:
      train_step(image_batch)



Results

Now that you've seen how to implement the training strategy, let's see how the use of Weights & Biases allows us to easily visualize important metrics and compare them using Panels. For example, here's a quick comparison of the Discriminator loss grouped by the noise dimension, which you'll find linked in the Colab above:

Run set
2

As we can see from the plots, a noise dimension of 100 leads to a lower loss probably because it allows for more learned information, you can also try hyperparameter tuning by changing the batch size or the image size.
Weights & Biases Sweeps makes this incredibly easy by automatically running your pipeline using an agent. For more details, please refer to our Sweeps Quickstart Guide.
If you'd like to try this yourself, here's the Colab to do so:



Summary

In this article, you saw how you can implement Deep Convolutional Generative Adversarial Networks using the Tensorflow Framework and how the use of Weights and Biases allows you to easily visualize important metrics. To see the full suite of W&B features, please check out this short 5 minutes guide.
If you want more reports covering the math and "from-scratch" code implementations, let us know in the comments below or on our forum ✨!
Check out these other reports on Fully Connected covering other fundamental development topics like GPU Utilization and Saving Models.

Tags: GenAI, GAN, DCGAN, Beginner, Articles, Plots, Computer Vision, Tutorial

Created with ❤️ on Weights & Biases.

https://wandb.ai/generative-adversarial-networks/dcgan-tensorflow/reports/How-to-Implement-Deep-Convolutional-Generative-Adversarial-Networks-DCGAN-in-Tensorflow--VmlldzoxNzkzNDg5
Made with Weights & Biases. Sign up or log in to create reports like this one.
Iterate on AI agents and models faster. Try Weights & Biases today.
Sign upTry W&B now