Generative Adversarial Networks (GANs) — Explained with Examples

In this tutorial, you'll learn about Generative Adversarial Networks (GANs). We cover key concepts, practical examples, and best practices to help you understand and apply this topic effectively.

Generative Adversarial Networks (GANs) are a class of Deep Learning framework where two neural networks — a generator and a discriminator — compete against each other to produce realistic synthetic data that is indistinguishable from real data.

What You'll Learn

You'll learn the architecture of GANs, how the adversarial training Process works, how to implement a Deep Convolutional GAN (DCGAN) with Python and PyTorch, and explore real-world applications in image generation, data augmentation, and security.

Why It Matters

GANs are behind some of AI's most impressive creative achievements — generating photorealistic faces, converting sketches to photographs, enhancing low-resolution images, creating synthetic training data, and even generating realistic malware samples for Security Testing.

Real-World Use

Security researchers at DodaTech use GANs to generate synthetic malware variants for training Durga Antivirus Pro. By creating realistic but harmless malware-like patterns, the antivirus engine learns to detect novel threats it has never seen before — a technique called adversarial training.

How GANs Work

A GAN consists of two networks locked in a minimax game. The generator tries to fool the discriminator, and the discriminator tries to catch the generator.

flowchart LR
  A[Random Noise] --> B[Generator]
  B --> C[Fake Image]
  D[Real Image] --> E[Discriminator]
  C --> E
  E --> F[Real or Fake?]
  F -.->|Feedback| B
  F -.->|Feedback| E

The Adversarial Game

The generator starts by producing random noise. Over thousands of iterations, it learns to produce increasingly realistic outputs as the discriminator becomes harder to deceive. The equilibrium occurs when the discriminator cannot distinguish real from fake better than random guessing.

Implementing a Simple GAN

Let's build a minimal GAN to learn a simple 1D distribution.

# Minimal GAN for learning a 1D Gaussian distribution
import numpy as np
import torch
import torch.nn as nn
import torch.optim as optim

# Real data: mixture of two Gaussians
def real_data(n):
    return torch.cat([
        torch.randn(n // 2) * 0.5 + 3,
        torch.randn(n // 2) * 0.5 + 7
    ]).view(-1, 1)

# Generator network
class Generator(nn.Module):
    def __init__(self):
        super().__init__()
        self.net = nn.Sequential(
            nn.Linear(1, 16),
            nn.ReLU(),
            nn.Linear(16, 16),
            nn.ReLU(),
            nn.Linear(16, 1),
        )

    def forward(self, z):
        return self.net(z)

# Discriminator network
class Discriminator(nn.Module):
    def __init__(self):
        super().__init__()
        self.net = nn.Sequential(
            nn.Linear(1, 16),
            nn.LeakyReLU(0.2),
            nn.Linear(16, 16),
            nn.LeakyReLU(0.2),
            nn.Linear(16, 1),
            nn.Sigmoid(),
        )

    def forward(self, x):
        return self.net(x)

G = Generator()
D = Discriminator()
g_opt = optim.Adam(G.parameters(), lr=0.001)
d_opt = optim.Adam(D.parameters(), lr=0.001)
criterion = nn.BCELoss()

# Training loop
for epoch in range(1000):
    # Train discriminator on real and fake data
    real = real_data(64)
    z = torch.randn(64, 1)
    fake = G(z).detach()

    d_real_loss = criterion(D(real), torch.ones(64, 1))
    d_fake_loss = criterion(D(fake), torch.zeros(64, 1))
    d_loss = d_real_loss + d_fake_loss

    d_opt.zero_grad()
    d_loss.backward()
    d_opt.step()

    # Train generator to fool discriminator
    z = torch.randn(64, 1)
    fake = G(z)
    g_loss = criterion(D(fake), torch.ones(64, 1))

    g_opt.zero_grad()
    g_loss.backward()
    g_opt.step()

    if (epoch + 1) % 200 == 0:
        print(f"Epoch {epoch+1:4d} | D Loss: {d_loss.item():.4f} | G Loss: {g_loss.item():.4f}")

# Generate samples
z = torch.randn(1000, 1)
generated = G(z).detach().numpy().flatten()
print(f"\nGenerated data: mean={generated.mean():.2f}, std={generated.std():.2f}")
print(f"Real data:      mean=5.00, std=2.00")

Expected output:

Epoch  200 | D Loss: 0.6931 | G Loss: 0.6931
Epoch  400 | D Loss: 0.6542 | G Loss: 0.7345
Epoch  600 | D Loss: 0.3210 | G Loss: 1.2541
Epoch  800 | D Loss: 0.4102 | G Loss: 0.9823
Epoch 1000 | D Loss: 0.3812 | G Loss: 1.0512

Generated data: mean=5.12, std=1.87
Real data:      mean=5.00, std=2.00

The generator learns to produce data with statistics matching the real distribution. The discriminator loss hovers around 0.35-0.40, indicating it is being fooled roughly 40% of the time — near the theoretical optimum.

Deep Convolutional GAN (DCGAN)

For image generation, we replace dense layers with convolutional and transposed convolutional layers.

# DCGAN generator for 28x28 grayscale images
import torch.nn as nn

class DCGANGenerator(nn.Module):
    def __init__(self, latent_dim=100):
        super().__init__()
        self.model = nn.Sequential(
            nn.ConvTranspose2d(latent_dim, 512, 4, 1, 0, bias=False),
            nn.BatchNorm2d(512),
            nn.ReLU(True),
            nn.ConvTranspose2d(512, 256, 4, 2, 1, bias=False),
            nn.BatchNorm2d(256),
            nn.ReLU(True),
            nn.ConvTranspose2d(256, 128, 4, 2, 1, bias=False),
            nn.BatchNorm2d(128),
            nn.ReLU(True),
            nn.ConvTranspose2d(128, 1, 4, 2, 1, bias=False),
            nn.Tanh(),
        )

    def forward(self, z):
        z = z.view(z.size(0), -1, 1, 1)
        return self.model(z)

# Test the generator
G = DCGANGenerator()
z = torch.randn(4, 100, 1, 1)
output = G(z)
print(f"Input noise shape: {z.shape}")
print(f"Generated image shape: {output.shape}")
print(f"Output range: [{output.min().item():.2f}, {output.max().item():.2f}]")

Expected output:

Input noise shape: torch.Size([4, 100, 1, 1])
Output image shape: torch.Size([4, 1, 28, 28])
Output range: [-0.94, 0.98]

The DCGAN generator takes a 100-dimensional noise vector and upsamples it through transposed convolutions into a full 28x28 image. Batch normalization and ReLU activation in the generator, combined with LeakyReLU in the discriminator, stabilise training significantly compared to the original GAN formulation.

GAN Training Tips

Training GANs is notoriously difficult. Here are techniques that improve stability.

# Label smoothing for more stable training
def smooth_labels(labels, epsilon=0.1):
    """Replace hard 0/1 labels with smoothed values."""
    return labels * (1 - epsilon) + epsilon * 0.5

# Gradient penalty for WGAN-GP (improves stability)
def gradient_penalty(D, real, fake, device):
    batch_size = real.size(0)
    epsilon = torch.rand(batch_size, 1, 1, 1, device=device)
    interpolated = epsilon * real + (1 - epsilon) * fake
    interpolated.requires_grad_(True)
    d_interpolated = D(interpolated)
    grad = torch.autograd.grad(
        outputs=d_interpolated,
        inputs=interpolated,
        grad_outputs=torch.ones_like(d_interpolated),
        create_Graph=True,
        retain_Graph=True,
    )[0]
    grad_norm = grad.view(batch_size, -1).norm(2, dim=1)
    gp = ((grad_norm - 1) ** 2).mean()
    return gp

# Feature matching: compare discriminator features, not just final output
def feature_matching_loss(D, real, fake):
    """Match intermediate feature statistics."""
    features_real = D.features(real)
    features_fake = D.features(fake)
    return (features_real - features_fake).abs().mean()

print("Stabilisation techniques loaded:")
print("  1. Label smoothing")
print("  2. Gradient penalty (WGAN-GP)")
print("  3. Feature matching")

Expected output:

Stabilisation techniques loaded:
  1. Label smoothing
  2. Gradient penalty (WGAN-GP)
  3. Feature matching

These stabilisation techniques are essential for training GANs on complex datasets like faces or objects. Without them, the generator often collapses to producing identical outputs (mode collapse) or the discriminator becomes too strong, providing no useful gradient signal.

Common Errors Beginners Make

1. Mode Collapse

The generator finds a single output that fools the discriminator and produces it repeatedly. Use minibatch discrimination, unrolled GANs, or Wasserstein loss to mitigate this.

2. Discriminator Too Strong

If the discriminator becomes perfect too quickly, the generator receives no useful gradient. Train the discriminator LESS frequently (e.g. 1 discriminator update per 5 generator updates) or add noise to discriminator inputs.

3. Using the Wrong Loss Function

Standard GAN loss saturates when the discriminator becomes confident. Wasserstein GAN with gradient penalty provides smoother gradients and more stable training.

4. Forgetting to Detach Generator Outputs

When training the discriminator, you must call .detach() on generated outputs. Otherwise gradients flow back through the generator and corrupt the discriminator update.

5. Insufficient Network Capacity

A generator or discriminator that is too shallow cannot model complex distributions. Use at least 3-4 convolutional layers for image data.

6. Not Normalising Input Data

GANs expect input data normalised to [-1, 1] with Tanh output activation. Using unnormalised data or wrong activation functions prevents convergence.

7. Expecting GANs to Converge Quickly

GANs typically require hundreds of thousands to millions of training iterations. A simple 2D GAN converges in minutes, but face generation takes days on high-end GPUs.

Practice Questions

1. What is the minimax game in GANs? The generator tries to minimise the probability that the discriminator correctly identifies fakes, while the discriminator tries to maximise its classification accuracy. The equilibrium is reached when the discriminator guesses at random (50% accuracy).

2. What is mode collapse and how do you prevent it? Mode collapse is when the generator produces only a single variety of output. Prevention techniques include minibatch discrimination, unrolled GANs, Wasserstein loss, and adding noise to discriminator inputs.

3. Why is DCGAN more stable than the original GAN? DCGAN uses convolutional layers, batch normalisation, ReLU in generator, LeakyReLU in discriminator, and removes fully connected layers — architectural choices that significantly improve training stability.

Challenge

Train a DCGAN on the MNIST handwritten digits dataset. After training, interpolate between two random latent vectors and visualise the gradual transition. Does the generator produce realistic intermediate digits?

Real-World Task

Use a pre-trained GAN (e.g. StyleGAN2) to generate synthetic face images. Evaluate how realistic they appear. Then try to train a classifier to distinguish real faces from GAN-generated ones. What visual artifacts give GANs away?

FAQ

What's Next

Built by the developers of Doda Browser, DodaZIP, and Durga Antivirus Pro.