The adversarial discriminator was first used in the context of the GAN for generating random samples, but it can also be used as a prior that favors realism in tasks that translate one data example into another. This is most commonly done with images, where we might want to translate a grayscale image to color, a noisy image to a clean one, a blurry image to a sharp one, or a sketch to a photo-realistic image.
Here we discuss three image translation models that use different amounts of manual labeling. The Pix2Pix model uses before/after pairs for training. Models with adversarial losses use before/after pairs for the main model but also exploit unpaired “after” images in the discriminator. The CycleGAN model uses unpaired images.
Note that in these image generation tasks, we don’t really have a “generator” in the sense of generating from a random latent. Instead, these are more like supervised learning tasks with an adversarial loss that encourages realism.