Authors
Yamin Zhang, Fabing Duan, François Chapeau-Blondeau, Derek Abbott
Published in
Physical review. E. Volume 114. Issue 3-2. Pages 035304.
Abstract
In conventional generative adversarial networks (GANs), the discriminator typically employs a sigmoid activation to map features to probabilities. However, this activation suffers from the vanishing-gradient problem, which can lead to training instability and mode collapse. This work reformulates the final-layer activation of the discriminator as cumulative distribution functions (CDFs) of random variables, thereby introducing a learnable noise-scale parameter. We show analytically that, for CDF families with bounded reversed hazard rates in the negative tail, a noise-scale value smaller than unity scales the gradient of both the saturating and nonsaturating generator losses by a factor proportional to its reciprocal. Rather than fully resolving the vanishing-gradient problem, this mechanism provides more informative gradient signals to the generator when the discriminator is close to its optimum. Experiments on a two-dimensional Gaussian mixture and on MNIST show that the proposed CDF-based activation effectively improves mode coverage and training stability. Meanwhile, experiments on CIFAR-10 using a deep convolutional GAN (DCGAN) indicate that, when gradient flow is already stabilized by architectural features, the choice of final-layer activation plays a more limited role. An ablation study on MNIST further distinguishes the benefits of the proposed method from that of conventional noise regularization techniques. These results demonstrate that the proposed CDF-based activation also contributes a meaningful scheme for stabilizing adversarial training.
PMID:
42888738
Bibliographic data and abstract were imported from PubMed on 17 Oct 2026.
Read full publication at:
Please sign in
to see all details.
Advertisement
Stats
- Recommendations n/a n/a positive of 0 vote(s)
- Views 10
- Comments 0