A Consideration on Ternary Adversarial Generative Networks

Generative adversarial networks (GANs), which can generate and transform data, have been attracting attention. However, the model must be lightweight and fast when applied in the field. As for the ternarization of GAN, (TernaryGAN,) which restricts the value of the weights to {-1, 0, +1} during forwarding propagation of the generator, has already been proposed. In this paper, we investigated by experiment how ternary generator and/or discriminator affects the training of GANs. To make not only generator ternary, but also discriminator, we propose the DGR (Decomposition with Gradient Retained) method, which can change discriminator's input images to binary. We trained GANs for the cases where the generator and the discriminator are ternarized, and for the case where only one of them is ternarized, and measured the degree of image degradation using the FID (Fréchet inception distance) score. Only the ternarized generator is showed the lowest accuracy degradation, implying that GANs contain some parts that are not suitable for ternarization. We found the useful insight that when reducing the weight of GAN, the generator can be compressed relatively more, while the discriminator should not be so much.