Researchers from Adobe has created a program that allows you to alter various components of the image: for example, to remove wrinkles from the face, paint the background or to change the orientation of the head. For this they used a generative-adversarial neural networks and smashed their performance by the principal component method: it managed to highlight certain aspects of the images and change them individually. A Preprint describing the algorithm posted on the website arXiv.org.
Generative-adversarial networks now allow not only to create images from scratch, guided by the training sample, but also to change some parameters of the images. A good example is the transfer of styles: a year ago the company NVIDIA, for example, showed an algorithm that allows you to change the types of individual animals in the image, without spending a lot of the data the training sample.
The ability of such algorithms is not very wide: NVIDIA algorithm works on the basis of the change object classes in the image, but to correct some small details (for example, to increase the facility to change the background behind it or to tweak it a little) can not. It is theoretically generative: the competitive network can solve this problem: it is necessary to disassemble the cycle of the algorithm components and to isolate those that allow you to change certain aspects of the images.
The program, presented by developers from Adobe under the guidance of Eric Harkonen (Erik Härkönen), is based precisely on this principle. For its development, the authors considered two already trained algorithm based on generative-adversarial neural networks: StyleGAN and BigGAN — they are used just to transfer the styles.
In the analysis of neural networks the researchers broke the space of the hidden variables using the principal components method, which allows you to narrow the data set, and in this case is to highlight those components that are responsible for changing certain parameters of the image. It depends on what the picture shows: for example, if you change one face to another in the layers of the neural network can change the position of the head, color of hair, expressed emotions and wrinkles — and each of these components can be extracted and configured separately.
Based on this scientists have created a program in which, depending on the image, you can change some of its parameters. For example, to remove wrinkles on the face, change the background, to lengthen the neck of an animal to change the background image and the type of car. Each parameter in the program can be adjusted with the slider. The source code of the algorithm and running the program, the developers also laid out in open access on GitHub.
Another effective and useful application of generative-adversarial neural networks, dorisovyvanie images. One such algorithm in September showed the developers of Google.