French developer Diagné Cyril (Cyril Diagne) has developed a mobile application that allows you to scan objects using the camera, and then transferred them into Adobe Photoshop. Code and documentation of the project available on GitHub, and a brief description of the developer of published on Twitter.
Augmented reality allows to overlay virtual objects onto the real world like they really are. Some researchers in the field of software and interfaces propose to implement with augmented reality two-way communication between real and virtual objects. For example, Adobe recently showed a prototype of an application that allows you to see instructor comments superimposed on the textbook, and add your overlay layers in response. But even this app still based on virtual objects, albeit superimposed on a real book.
Diagné Cyril (Cyril Diagne) presented a prototype application that is able to find objects in front of the camera of your smartphone, copy them and paste them to the desired area of the graphic editor. The prototype consists of three components: a mobile application, a local application on your computer and server.
In a mobile application, the user points the camera at interesting his subject and tap the screen. Then for two and a half seconds on the screen there is a carved object, which you can restore on the computer screen with an open file in Adobe Photoshop. Then again to press on the screen and four seconds to cut the object is inserted as a new layer to the specified location.
App on the computer works as a link between the editor and smartphone. It is using the method scale-invariant transformation of signs finds a match between the screen image from the camera, and screenshot from the same screen. Thanks to this carved object is transferred exactly to the place specified by the user.
The third component running on a remote server because it is responsible for resource-intensive calculations. The server runs the neural network BASNet, which takes smartphone original photo determines the object and separates it from the background and returns the image with the cut out object in Photoshop. For simplicity, the developer has published a neural network in the form of a container for Knative, which communicates with the app on your phone and Photoshop on the computer via HTTP.
The developer notes that he wrote the app quickly, so it is poorly optimized. For example, it proposes to determine and to cut objects directly on the smartphone using the algorithm Deeplab implemented on the mobile framework TensorFlow Lite. In addition, he said that in the same way you can insert objects not only in Photoshop but in other programs or the operating system itself.
Diagné recently created another app-prototype. It is a Chrome browser extension that turns pictures in instagram in three-dimensional and animated. It is based on the presented recently the algorithm, realistic dorisovyvaet background behind objects in photos.