Google introduced the open method of automatic cropping of the video. The user sets basic parameters such as the desired aspect ratio, and the algorithm determines the most important objects in the scene and crops the video so that they remain in a new video, tell the developers blog Google AI. Code and instructions for running a program posted on GitHub.
Historically, almost all equipment for shooting and viewing video had a frame width greater than height. Because of this movies are almost always shot in horizontal format. However, due to the mass adoption of smartphones and social networking a significant part of the video now has a vertical format. This led to the fact that the creators of the videos often need to adapt the clips for several popular ratios of the screen. If you just crop the video in the center, then some of the objects on the edges will be lost, especially in the case of long rollers.
Because there already exists many algorithms for tracking objects in the frame, some of the developers in the last few years trying to use them for framing. For example, such a feature recently introduced by Adobe, but it has limitations, and in addition, access is restricted. Programmers at Google have created an open algorithm that can trim your videos to different formats without losing important parts of the animation.
In the beginning the user provides the algorithm of the source video and the aspect ratio for the resulting video, and can also specify whether you want to save all of the objects in the frame. After this, the algorithm lays out the original video on a different scene using the saturation histogram: if the histogram between the frames has changed dramatically, it probably means there was a change of scene.