Feature-matching based motion prediction for high efficiency video coding in cloud

Visual features of images and video frames have become pervasive and maturely developed in extensive research fields such as computer vision and visual search. For realtime retrieval applications, the compact visual features should be transmitted and stored at server side in cloud. These local feature descriptors are characterized by the invariance properties for the variances caused by camera motion, illumination changing, occlusion and different viewpoints. Inspired by these properties, the typical scale-invariant feature transform (SIFT) descriptor is leveraged to improve the video coding efficiency in this work. In particular the predicted motion using SIFT matching is used for merge mode and motion vector prediction (MVP) in the high efficiency video coding (HEVC) standard. A hierarchical motion derivation framework aiming at achieving robust and effective MVP is further proposed. Experimental results have shown that the proposed method can efficiently improve the coding performance according to the accurate feature-matching.