Instance Segmentation Based Background Reference Frame Generation for Surveillance Video Coding

Efficient intelligent analysis and video compression are critical modules in terms of the advanced surveillance system. However, existing solutions always deal each task with independent strategies, leading to low-efficiency of the surveillance system. In this paper, we propose to handle these two tasks in a hybrid manner. In particular, a hybrid surveillance processing scheme towards efficient analysis and compression is presented, where the extracted semantic information can not only be utilized in intelligent analysis tasks, but also used to improve compression efficiency by facilitating the background reference frame (BRF) generation. Moreover, we propose to remove background redundancy of surveillance video by introducing the high quality BRF, where motion metric and semantic metric work in a complementary way to ensure the accurate detection of background blocks. Experimental results manifest considerable advantages of the proposed BRF. When the proposed BRF is integrated into reference picture set (RPS), 3% coding gains are obtained compared with state-of-the-art method.