Rate distortion optimization with adaptive content modeling for random-access versatile video coding

In this paper, we study the capability of improving the rate-distortion (RD) performance based on the adaptive content modeling in the Versatile Video Coding (VVC) standard. In particular, the frame-level dependent relationships and inherent RD relationship are explored. As such, the rate dependency, distortion dependency and inherent RD characteristics are fully utilized in the global rate distortion optimization (RDO) process, and the quantization parameter (QP) for each frame could be adaptively solved. To facilitate the adaptive QP calculation, a two-pass coding strategy is proposed. In the first-pass coding, the video statistics are sufficiently collected with the proposed Dual Motion Compensation and Residual Coding (DMCRC) method to generate the parameters for the content-aware models. During the second-pass coding, the optimal QP at the frame level is obtained by optimizing the global RD performance with the dependent and inherent models. The proposed algorithm is implemented on VVC test model (VTM-4.0) and achieves significant performance gain for test sequences with constant and varying scenes.