Hybrid Laplace Distribution-Based Low Complexity Rate-Distortion Optimized Quantization

Rate distortion optimized quantization (RDOQ) is an efficient encoder optimization method that plays an important role in improving the rate-distortion (RD) performance of the high-efficiency video coding (HEVC) codecs. However, the superior performance of RDOQ is achieved at the expense of high computational complexity cost in two stages RD minimization, including the determination of optimal quantized level among available candidates for each transformed coefficient and the determination of best quantized coefficients for transform units with the minimum total cost, to softly optimize the quantized coefficients. To reduce the computational cost of the RDOQ algorithm in HEVC, we propose a low-complexity RDOQ scheme by modeling the statistics of the transform coefficients with hybrid Laplace distribution. In this manner, specifically designed block level rate and distortion models are established based on the coefficient distribution. Therefore, the optimal quantization levels can be directly determined by optimizing the RD performance of the whole block, while the complicated RD cost calculations can be eventually avoided. Extensive experimental results show that with about 0.3%-0.4% RD performance degradation, the proposed low-complexity RDOQ algorithm is able to reduce around 70% quantization time with up to 17% total encoding time reduction compared with the original RDOQ implementation in HEVC on average.