Image guided label map propagation in video sequences

In this paper, we propose a novel method to transmit the label maps by propagating from a key frame to non-key frames. The label map of a non-key frame is initialized by warping the label map of its corresponding key frame according to the motion estimation between them. Subsequently, the initialized label map is optimized with the guidance of its texture image. The optimization process minimizes an energy function which takes two constraints into consideration: (i) the data term measuring the similarity between an estimated label map and its initialized one, (ii) the regularization term enforcing the local smoothness in the label map and the consistency of region boundaries between the estimated label map and its corresponding texture image. Graph cuts based computation process is finally performed to generate the optimized label map. Experimental results show that our method achieves higher accuracy and better visual quality comparing with the state-of-the-art method.