Co-Saliency Detection for RGBD Images Based on Multi-Constraint Feature Matching and Cross Label Propagation

Introduction:

  • Different from the most existing co-saliency methods focusing on RGB images, in this paper, we propose a novel co-saliency detection model for RGBD images, which utilizes the depth information to enhance identification of co-saliency. In our method, the intra saliency map for each image is generated by the single image saliency model [1], while the multi-constraint feature matching is utilized to capture the constraint relationship among multiple images and produce the inter saliency map. Then, the optimization scheme, namely Cross Label Propagation (CLP), is used to update the intra and inter saliency maps in a cross way. Finally, all the original and optimized saliency maps are integrated to generate the final co-saliency result. The proposed method introduces the depth information and multi-constraint feature matching to improve the performance of co-saliency detection. Moreover, our proposed method can effectively exploit any existing single image saliency model to work well in co-saliency scenarios. Experiments on two RGBD co-saliency datasets demonstrate the efficiency of our proposed model.





Paper:

  • Runmin Cong, Jianjun Lei, Huazhu Fu, Qingming Huang, Xiaochun Cao, Chunping Hou, Co-saliency detection for RGBD images based on multi-constraint feature matching and cross label propagation, IEEE Transactions on Image Processing, vol.27, no.2, pp. 568-579, 2018. [PDF]

RGBD Cosal150 Dataset:

  • We construct a new RGBD co-saliency dataset, named RGBD Cosal150 dataset. In this dataset, we collected 21 image groups containing totally 150 images from RGBD NJU-1985 dataset [2] with pixel-level ground truth, and the original depth maps are provided by the dataset itself: RGBD Cosal150 dataset (~13.9MB) [Download Link]

Code and Results:

  • We evaluate the proposed co-saliency model on two datasets (RGBD Cosal150 dataset and RGBD Coseg183 dataset). Even though the images own complex and variable backgrounds or the salient objects exhibit large variations in shape and direction, the proposed method effectively highlights the common salient objects from the image group. In addition, the proposed co-saliency model achieves the best performance on both of the RGBD Coal150 and Coseg183 datasets with the highest F-measure and the smallest MAE score: RGBD Co-saliency Results (~5.7MB) [Download Link]