HSCS: Hierarchical Sparsity Based Co-saliency Detection for RGBD Images

Introduction:

  • Co-saliency detection aims to discover common and salient objects in an image group containing more than two relevant images. Moreover, depth information has been demonstrated to be effective for many computer vision tasks. In this paper, we propose a novel co-saliency detection method for RGBD images based on hierarchical sparsity reconstruction and energy function refinement. With the assistance of the intra saliency map, the inter-image correspondence is formulated as a hierarchical sparsity reconstruction framework. The global sparsity reconstruction model with a ranking scheme focuses on capturing the global characteristics among the whole image group through a common foreground dictionary. The pairwise sparsity reconstruction model aims to explore the corresponding relationship between pairwise images through a set of pairwise dictionaries. In order to improve the intra-image smoothness and inter-image consistency, an energy function refinement model is proposed, which includes the unary data term, spatial smooth term, and holistic consistency term. Experiments on two RGBD co-saliency detection benchmarks demonstrate that the proposed method outperforms the state-of-the-art algorithms both qualitatively and quantitatively.





Paper:

  • Runmin Cong, Jianjun Lei, Huazhu Fu, Qingming Huang, Xiaochun Cao, Nam Ling, HSCS: Hierarchical Sparsity Based Co-saliency Detection for RGBD Images, IEEE Transactions on Multimedia, vol. 21, no. 7, pp. 1660-1671, 2019. [PDF]

Codes and Results:

  • The code is tested on Windows 10 64bit with MATLAB 2014a. If you use our code, please cite our papers. [Code]

    The proposed co-saliency framework is evaluated on two RGBD co-saliency benchmarks: the RGBD Cosal150 dataset [1] and the RGBD Coseg183 dataset [2]: [RGBD Co-saliency Results]

References:

  • [1] R. Cong, J. Lei, H. Fu, Q. Huang, X. Cao, and C. Hou, "Co-saliency detection for RGBD images based on multi-constraint feature matching and cross label propagation," IEEE Transactions on Image Processing, vol. 27, no. 2, pp. 568-579, 2018. [Project]
    [2] H. Fu, D. Xu, S. Lin, and J. Liu, “Object-based RGBD image co-segmentation with mutex constraint,” in Proc. CVPR, 2015, pp. 4428-4436. [Project]
    [3] R. Cong, J. Lei, H. Fu, W. Lin, Q. Huang, X. Cao, and C. Hou, "An iterative co-saliency framework for RGBD images", IEEE Transactions on Cybernetics, vol. 49, no. 1, pp. 233-246, 2019. [Project]