The paper studies the problem of combining region and boundary cues for natural image segmentation. We employ a large database of manually segmented images in order to learn an optimal affinity function between pairs of pixels. These pairwise affinities can then be used to cluster the pixels into visually coherent groups. Region cues are computed as the similarity in brightness, color, and texture between image patches. Boundary cues are incorporated by looking for the presence of an "intervening contour", a large gradient along a straight line connecting two pixels. We first use the dataset of human segmentations to individually optimize parameters of the patch and gradient features for brightness, color, and texture cues. We then quantitatively measure the power of different feature combinations by computing the precision and recall of classifiers trained using those features. The mutual information between the output of the classifiers and the same-segment indicator function provides an alternative evaluation technique that yields identical conclusions. As expected, the best classifier makes use of brightness, color, and texture features, in both patch and gradient forms. We find that for brightness, the gradient cue outperforms the patch similarity. In contrast, using color patch similarity yields better results than using color gradients. Texture is the most powerful of the three channels, with both patches and gradients carrying significant independent information. Interestingly, the proximity of the two pixels does not add any information beyond that provided by the similarity cues. We also find that the convexity assumptions made by the intervening contour approach are supported by the ecological statistics of the dataset.


    Access

    Check access

    Check availability in my library

    Order at Subito €


    Export, share and cite



    Title :

    Learning affinity functions for image segmentation: combining patch-based and gradient-based approaches


    Contributors:
    Fowlkes, C. (author) / Martin, D. (author) / Malik, J. (author)


    Publication date :

    2003-01-01


    Size :

    1177414 byte





    Type of media :

    Conference paper


    Type of material :

    Electronic Resource


    Language :

    English



    Learning Affinity Functions for Image Segmentation: Combining Patch-Based and Gradient-Based Approaches

    Fowlkes, C. / Martin, D. / Malik, J. et al. | British Library Conference Proceedings | 2003


    Aurora image segmentation by combining patch and texture thresholding

    Gao, X. / Fu, R. / Li, X. et al. | British Library Online Contents | 2011


    Affinity functions in fuzzy connectedness based image segmentation I: Equivalence of affinities

    Ciesielski, K. C. / Udupa, J. K. | British Library Online Contents | 2010



    Patch-Based Texture Edges and Segmentation

    Wolf, L. / Huang, X. / Martin, I. et al. | British Library Conference Proceedings | 2006