The exploitation of semantic information in computer vision problems can be difficult because of the large difference in representations and levels of knowledge. Image analysis is formulated in terms of low-level features describing image structure and intensity, while high-level knowledge such as purpose and common sense are encoded in abstract, non-geometric representations. In this work we attempt to bridge this gap through the integration of image analysis algorithms with WordNet, a large semantic network that explicitly links related words in a hierarchical structure. Our problem domain is the understanding of broadcast news, as this provides both linguistic information in the transcript and video information. Visual detection algorithms such as face detection and object tracking are applied to the video to extract basic object information, which is indexed into WordNet. The transcript provides topic information in the form of detected keywords. Together, both types of information are used to constrain a search within WordNet for a description of the video content in terms of the most likely WordNet concepts. This project is in its early stages; the general ideas and concepts are presented here.


    Access

    Check access

    Check availability in my library

    Order at Subito €


    Export, share and cite



    Title :

    Multi-modal fusion for video understanding


    Contributors:
    Hoogs, A. (author) / Mundy, J. (author) / Cross, G. (author)


    Publication date :

    2001-01-01


    Size :

    705664 byte




    Type of media :

    Conference paper


    Type of material :

    Electronic Resource


    Language :

    English



    Multi-Modal Fusion for Video Understanding

    Hoogs, A. / Mundy, J. / Cross, G. | British Library Conference Proceedings | 2001



    News Video Classification Based on Multi-modal Information Fusion

    Lie, W.-N. / Su, C.-K. | British Library Conference Proceedings | 2005


    Tracking Humans using Multi-modal Fusion

    Xiaotao Zou, / Bhanu, B. | IEEE | 2005


    Understanding Modal Analysis

    Walgrave,S.C. / Ehlbeck,J.M. / Freightliner,US | Automotive engineering | 1978