When rendering virtual objects in a mixed reality application, it is helpful to have access to an environment map that captures the appearance of the scene from the perspective of the virtual object. It is straightforward to render virtual objects into such maps, but capturing and correctly rendering the real components of the scene into the map is much more challenging. This information is often recovered from physical light probes, such as reflective spheres or fisheye cameras, placed at the location of the virtual object in the scene. For many application areas, however, real light probes would be intrusive or impractical. Ideally, all of the information necessary to produce detailed environment maps could be captured using a single device. We introduce a method using an RGBD camera and a small fisheye camera, contained in a single unit, to create environment maps at any location in an indoor scene. The method combines the output from both cameras to correct for their limited field of view and the displacement from the virtual object, producing complete environment maps suitable for rendering the virtual content in real time. Our method improves on previous probeless approaches by its ability to recover high-frequency environment maps. We demonstrate how this can be used to render virtual objects which shadow, reflect and refract their environment convincingly.


    Access

    Download


    Export, share and cite



    Title :

    Synthesis of environment maps for mixed reality


    Contributors:
    Walton, DR (author) / Thomas, D (author) / Steed, AJ (author) / Sugimoto, A (author)

    Publication date :

    2017-10-09


    Remarks:

    In: Mixed and Augmented Reality (ISMAR), 2017 IEEE International Symposium on. IEEE (2017)


    Type of media :

    Paper


    Type of material :

    Electronic Resource


    Language :

    English



    Classification :

    DDC:    629



    The GrImage Platform: A Mixed Reality Environment for Interactions

    Allard, J. / Franco, J. / Menier, C. et al. | IEEE | 2006


    Real-time unmanned aerial vehicle 3D environment exploration in a mixed reality environment

    Ai, Zhuming / Livingston, Mark A. / Moskowitz, Ira S. | IEEE | 2016



    Mixed Reality-Enhanced Operator Interface for Teleoperation Systems in Unstructured Environment

    Wang, X. / Dunston, P. S. / American Society of Civil Engineers | British Library Conference Proceedings | 2006


    Mixed Reality-Enhanced Operator Interface for Teleoperation Systems in Unstructured Environment

    Wang, X. / Dunston, P. S. / American Society of Civil Engineers | British Library Conference Proceedings | 2006