High-fidelity Stereo Imaging Depth Estimation for Metaverse Devices

Projects supported by Funding Scheme for student projects/activities by Tam Wing Fan Innovation Fund and Philomathia Foundation Innovation Fund

Supervisor: Prof. Evan Y. PENG (Department of Electrical and Electronic Engineering)

Project information

Project descriptions

This project is about high-fidelity stereo RGBD imaging for Metaverse devices.This work proposes to design and realize a fast and accurate 3D imaging architecture that Metaverse VR headsets can leverage. Different from the previous imaging framework, we use the prior of optical encoding, and joint optimization structure in deep optics to explore a new and improved framework for high-resolution RGBD imaging. A state-of-the-art stereo-matching algorithm is explored with jointly optimized lenses and an image recovery network. Coventional imaging algorithms consider only the post processing of captured images, which is less fleasible and with high computational complexity. Our deep stereo depth estimation integrates the optical preprossessing and encoding into the advanced decoding neural networks to achieve a higher accuracy of RGBD imaging with higher resolution. It fully utilizes the information obtained from the stereo camera with a pair of asymmetric lens in a Metaverse device to achieve a more extended depth range of all-in-focus imaging.

Project innovation

We have developed a preliminary RGBD imaging model and constructed a stereo camera system with two lenses that have distinct optical characteristics, currently differing in focal length. This setup enables real-time capture and RGBD imaging capabilities.


Team information

Project leader: LIU Yuhui, MSc(EEE)

Team member(s): OU Liangxun, MSc(EEE)

Project poster
Project video
Project images
Media links

Winner of the Inno Show Award @ The 6th Inno Show

This project team was selected for the Inno Show award at the 6th Inno Show. 

This team has received HK$20,000 sponsorship to participate in international design competition(s) and consumables expenses to support further development of the project.

The best project award - COMP3329 Computer Game Design and Programming @ The 1st Engineering InnoShow

This project team was selected for the best project award – COMP3329 Computer Game Design and Programming at the 1st Engineering InnoShow.

PERfECT Wearables: for Minimally-Invasive and Continuous Blood-Glucose Monitoring was selected for the Inno Show Award
Photos in the Engineering Inno Show