High-quality 3D Video by Fusing Active and Passive Sensors

Project goals 1. To analyze strengths and weaknesses of active and passive depth sensors 2. To design a novel fusion algorithm by exploiting synergy between different sensing methods that is improved over the state-of-the-art 3. To represent very fuzzy objects which are very difficult to acquire with the conventional depth estimation method 4. To achieve high-quality 3D video by incorporating both spatial and temporal coherence Description of the work First, we will find the best hardware configuration for active and passive sensors to achieve accurate system calibration and robust depth acquisition. In the system setup and calibration phase, we will conduct photometric, colorimetric, and geometric calibration of each depth sensor and geometric calibration of the entire system. In the design of new fusion models phase, we will be working on the design of new fusion models. We will thereby head in different directions by elaborating different ideas gained in the system calibration phase. The detail of designing a novel fusion model has been stated under "3.2 development of novel fusion method". Finally, we expect a first version of the framework 9 months after the beginning of the project. In this phase, ideas for extending our experiments by including other existing methods will come up as soon as the first results of our benchmark test are available. The phase of experiments aims at providing a competitive performance evaluation. Expected results and impacts 1. The proposed method that combines both active and passive depth sensing is expected to perform better than either one alone. If this approach is successfully developed, it will improve accuracy and robustness of depth maps. 2. The estimation of accurate depth information will have a positive impact on all depth image-based technologies since their results are is very dependent on the quality of the estimated depth map. 3. The proposed research will have a great impact on the broadcasting industry since the fast and easy generation of 3D video leads to the various 3D contents production.