Bathymetric 3D reconstruction with Neural Radiance Fields

Because of climate change, a dramatic change in underwater vegetation can be observed in Alpine waters. As a result, the composition of the highly sensitive littoral ecosystem is changing and native species are coming under pressure, particularly due to the spread of invasive species. Recording and monitoring the shallow shore area is therefore of great ecological and economic importance, but there is a lack of cost-effective methods for monitoring. Camera drones (UAVs) have made it relatively easy to capture high-quality images at centimeter resolution. However, even with this data base, the reconstruction of complex underwater scenes often fails when using conventional photogrammetric evaluation methods. The reason for the failure of common approaches is the complex beam refraction on the wavy water surface. Recent developments in the field of artificial intelligence for the evaluation of image data, the so- called Neural Radiance Fields (NeRFs), have shown astonishing results for the purely image-based 3D reconstruction of complex objects. The results in the forest sector are also very promising. However, it is still unclear whether NeRFs can also provide an accurate derivation of 3D point clouds of complex underwater scenes. Against this background, the research objectives of the FWF Weave project BathyNerF are (i) the improvement of image-based 3D reconstruction of underwater topography and vegetation from drone images using specially adapted NeRFs and (ii)) the quantitative evaluation of the results obtained. For this purpose, different NeRF models will be tested. One approach is to divide the area into two parts above and below the water surface, each modelled with a separate NeRF. For the underwater NeRF model, the beam refraction at a horizontally assumed water surface is included and the deflected image ray is considered in the model. Another approach also includes the different inclination of the water surface due to waves. Finally, the most ambitious approach attempts to model the influence of refraction in a single NeRF by adding an optical density parameter to the NeRF. The results obtained will be validated by comparison with conventional methods of multi-media photogrammetry and with bathymetric laser scanning as independent references. A further aim of the project is to derive the volume, density and biomass of underwater vegetation. This represents an innovative approach that can contribute to documenting the effects of climate change. The three-year research project is carried out on a transnational basis in cooperation between the Department of Geodesy and Geoinformation at TU Wien (GEO/TUW) and the Karlsruhe Institute of Technology (KIT).