Depth Map Estimation from Multi-View Images with Nerf-Based Refinement

In this paper, we propose a method to refine depth maps estimated by Multi-View Stereo (MVS) with Neural Radiance Field (NeRF) optimization to estimate depth maps from multi-view images with high accuracy. MVS estimates the depths on object surfaces with high accuracy, and NeRF estimates the depths at object boundaries with high accuracy. The key ideas of the proposed method are (i) to combine MVS and NeRF to utilize the advantages of both in depth map estimation, (ii) not to require any training process, therefore no training dataset and ground truth are required, and (iii) to use NeRF for depth map refinement. Through a set of experiments using the Redwood-3dscan dataset, we demonstrate the effectiveness of the proposed method compared to conventional depth map estimation methods.