Síntese de fotografias e vídeos com depth-image-based rendering

• Autor(a) principal: Oliveira, Adriano Quilião de
• Data de Publicação: 2019
• Tipo de documento: Tese
• Idioma: por
• Título da fonte: Biblioteca Digital de Teses e Dissertações da UFRGS
• Texto Completo: http://hdl.handle.net/10183/201264
• topic: Computação gráfica; Processamento de imagens; Vídeo digital; view synthesis; inpainting; temporal information

The view synthesis process with Depth-Image-Based Rendering (DIBR) is presented as a promising way to enable applications like TV3D, Free Viewpoint Video, and others related to Virtual Reality and Augmented Reality. DIBR allows numerous virtual views of the same scene to be produced using only a single reference image and its depth map. However, artifacts (cracks and ghosts) and regions without information (holes) are formed in the synthesis process, which need to be treated or filled. In this thesis, we present two approaches for view synthesis with DIBR: ATA and DHS. We developed the ATA approach from in-depth studies on the generation of synthetic still images. This approach identifies empty and translucent cracks and reconstructs the affected regions with a specialized algorithm. Then, ghosts are identified through an evaluation process and warped to the correct positions. Finally, the remaining empty regions are filled with an adaptation of a popular inpainting algorithm that employs dynamically sized patches copied from the reference image and fits different hole types. The DHS approach, on the other hand, uses the advances produced with ATA, presenting an even more robust and reliable hole reconstruction method, aware of the image structure and composition. Ghosts are treated before the virtual view generation. An inpainting algorithm based on hierarchical superpixels is used for hole filling, which reconstructs empty regions based on their neighborhood composition by copying the contents of the reference image. Additionally, we propose a robust method for generating an incremental background model for videos that can be incorporated into any DIBR approach. As an example, we detailed its integration with DHS, which presented better results in the frame-by-frame evaluation. Exhaustive testing proves that the proposed approaches yield better quantitative and qualitative results when compared to several recent and competitive methods, both in the generation of synthetic still images as in videos, in tests with ground truth and real depth maps. As an additional contribution of this thesis, we evaluated the impact of using real depth maps produced with stereo matching in the synthesis process with DIBR and analyzed the relationship between quality metrics employed in both problems.