A Comparative Study of Image-to-Image Translation Techniques for Virtual Object Illumination in Augmented Images

• Autor(a) principal: Curinga, Artur Maricato
• Data de Publicação: 2022
• Tipo de documento: Trabalho de conclusão de curso
• Idioma: eng
• Título da fonte: Repositório Institucional da UFRN
• Texto Completo: https://repositorio.ufrn.br/handle/123456789/48844
• topic: Augmented Reality; Neural Networks; Illumination; Realidade Aumentada; Redes Neurais; Iluminação

This work is a comparison study between two Deep Neural Network (DNN) models in the augmented reality context, aiming to produce visually coherent augmented indoor images with a virtual object inserted. We trained DNN models to generate coherent shadows and illumination for an unlit object given a computer generated photorealistic indoor environment as a reference. The goal is to add the artificially lit object to the reference scene and make it blend in nicely when seen by an human viewer unaware of the interference. We develop a dataset Indoor Shadows with 4826 set of images from the 3D-Front scene dataset, in order to use it as our benchmark. The Pix2Pix and ShadowGAN were trained using the SGD, and Adam, and compared regarding the generated images with a ground truth. We used the L1, L2, and MSSIM metrics to evaluate the results of the trained models. We found that the ShadowGAN trained with Adam had the best results regarding the MSSIM metric and the Pix2Pix trained with SGD and the best results with L1, and L2. We concluded that both techniques are very limited, and the generated images are easily distinguishable from the ground truth.