Mapping the unseen: exploiting super-resolution for semantic segmentation in low-resolution images

• Autor(a) principal: Matheus Barros Pereira
• Data de Publicação: 2019
• Tipo de documento: Dissertação
• Idioma: eng
• Título da fonte: Repositório Institucional da UFMG
• Texto Completo: http://hdl.handle.net/1843/36706
• topic: Remote sensing; Super resolution; Semantic segmentation; Computação – Teses; Visão por computador – Teses; Super-resolução – Teses; Sensoriamento remoto – Teses; Segmentação semântica – Teses

High-resolution aerial images are desirable for most of the deep-based remote sensing applications. This type of data, however, is not always accessible or affordable. On the other hand, coarse resolution remote sensing images, such as LANDSAT and MODIS, are easily found in public open repositories and, therefore, are widely used in many studies. The problem is that the amount of spacial information compressed into one single pixel in a low-resolution representation can compromise pattern recognition algorithms. Thus, the use of coarse-resolution data for automatic creation of thematic maps is very restricted since most of the deep-based semantic segmentation (a.k.a dense labeling) approaches are only suitable for subdecimeter data. Super-resolution is a classic computer vision problem that aims to restore the quality of degraded, low-resolution images. In this work, we design two frameworks in order to evaluate the effectiveness of deep-based super-resolution in the semantic segmentation of low-resolution remote sensing images. Our objective is to evaluate how effective is deep-based super-resolution to different levels of degradation, how it compares to unsupervised bicubic interpolation and if it is able to reconstruct small objects and, consequently, contribute to semantic segmentation improvement. The first framework uses super-resolution as a pre-processing step for the semantic segmentation task (two-stage framework). The second framework is an end-to-end approach that trains both networks at the same time while sharing their losses. We carried out an extensive set of experiments on remote sensing datasets with distinct nature and properties. For the agricultural dataset of coffee mapping, which only contains two labels (coffee and non-coffee), the use of low-resolution images achieved only 50% normalized accuracy with 8x up-scaling factor. The two stage framework with super-resolution in the same condition increased this value to 72%. The end-to-end framework further increased the value to 77%, compared to 81% of high-resolution data. For the urban dataset of Vaihingen, using super-resolution in the two stage framework increased the accuracy of car segmentation from 19% to 58% with 8x up-scaling factor, while the end-to-end framework achieved 65%. In this case, with high-resolution data, the accuracy was 69%, which is not far from the super-resolution result. Both cases are examples of how super-resolution is able to recover important texture details (for coffee crops, for example) and is also able to make more discernible small objects that were difficult to see in a low-resolution representation (such as cars). The results show that super-resolution is effective to improve semantic segmentation performance on low-resolution aerial imagery. It not only outperforms unsupervised interpolation but also achieves semantic segmentation results comparable to high-resolution data. Even with a few training data, the use of the frameworks still achieved better results than bicubic interpolation. Thus, using super-resolution has proven to be a more effective approach than directly inputting low-resolution images to a semantic segmentation network. This is especially true for high degrading factors, which are the cases that super-resolution surpasses more the performance of low-resolution data.