UR-SRGAN : a generative adversarial network for real-world super-resolution with a U-Net-based discriminator
Autor(a) principal: | |
---|---|
Data de Publicação: | 2022 |
Tipo de documento: | Dissertação |
Idioma: | eng |
Título da fonte: | Repositório Institucional da UFPE |
Texto Completo: | https://repositorio.ufpe.br/handle/123456789/47361 |
Resumo: | Despite several improvements in Super-Resolution deep learning techniques, these proposed methods tend to fail in many real-world scenarios since their models are usually trained using a pre-defined degradation process from high-resolution (HR) ground truth images to low-resolution (LR) ones. In this work, we propose a supervised Generative Adversarial Network (GAN) model for Image Super-Resolution which has as the first stage to estimate blur kernels and noise estimation from real-world images to generate LR images for the training phase. Furthermore, the proposal includes implementing a novel U-Net-based discriminator, to consider an input image’s global and local context, and it allows employing a CutMix data augmentation for consistency regularization in the two-dimensional output space of the decoder. The proposed model was applied to three main datasets that are ordinarily used in super-resolution official competitions. The commonly-used evaluation metrics for image restoration were used for this evaluation: Peak Signal-to-Noise Ratio (PSNR), Structural Similarity (SSIM), Learned Perceptual Image Patch Similarity (LPIPS) and Natural Image Quality Evaluator (NIQE). After implementing this new architecture, three other prominent models in the state-of-the-art GAN proposals for super-resolution were trained with the same parameters and databases to perform a global comparison between all of them. Finally, the results of the experimentation in training and evaluation tasks between all the models suggest an improvement in the performance of the presented work compared to the other architectures based on the established metrics. |
id |
UFPE_814e818666bb5207e6e6d3344040df72 |
---|---|
oai_identifier_str |
oai:repositorio.ufpe.br:123456789/47361 |
network_acronym_str |
UFPE |
network_name_str |
Repositório Institucional da UFPE |
repository_id_str |
2221 |
spelling |
VARGAS, Kevin Ian Ruizhttps://lattes.cnpq.br/6685368039034715http://lattes.cnpq.br/3084134533707587REN, Tsang Ing2022-11-08T11:27:51Z2022-11-08T11:27:51Z2022-08-08VARGAS, Kevin Ian Ruiz. UR-SRGAN: a generative adversarial network for real-world super-resolution with a U-Net-based discriminator. 2022. Dissertação (Mestrado em Ciência da Computação) – Universidade Federal de Pernambuco, Recife, 2022.https://repositorio.ufpe.br/handle/123456789/47361Despite several improvements in Super-Resolution deep learning techniques, these proposed methods tend to fail in many real-world scenarios since their models are usually trained using a pre-defined degradation process from high-resolution (HR) ground truth images to low-resolution (LR) ones. In this work, we propose a supervised Generative Adversarial Network (GAN) model for Image Super-Resolution which has as the first stage to estimate blur kernels and noise estimation from real-world images to generate LR images for the training phase. Furthermore, the proposal includes implementing a novel U-Net-based discriminator, to consider an input image’s global and local context, and it allows employing a CutMix data augmentation for consistency regularization in the two-dimensional output space of the decoder. The proposed model was applied to three main datasets that are ordinarily used in super-resolution official competitions. The commonly-used evaluation metrics for image restoration were used for this evaluation: Peak Signal-to-Noise Ratio (PSNR), Structural Similarity (SSIM), Learned Perceptual Image Patch Similarity (LPIPS) and Natural Image Quality Evaluator (NIQE). After implementing this new architecture, three other prominent models in the state-of-the-art GAN proposals for super-resolution were trained with the same parameters and databases to perform a global comparison between all of them. Finally, the results of the experimentation in training and evaluation tasks between all the models suggest an improvement in the performance of the presented work compared to the other architectures based on the established metrics.CAPESApesar de várias melhorias nas técnicas de aprendizado profundo de super-resolução, esses métodos propostos tendem a falhar em muitos cenários do mundo real, pois seus modelos geralmente são treinados usando um processo de degradação predefinido de imagens de verdade de alta resolução - High Resolution (HR) para baixa resolução - Low Resolution (LR). Neste trabalho, propomos um modelo supervisionado de Generative Ad-versarial Network (GAN) para Super-Resolução de Imagem que tem como primeira etapa estimar kernels de borramento e estimativa de ruído de imagens do mundo real para gerar imagens LR para a fase de treinamento. Além disso, a proposta inclui a implementação de um novo discriminador baseado em U-Net, para considerar o contexto global e local de uma imagem de entrada, e permite empregar um aumento de dados CutMix para regularização de consistência no espaço de saída bidimensional do decodificador. O modelo proposto foi aplicado a três conjuntos de dados principais que são normalmente usados em competições oficiais de super-resolução. As métricas de avaliação comumente usadas para restauração de imagem foram usadas para esta avaliação: Peak Signal-to-Noise Ra-tio (PSNR), Structural Similarity (SSIM), Learned Perceptual Image Patch Similarity (LPIPS) e Natural Image Quality Evaluator (NIQE). Após a implementação desta nova arquitetura, três outros modelos de destaque nas propostas GAN de super-resolução de última geração foram treinados com os mesmos parâmetros e bancos de dados para realizar uma comparação global entre todos eles. Por fim, os resultados da experimentação em tarefas de treinamento e avaliação entre todos os modelos sugerem uma melhora no desempenho do trabalho apresentado em relação às demais arquiteturas baseadas nas métricas estabelecidas.engUniversidade Federal de PernambucoPrograma de Pos Graduacao em Ciencia da ComputacaoUFPEBrasilhttp://creativecommons.org/licenses/by-nc-nd/3.0/br/info:eu-repo/semantics/openAccessInteligência computacionalResolução de imagemModelagemUR-SRGAN : a generative adversarial network for real-world super-resolution with a U-Net-based discriminatorinfo:eu-repo/semantics/publishedVersioninfo:eu-repo/semantics/masterThesismestradoreponame:Repositório Institucional da UFPEinstname:Universidade Federal de Pernambuco (UFPE)instacron:UFPELICENSElicense.txtlicense.txttext/plain; charset=utf-82362https://repositorio.ufpe.br/bitstream/123456789/47361/3/license.txt5e89a1613ddc8510c6576f4b23a78973MD53ORIGINALDISSERTAÇÃO Kevin Ian Ruiz Vargas.pdfDISSERTAÇÃO Kevin Ian Ruiz Vargas.pdfapplication/pdf4502467https://repositorio.ufpe.br/bitstream/123456789/47361/1/DISSERTA%c3%87%c3%83O%20Kevin%20Ian%20Ruiz%20Vargas.pdfc997c67031ad74cce816b10ff35f7a69MD51CC-LICENSElicense_rdflicense_rdfapplication/rdf+xml; charset=utf-8811https://repositorio.ufpe.br/bitstream/123456789/47361/2/license_rdfe39d27027a6cc9cb039ad269a5db8e34MD52TEXTDISSERTAÇÃO Kevin Ian Ruiz Vargas.pdf.txtDISSERTAÇÃO Kevin Ian Ruiz Vargas.pdf.txtExtracted texttext/plain183564https://repositorio.ufpe.br/bitstream/123456789/47361/4/DISSERTA%c3%87%c3%83O%20Kevin%20Ian%20Ruiz%20Vargas.pdf.txt7b80368cb30ced2e7b9e8ca67e2dcc5bMD54THUMBNAILDISSERTAÇÃO Kevin Ian Ruiz Vargas.pdf.jpgDISSERTAÇÃO Kevin Ian Ruiz Vargas.pdf.jpgGenerated Thumbnailimage/jpeg1145https://repositorio.ufpe.br/bitstream/123456789/47361/5/DISSERTA%c3%87%c3%83O%20Kevin%20Ian%20Ruiz%20Vargas.pdf.jpge264b8374d058fbe2c06324b958fc493MD55123456789/473612022-11-09 02:23:20.673oai:repositorio.ufpe.br:123456789/47361VGVybW8gZGUgRGVww7NzaXRvIExlZ2FsIGUgQXV0b3JpemHDp8OjbyBwYXJhIFB1YmxpY2l6YcOnw6NvIGRlIERvY3VtZW50b3Mgbm8gUmVwb3NpdMOzcmlvIERpZ2l0YWwgZGEgVUZQRQoKCkRlY2xhcm8gZXN0YXIgY2llbnRlIGRlIHF1ZSBlc3RlIFRlcm1vIGRlIERlcMOzc2l0byBMZWdhbCBlIEF1dG9yaXphw6fDo28gdGVtIG8gb2JqZXRpdm8gZGUgZGl2dWxnYcOnw6NvIGRvcyBkb2N1bWVudG9zIGRlcG9zaXRhZG9zIG5vIFJlcG9zaXTDs3JpbyBEaWdpdGFsIGRhIFVGUEUgZSBkZWNsYXJvIHF1ZToKCkkgLSBvcyBkYWRvcyBwcmVlbmNoaWRvcyBubyBmb3JtdWzDoXJpbyBkZSBkZXDDs3NpdG8gc8OjbyB2ZXJkYWRlaXJvcyBlIGF1dMOqbnRpY29zOwoKSUkgLSAgbyBjb250ZcO6ZG8gZGlzcG9uaWJpbGl6YWRvIMOpIGRlIHJlc3BvbnNhYmlsaWRhZGUgZGUgc3VhIGF1dG9yaWE7CgpJSUkgLSBvIGNvbnRlw7pkbyDDqSBvcmlnaW5hbCwgZSBzZSBvIHRyYWJhbGhvIGUvb3UgcGFsYXZyYXMgZGUgb3V0cmFzIHBlc3NvYXMgZm9yYW0gdXRpbGl6YWRvcywgZXN0YXMgZm9yYW0gZGV2aWRhbWVudGUgcmVjb25oZWNpZGFzOwoKSVYgLSBxdWFuZG8gdHJhdGFyLXNlIGRlIG9icmEgY29sZXRpdmEgKG1haXMgZGUgdW0gYXV0b3IpOiB0b2RvcyBvcyBhdXRvcmVzIGVzdMOjbyBjaWVudGVzIGRvIGRlcMOzc2l0byBlIGRlIGFjb3JkbyBjb20gZXN0ZSB0ZXJtbzsKClYgLSBxdWFuZG8gdHJhdGFyLXNlIGRlIFRyYWJhbGhvIGRlIENvbmNsdXPDo28gZGUgQ3Vyc28sIERpc3NlcnRhw6fDo28gb3UgVGVzZTogbyBhcnF1aXZvIGRlcG9zaXRhZG8gY29ycmVzcG9uZGUgw6AgdmVyc8OjbyBmaW5hbCBkbyB0cmFiYWxobzsKClZJIC0gcXVhbmRvIHRyYXRhci1zZSBkZSBUcmFiYWxobyBkZSBDb25jbHVzw6NvIGRlIEN1cnNvLCBEaXNzZXJ0YcOnw6NvIG91IFRlc2U6IGVzdG91IGNpZW50ZSBkZSBxdWUgYSBhbHRlcmHDp8OjbyBkYSBtb2RhbGlkYWRlIGRlIGFjZXNzbyBhbyBkb2N1bWVudG8gYXDDs3MgbyBkZXDDs3NpdG8gZSBhbnRlcyBkZSBmaW5kYXIgbyBwZXLDrW9kbyBkZSBlbWJhcmdvLCBxdWFuZG8gZm9yIGVzY29saGlkbyBhY2Vzc28gcmVzdHJpdG8sIHNlcsOhIHBlcm1pdGlkYSBtZWRpYW50ZSBzb2xpY2l0YcOnw6NvIGRvIChhKSBhdXRvciAoYSkgYW8gU2lzdGVtYSBJbnRlZ3JhZG8gZGUgQmlibGlvdGVjYXMgZGEgVUZQRSAoU0lCL1VGUEUpLgoKIApQYXJhIHRyYWJhbGhvcyBlbSBBY2Vzc28gQWJlcnRvOgoKTmEgcXVhbGlkYWRlIGRlIHRpdHVsYXIgZG9zIGRpcmVpdG9zIGF1dG9yYWlzIGRlIGF1dG9yIHF1ZSByZWNhZW0gc29icmUgZXN0ZSBkb2N1bWVudG8sIGZ1bmRhbWVudGFkbyBuYSBMZWkgZGUgRGlyZWl0byBBdXRvcmFsIG5vIDkuNjEwLCBkZSAxOSBkZSBmZXZlcmVpcm8gZGUgMTk5OCwgYXJ0LiAyOSwgaW5jaXNvIElJSSwgYXV0b3Jpem8gYSBVbml2ZXJzaWRhZGUgRmVkZXJhbCBkZSBQZXJuYW1idWNvIGEgZGlzcG9uaWJpbGl6YXIgZ3JhdHVpdGFtZW50ZSwgc2VtIHJlc3NhcmNpbWVudG8gZG9zIGRpcmVpdG9zIGF1dG9yYWlzLCBwYXJhIGZpbnMgZGUgbGVpdHVyYSwgaW1wcmVzc8OjbyBlL291IGRvd25sb2FkIChhcXVpc2nDp8OjbykgYXRyYXbDqXMgZG8gc2l0ZSBkbyBSZXBvc2l0w7NyaW8gRGlnaXRhbCBkYSBVRlBFIG5vIGVuZGVyZcOnbyBodHRwOi8vd3d3LnJlcG9zaXRvcmlvLnVmcGUuYnIsIGEgcGFydGlyIGRhIGRhdGEgZGUgZGVww7NzaXRvLgoKIApQYXJhIHRyYWJhbGhvcyBlbSBBY2Vzc28gUmVzdHJpdG86CgpOYSBxdWFsaWRhZGUgZGUgdGl0dWxhciBkb3MgZGlyZWl0b3MgYXV0b3JhaXMgZGUgYXV0b3IgcXVlIHJlY2FlbSBzb2JyZSBlc3RlIGRvY3VtZW50bywgZnVuZGFtZW50YWRvIG5hIExlaSBkZSBEaXJlaXRvIEF1dG9yYWwgbm8gOS42MTAgZGUgMTkgZGUgZmV2ZXJlaXJvIGRlIDE5OTgsIGFydC4gMjksIGluY2lzbyBJSUksIGF1dG9yaXpvIGEgVW5pdmVyc2lkYWRlIEZlZGVyYWwgZGUgUGVybmFtYnVjbyBhIGRpc3BvbmliaWxpemFyIGdyYXR1aXRhbWVudGUsIHNlbSByZXNzYXJjaW1lbnRvIGRvcyBkaXJlaXRvcyBhdXRvcmFpcywgcGFyYSBmaW5zIGRlIGxlaXR1cmEsIGltcHJlc3PDo28gZS9vdSBkb3dubG9hZCAoYXF1aXNpw6fDo28pIGF0cmF2w6lzIGRvIHNpdGUgZG8gUmVwb3NpdMOzcmlvIERpZ2l0YWwgZGEgVUZQRSBubyBlbmRlcmXDp28gaHR0cDovL3d3dy5yZXBvc2l0b3Jpby51ZnBlLmJyLCBxdWFuZG8gZmluZGFyIG8gcGVyw61vZG8gZGUgZW1iYXJnbyBjb25kaXplbnRlIGFvIHRpcG8gZGUgZG9jdW1lbnRvLCBjb25mb3JtZSBpbmRpY2FkbyBubyBjYW1wbyBEYXRhIGRlIEVtYmFyZ28uCg==Repositório InstitucionalPUBhttps://repositorio.ufpe.br/oai/requestattena@ufpe.bropendoar:22212022-11-09T05:23:20Repositório Institucional da UFPE - Universidade Federal de Pernambuco (UFPE)false |
dc.title.pt_BR.fl_str_mv |
UR-SRGAN : a generative adversarial network for real-world super-resolution with a U-Net-based discriminator |
title |
UR-SRGAN : a generative adversarial network for real-world super-resolution with a U-Net-based discriminator |
spellingShingle |
UR-SRGAN : a generative adversarial network for real-world super-resolution with a U-Net-based discriminator VARGAS, Kevin Ian Ruiz Inteligência computacional Resolução de imagem Modelagem |
title_short |
UR-SRGAN : a generative adversarial network for real-world super-resolution with a U-Net-based discriminator |
title_full |
UR-SRGAN : a generative adversarial network for real-world super-resolution with a U-Net-based discriminator |
title_fullStr |
UR-SRGAN : a generative adversarial network for real-world super-resolution with a U-Net-based discriminator |
title_full_unstemmed |
UR-SRGAN : a generative adversarial network for real-world super-resolution with a U-Net-based discriminator |
title_sort |
UR-SRGAN : a generative adversarial network for real-world super-resolution with a U-Net-based discriminator |
author |
VARGAS, Kevin Ian Ruiz |
author_facet |
VARGAS, Kevin Ian Ruiz |
author_role |
author |
dc.contributor.authorLattes.pt_BR.fl_str_mv |
https://lattes.cnpq.br/6685368039034715 |
dc.contributor.advisorLattes.pt_BR.fl_str_mv |
http://lattes.cnpq.br/3084134533707587 |
dc.contributor.author.fl_str_mv |
VARGAS, Kevin Ian Ruiz |
dc.contributor.advisor1.fl_str_mv |
REN, Tsang Ing |
contributor_str_mv |
REN, Tsang Ing |
dc.subject.por.fl_str_mv |
Inteligência computacional Resolução de imagem Modelagem |
topic |
Inteligência computacional Resolução de imagem Modelagem |
description |
Despite several improvements in Super-Resolution deep learning techniques, these proposed methods tend to fail in many real-world scenarios since their models are usually trained using a pre-defined degradation process from high-resolution (HR) ground truth images to low-resolution (LR) ones. In this work, we propose a supervised Generative Adversarial Network (GAN) model for Image Super-Resolution which has as the first stage to estimate blur kernels and noise estimation from real-world images to generate LR images for the training phase. Furthermore, the proposal includes implementing a novel U-Net-based discriminator, to consider an input image’s global and local context, and it allows employing a CutMix data augmentation for consistency regularization in the two-dimensional output space of the decoder. The proposed model was applied to three main datasets that are ordinarily used in super-resolution official competitions. The commonly-used evaluation metrics for image restoration were used for this evaluation: Peak Signal-to-Noise Ratio (PSNR), Structural Similarity (SSIM), Learned Perceptual Image Patch Similarity (LPIPS) and Natural Image Quality Evaluator (NIQE). After implementing this new architecture, three other prominent models in the state-of-the-art GAN proposals for super-resolution were trained with the same parameters and databases to perform a global comparison between all of them. Finally, the results of the experimentation in training and evaluation tasks between all the models suggest an improvement in the performance of the presented work compared to the other architectures based on the established metrics. |
publishDate |
2022 |
dc.date.accessioned.fl_str_mv |
2022-11-08T11:27:51Z |
dc.date.available.fl_str_mv |
2022-11-08T11:27:51Z |
dc.date.issued.fl_str_mv |
2022-08-08 |
dc.type.status.fl_str_mv |
info:eu-repo/semantics/publishedVersion |
dc.type.driver.fl_str_mv |
info:eu-repo/semantics/masterThesis |
format |
masterThesis |
status_str |
publishedVersion |
dc.identifier.citation.fl_str_mv |
VARGAS, Kevin Ian Ruiz. UR-SRGAN: a generative adversarial network for real-world super-resolution with a U-Net-based discriminator. 2022. Dissertação (Mestrado em Ciência da Computação) – Universidade Federal de Pernambuco, Recife, 2022. |
dc.identifier.uri.fl_str_mv |
https://repositorio.ufpe.br/handle/123456789/47361 |
identifier_str_mv |
VARGAS, Kevin Ian Ruiz. UR-SRGAN: a generative adversarial network for real-world super-resolution with a U-Net-based discriminator. 2022. Dissertação (Mestrado em Ciência da Computação) – Universidade Federal de Pernambuco, Recife, 2022. |
url |
https://repositorio.ufpe.br/handle/123456789/47361 |
dc.language.iso.fl_str_mv |
eng |
language |
eng |
dc.rights.driver.fl_str_mv |
http://creativecommons.org/licenses/by-nc-nd/3.0/br/ info:eu-repo/semantics/openAccess |
rights_invalid_str_mv |
http://creativecommons.org/licenses/by-nc-nd/3.0/br/ |
eu_rights_str_mv |
openAccess |
dc.publisher.none.fl_str_mv |
Universidade Federal de Pernambuco |
dc.publisher.program.fl_str_mv |
Programa de Pos Graduacao em Ciencia da Computacao |
dc.publisher.initials.fl_str_mv |
UFPE |
dc.publisher.country.fl_str_mv |
Brasil |
publisher.none.fl_str_mv |
Universidade Federal de Pernambuco |
dc.source.none.fl_str_mv |
reponame:Repositório Institucional da UFPE instname:Universidade Federal de Pernambuco (UFPE) instacron:UFPE |
instname_str |
Universidade Federal de Pernambuco (UFPE) |
instacron_str |
UFPE |
institution |
UFPE |
reponame_str |
Repositório Institucional da UFPE |
collection |
Repositório Institucional da UFPE |
bitstream.url.fl_str_mv |
https://repositorio.ufpe.br/bitstream/123456789/47361/3/license.txt https://repositorio.ufpe.br/bitstream/123456789/47361/1/DISSERTA%c3%87%c3%83O%20Kevin%20Ian%20Ruiz%20Vargas.pdf https://repositorio.ufpe.br/bitstream/123456789/47361/2/license_rdf https://repositorio.ufpe.br/bitstream/123456789/47361/4/DISSERTA%c3%87%c3%83O%20Kevin%20Ian%20Ruiz%20Vargas.pdf.txt https://repositorio.ufpe.br/bitstream/123456789/47361/5/DISSERTA%c3%87%c3%83O%20Kevin%20Ian%20Ruiz%20Vargas.pdf.jpg |
bitstream.checksum.fl_str_mv |
5e89a1613ddc8510c6576f4b23a78973 c997c67031ad74cce816b10ff35f7a69 e39d27027a6cc9cb039ad269a5db8e34 7b80368cb30ced2e7b9e8ca67e2dcc5b e264b8374d058fbe2c06324b958fc493 |
bitstream.checksumAlgorithm.fl_str_mv |
MD5 MD5 MD5 MD5 MD5 |
repository.name.fl_str_mv |
Repositório Institucional da UFPE - Universidade Federal de Pernambuco (UFPE) |
repository.mail.fl_str_mv |
attena@ufpe.br |
_version_ |
1802310897528471552 |