Beyond landscapes : an exemplar-based image colorization method
Autor(a) principal: | |
---|---|
Data de Publicação: | 2018 |
Tipo de documento: | Dissertação |
Idioma: | eng |
Título da fonte: | Repositório Institucional da UFPE |
Texto Completo: | https://repositorio.ufpe.br/handle/123456789/43483 |
Resumo: | Image colorization consists in, given a grayscale image, generating a plausible color version of this image, which can be performed as a manual/artistic process1 but also as a computer assisted or even fully automated process. Colorization is a underconstrained problem, which requires extra information in order to provide a unique solution. This dissertation focuses on exemplar-based colorization methods, in which the extra informa- tion comes from a user-selected color reference image with similar semantic content to the target. While the user selects the reference based on content similarity, the algorithms estimate similarity based on local descriptors of image regions. This difference in abstrac- tion between the user and algorithm perspective can lead to the algorithms not always being able to transfer colors between semantic corresponding elements in the image pair, specially in images in which the mapping between content/color and local descriptors is complex. Most exemplar-based methods in the literature display successful examples mostly limited to simple instances, such as landscapes, animals and simple buildings. Based on this observation, in this research we propose a new exemplar-based method that aims at generating plausible colorizations for a wider range of image pairs, including images of higher complexity. To that end, the proposed method features a two-stage clas- sification scheme that uses the available features in a more consistent manner and makes the initial color assignments more robust. It also includes an edge-aware relabeling method that enhances the spatial coherence and mitigates the impact of the multimodality, inher- ent to the colorization problem, over the method’s colorized outputs. In this dissertation, we present a broad review of the colorization literature introducing a taxonomy that cat- egorizes colorization techniques based on the source of prior information used to guide their color assignments. The proposed method pipeline is then described in details, and its key modules are validated through experiments. Moreover, a comparative analysis is performed which subjects the proposed and baseline methods to different source/target pairs to visually assess and compare their results. Experimental results indicate that the proposed method yields colorization results that are more coherent and of higher visual quality compared to two state-of-the-art exemplar-based colorization algorithms, both in simple and complex image sets. The results also indicate that exemplar-based methods can achieve results of comparable visual aspect to those of modern deep learning approaches while allowing more user control. |
id |
UFPE_db20003b5874b056e09e934ab314e8c8 |
---|---|
oai_identifier_str |
oai:repositorio.ufpe.br:123456789/43483 |
network_acronym_str |
UFPE |
network_name_str |
Repositório Institucional da UFPE |
repository_id_str |
2221 |
spelling |
PEREIRA SOBRINHO, Saulo César Rodrigueshttp://lattes.cnpq.br/9595267291106910http://lattes.cnpq.br/7532050172035129KELNER, Judith2022-03-23T17:45:10Z2022-03-23T17:45:10Z2018-12-19PEREIRA SOBRINHO, Saulo César Rodrigues. Beyond landscapes: an exemplar-based image colorization method. 2018. Dissertação (Mestrado em Ciência da Computação) - Universidade Federal de Pernambuco, Recife, 2018.https://repositorio.ufpe.br/handle/123456789/43483Image colorization consists in, given a grayscale image, generating a plausible color version of this image, which can be performed as a manual/artistic process1 but also as a computer assisted or even fully automated process. Colorization is a underconstrained problem, which requires extra information in order to provide a unique solution. This dissertation focuses on exemplar-based colorization methods, in which the extra informa- tion comes from a user-selected color reference image with similar semantic content to the target. While the user selects the reference based on content similarity, the algorithms estimate similarity based on local descriptors of image regions. This difference in abstrac- tion between the user and algorithm perspective can lead to the algorithms not always being able to transfer colors between semantic corresponding elements in the image pair, specially in images in which the mapping between content/color and local descriptors is complex. Most exemplar-based methods in the literature display successful examples mostly limited to simple instances, such as landscapes, animals and simple buildings. Based on this observation, in this research we propose a new exemplar-based method that aims at generating plausible colorizations for a wider range of image pairs, including images of higher complexity. To that end, the proposed method features a two-stage clas- sification scheme that uses the available features in a more consistent manner and makes the initial color assignments more robust. It also includes an edge-aware relabeling method that enhances the spatial coherence and mitigates the impact of the multimodality, inher- ent to the colorization problem, over the method’s colorized outputs. In this dissertation, we present a broad review of the colorization literature introducing a taxonomy that cat- egorizes colorization techniques based on the source of prior information used to guide their color assignments. The proposed method pipeline is then described in details, and its key modules are validated through experiments. Moreover, a comparative analysis is performed which subjects the proposed and baseline methods to different source/target pairs to visually assess and compare their results. Experimental results indicate that the proposed method yields colorization results that are more coherent and of higher visual quality compared to two state-of-the-art exemplar-based colorization algorithms, both in simple and complex image sets. The results also indicate that exemplar-based methods can achieve results of comparable visual aspect to those of modern deep learning approaches while allowing more user control.CNPqA colorização de imagens consiste em, dada uma imagem em tons de cinza, gerar uma versão colorida plausível desta imagem, o que pode ser realizado como um processo manual/artístico2 ou (semi-)automático. A colorização é um problema subdeterminado, sendo necessária informação extra para a obtenção de uma solução única. Esta dissertação foca em métodos de colorização baseados em exemplo, nos quais a informação extra vem de uma imagem de referência colorida e de conteúdo similar, selecionada pelo usuário. Enquanto a escolha da referência é baseada em similaridade de conteúdo, o algoritmo estima similaridades baseado em descritores locais. Esta diferença de nível de abstração faz com que tais métodos nem sempre sejam capazes de transferir cores entre elementos de semântica correspondente no par de imagens, sobretudo em imagens onde o mapeamento entre semântica/cores e descritores locais é complexo. A maioria dos métodos baseados em exemplo mostram resultados bem sucedidos em sua maioria limitados a imagens de mapeamento simples como paisagens, animais e construções simples. Nesta pesquisa, um método de colorização baseado em exemplos é proposto, com objetivo de criar coloriza- ções plausíveis para um conjunto mais abrangente de pares de imagens de entrada do que os métodos existentes, especialmente imagens de maior complexidade. Para alcançar este objetivo, o método proposto conta com um mecanismo de classificação em duas eta- pas, que faz uso do conjunto de características extraídas das imagens de uma maneira mais eficiente. O método ainda inclui um mecanismo de refinamento da classificação ini- cial baseado nas bordas da imagem original, proporcionando maior coerência espacial ao resultado e ao mesmo tempo reduzindo o impacto da multimodalidade. Nesta disser- tação, apresentamos uma revisão abrangente da literatura em colorização, introduzindo uma taxonomia unificada que categoriza as técnicas baseada na fonte de informação a priori utilizada para guiar a atribuição de cores. O método proposto é descrito em de- talhes e seus principais componentes são validados experimentalmente. Além disso, uma análise experimental é realizada submetendo a técnica proposta e algoritmos selecionados da literatura à diferentes pares de imagens para avaliar e comparar visualmente os seus resultados. Os experimentos indicaram que o método proposto é capaz de gerar coloriza- ções que são mais coerentes e de maior qualidade visual, em imagens simples e complexas, quando comparado com dois algoritmos baseados em exemplo do estado da arte. Os resul- tados obtidos também indicam que métodos baseados em exemplo são capazes de obter, em certas instâncias, resultados comparáveis aos dos novos algoritmos de aprendizagem profunda, enquanto permitem maior controle do usuário sobre o resultado.engUniversidade Federal de PernambucoPrograma de Pos Graduacao em Ciencia da ComputacaoUFPEBrasilAttribution-NonCommercial-NoDerivs 3.0 Brazilhttp://creativecommons.org/licenses/by-nc-nd/3.0/br/info:eu-repo/semantics/openAccessInteligência artificialColorização de imagensBeyond landscapes : an exemplar-based image colorization methodinfo:eu-repo/semantics/publishedVersioninfo:eu-repo/semantics/masterThesismestradoreponame:Repositório Institucional da UFPEinstname:Universidade Federal de Pernambuco (UFPE)instacron:UFPEORIGINALDISSERTAÇÃO Saulo César Rodrigues Pereira Sobrinho.pdfDISSERTAÇÃO Saulo César Rodrigues Pereira Sobrinho.pdfapplication/pdf4589218https://repositorio.ufpe.br/bitstream/123456789/43483/1/DISSERTA%c3%87%c3%83O%20Saulo%20C%c3%a9sar%20Rodrigues%20Pereira%20Sobrinho.pdf507fe1419ecd61fc73d3ea2a96b0a682MD51CC-LICENSElicense_rdflicense_rdfapplication/rdf+xml; charset=utf-8811https://repositorio.ufpe.br/bitstream/123456789/43483/2/license_rdfe39d27027a6cc9cb039ad269a5db8e34MD52LICENSElicense.txtlicense.txttext/plain; charset=utf-82142https://repositorio.ufpe.br/bitstream/123456789/43483/3/license.txt6928b9260b07fb2755249a5ca9903395MD53TEXTDISSERTAÇÃO Saulo César Rodrigues Pereira Sobrinho.pdf.txtDISSERTAÇÃO Saulo César Rodrigues Pereira Sobrinho.pdf.txtExtracted texttext/plain144312https://repositorio.ufpe.br/bitstream/123456789/43483/4/DISSERTA%c3%87%c3%83O%20Saulo%20C%c3%a9sar%20Rodrigues%20Pereira%20Sobrinho.pdf.txt0f9f1a39795579d44f1b00ac780c8d88MD54THUMBNAILDISSERTAÇÃO Saulo César Rodrigues Pereira Sobrinho.pdf.jpgDISSERTAÇÃO Saulo César Rodrigues Pereira Sobrinho.pdf.jpgGenerated Thumbnailimage/jpeg1247https://repositorio.ufpe.br/bitstream/123456789/43483/5/DISSERTA%c3%87%c3%83O%20Saulo%20C%c3%a9sar%20Rodrigues%20Pereira%20Sobrinho.pdf.jpg5d298ddaa317789f547f8141c1705116MD55123456789/434832022-03-24 02:14:58.269oai:repositorio.ufpe.br:123456789/43483VGVybW8gZGUgRGVww7NzaXRvIExlZ2FsIGUgQXV0b3JpemHDp8OjbyBwYXJhIFB1YmxpY2HDp8OjbyBkZSBEb2N1bWVudG9zIG5vIFJlcG9zaXTDs3JpbyBEaWdpdGFsIGRhIFVGUEUKIAoKRGVjbGFybyBlc3RhciBjaWVudGUgZGUgcXVlIGVzdGUgVGVybW8gZGUgRGVww7NzaXRvIExlZ2FsIGUgQXV0b3JpemHDp8OjbyB0ZW0gbyBvYmpldGl2byBkZSBkaXZ1bGdhw6fDo28gZG9zIGRvY3VtZW50b3MgZGVwb3NpdGFkb3Mgbm8gUmVwb3NpdMOzcmlvIERpZ2l0YWwgZGEgVUZQRSBlIGRlY2xhcm8gcXVlOgoKSSAtICBvIGNvbnRlw7pkbyBkaXNwb25pYmlsaXphZG8gw6kgZGUgcmVzcG9uc2FiaWxpZGFkZSBkZSBzdWEgYXV0b3JpYTsKCklJIC0gbyBjb250ZcO6ZG8gw6kgb3JpZ2luYWwsIGUgc2UgbyB0cmFiYWxobyBlL291IHBhbGF2cmFzIGRlIG91dHJhcyBwZXNzb2FzIGZvcmFtIHV0aWxpemFkb3MsIGVzdGFzIGZvcmFtIGRldmlkYW1lbnRlIHJlY29uaGVjaWRhczsKCklJSSAtIHF1YW5kbyB0cmF0YXItc2UgZGUgVHJhYmFsaG8gZGUgQ29uY2x1c8OjbyBkZSBDdXJzbywgRGlzc2VydGHDp8OjbyBvdSBUZXNlOiBvIGFycXVpdm8gZGVwb3NpdGFkbyBjb3JyZXNwb25kZSDDoCB2ZXJzw6NvIGZpbmFsIGRvIHRyYWJhbGhvOwoKSVYgLSBxdWFuZG8gdHJhdGFyLXNlIGRlIFRyYWJhbGhvIGRlIENvbmNsdXPDo28gZGUgQ3Vyc28sIERpc3NlcnRhw6fDo28gb3UgVGVzZTogZXN0b3UgY2llbnRlIGRlIHF1ZSBhIGFsdGVyYcOnw6NvIGRhIG1vZGFsaWRhZGUgZGUgYWNlc3NvIGFvIGRvY3VtZW50byBhcMOzcyBvIGRlcMOzc2l0byBlIGFudGVzIGRlIGZpbmRhciBvIHBlcsOtb2RvIGRlIGVtYmFyZ28sIHF1YW5kbyBmb3IgZXNjb2xoaWRvIGFjZXNzbyByZXN0cml0bywgc2Vyw6EgcGVybWl0aWRhIG1lZGlhbnRlIHNvbGljaXRhw6fDo28gZG8gKGEpIGF1dG9yIChhKSBhbyBTaXN0ZW1hIEludGVncmFkbyBkZSBCaWJsaW90ZWNhcyBkYSBVRlBFIChTSUIvVUZQRSkuCgogClBhcmEgdHJhYmFsaG9zIGVtIEFjZXNzbyBBYmVydG86CgpOYSBxdWFsaWRhZGUgZGUgdGl0dWxhciBkb3MgZGlyZWl0b3MgYXV0b3JhaXMgZGUgYXV0b3IgcXVlIHJlY2FlbSBzb2JyZSBlc3RlIGRvY3VtZW50bywgZnVuZGFtZW50YWRvIG5hIExlaSBkZSBEaXJlaXRvIEF1dG9yYWwgbm8gOS42MTAsIGRlIDE5IGRlIGZldmVyZWlybyBkZSAxOTk4LCBhcnQuIDI5LCBpbmNpc28gSUlJLCBhdXRvcml6byBhIFVuaXZlcnNpZGFkZSBGZWRlcmFsIGRlIFBlcm5hbWJ1Y28gYSBkaXNwb25pYmlsaXphciBncmF0dWl0YW1lbnRlLCBzZW0gcmVzc2FyY2ltZW50byBkb3MgZGlyZWl0b3MgYXV0b3JhaXMsIHBhcmEgZmlucyBkZSBsZWl0dXJhLCBpbXByZXNzw6NvIGUvb3UgZG93bmxvYWQgKGFxdWlzacOnw6NvKSBhdHJhdsOpcyBkbyBzaXRlIGRvIFJlcG9zaXTDs3JpbyBEaWdpdGFsIGRhIFVGUEUgbm8gZW5kZXJlw6dvIGh0dHA6Ly93d3cucmVwb3NpdG9yaW8udWZwZS5iciwgYSBwYXJ0aXIgZGEgZGF0YSBkZSBkZXDDs3NpdG8uCgogClBhcmEgdHJhYmFsaG9zIGVtIEFjZXNzbyBSZXN0cml0bzoKCk5hIHF1YWxpZGFkZSBkZSB0aXR1bGFyIGRvcyBkaXJlaXRvcyBhdXRvcmFpcyBkZSBhdXRvciBxdWUgcmVjYWVtIHNvYnJlIGVzdGUgZG9jdW1lbnRvLCBmdW5kYW1lbnRhZG8gbmEgTGVpIGRlIERpcmVpdG8gQXV0b3JhbCBubyA5LjYxMCBkZSAxOSBkZSBmZXZlcmVpcm8gZGUgMTk5OCwgYXJ0LiAyOSwgaW5jaXNvIElJSSwgYXV0b3Jpem8gYSBVbml2ZXJzaWRhZGUgRmVkZXJhbCBkZSBQZXJuYW1idWNvIGEgZGlzcG9uaWJpbGl6YXIgZ3JhdHVpdGFtZW50ZSwgc2VtIHJlc3NhcmNpbWVudG8gZG9zIGRpcmVpdG9zIGF1dG9yYWlzLCBwYXJhIGZpbnMgZGUgbGVpdHVyYSwgaW1wcmVzc8OjbyBlL291IGRvd25sb2FkIChhcXVpc2nDp8OjbykgYXRyYXbDqXMgZG8gc2l0ZSBkbyBSZXBvc2l0w7NyaW8gRGlnaXRhbCBkYSBVRlBFIG5vIGVuZGVyZcOnbyBodHRwOi8vd3d3LnJlcG9zaXRvcmlvLnVmcGUuYnIsIHF1YW5kbyBmaW5kYXIgbyBwZXLDrW9kbyBkZSBlbWJhcmdvIGNvbmRpemVudGUgYW8gdGlwbyBkZSBkb2N1bWVudG8sIGNvbmZvcm1lIGluZGljYWRvIG5vIGNhbXBvIERhdGEgZGUgRW1iYXJnby4KRepositório InstitucionalPUBhttps://repositorio.ufpe.br/oai/requestattena@ufpe.bropendoar:22212022-03-24T05:14:58Repositório Institucional da UFPE - Universidade Federal de Pernambuco (UFPE)false |
dc.title.pt_BR.fl_str_mv |
Beyond landscapes : an exemplar-based image colorization method |
title |
Beyond landscapes : an exemplar-based image colorization method |
spellingShingle |
Beyond landscapes : an exemplar-based image colorization method PEREIRA SOBRINHO, Saulo César Rodrigues Inteligência artificial Colorização de imagens |
title_short |
Beyond landscapes : an exemplar-based image colorization method |
title_full |
Beyond landscapes : an exemplar-based image colorization method |
title_fullStr |
Beyond landscapes : an exemplar-based image colorization method |
title_full_unstemmed |
Beyond landscapes : an exemplar-based image colorization method |
title_sort |
Beyond landscapes : an exemplar-based image colorization method |
author |
PEREIRA SOBRINHO, Saulo César Rodrigues |
author_facet |
PEREIRA SOBRINHO, Saulo César Rodrigues |
author_role |
author |
dc.contributor.authorLattes.pt_BR.fl_str_mv |
http://lattes.cnpq.br/9595267291106910 |
dc.contributor.advisorLattes.pt_BR.fl_str_mv |
http://lattes.cnpq.br/7532050172035129 |
dc.contributor.author.fl_str_mv |
PEREIRA SOBRINHO, Saulo César Rodrigues |
dc.contributor.advisor1.fl_str_mv |
KELNER, Judith |
contributor_str_mv |
KELNER, Judith |
dc.subject.por.fl_str_mv |
Inteligência artificial Colorização de imagens |
topic |
Inteligência artificial Colorização de imagens |
description |
Image colorization consists in, given a grayscale image, generating a plausible color version of this image, which can be performed as a manual/artistic process1 but also as a computer assisted or even fully automated process. Colorization is a underconstrained problem, which requires extra information in order to provide a unique solution. This dissertation focuses on exemplar-based colorization methods, in which the extra informa- tion comes from a user-selected color reference image with similar semantic content to the target. While the user selects the reference based on content similarity, the algorithms estimate similarity based on local descriptors of image regions. This difference in abstrac- tion between the user and algorithm perspective can lead to the algorithms not always being able to transfer colors between semantic corresponding elements in the image pair, specially in images in which the mapping between content/color and local descriptors is complex. Most exemplar-based methods in the literature display successful examples mostly limited to simple instances, such as landscapes, animals and simple buildings. Based on this observation, in this research we propose a new exemplar-based method that aims at generating plausible colorizations for a wider range of image pairs, including images of higher complexity. To that end, the proposed method features a two-stage clas- sification scheme that uses the available features in a more consistent manner and makes the initial color assignments more robust. It also includes an edge-aware relabeling method that enhances the spatial coherence and mitigates the impact of the multimodality, inher- ent to the colorization problem, over the method’s colorized outputs. In this dissertation, we present a broad review of the colorization literature introducing a taxonomy that cat- egorizes colorization techniques based on the source of prior information used to guide their color assignments. The proposed method pipeline is then described in details, and its key modules are validated through experiments. Moreover, a comparative analysis is performed which subjects the proposed and baseline methods to different source/target pairs to visually assess and compare their results. Experimental results indicate that the proposed method yields colorization results that are more coherent and of higher visual quality compared to two state-of-the-art exemplar-based colorization algorithms, both in simple and complex image sets. The results also indicate that exemplar-based methods can achieve results of comparable visual aspect to those of modern deep learning approaches while allowing more user control. |
publishDate |
2018 |
dc.date.issued.fl_str_mv |
2018-12-19 |
dc.date.accessioned.fl_str_mv |
2022-03-23T17:45:10Z |
dc.date.available.fl_str_mv |
2022-03-23T17:45:10Z |
dc.type.status.fl_str_mv |
info:eu-repo/semantics/publishedVersion |
dc.type.driver.fl_str_mv |
info:eu-repo/semantics/masterThesis |
format |
masterThesis |
status_str |
publishedVersion |
dc.identifier.citation.fl_str_mv |
PEREIRA SOBRINHO, Saulo César Rodrigues. Beyond landscapes: an exemplar-based image colorization method. 2018. Dissertação (Mestrado em Ciência da Computação) - Universidade Federal de Pernambuco, Recife, 2018. |
dc.identifier.uri.fl_str_mv |
https://repositorio.ufpe.br/handle/123456789/43483 |
identifier_str_mv |
PEREIRA SOBRINHO, Saulo César Rodrigues. Beyond landscapes: an exemplar-based image colorization method. 2018. Dissertação (Mestrado em Ciência da Computação) - Universidade Federal de Pernambuco, Recife, 2018. |
url |
https://repositorio.ufpe.br/handle/123456789/43483 |
dc.language.iso.fl_str_mv |
eng |
language |
eng |
dc.rights.driver.fl_str_mv |
Attribution-NonCommercial-NoDerivs 3.0 Brazil http://creativecommons.org/licenses/by-nc-nd/3.0/br/ info:eu-repo/semantics/openAccess |
rights_invalid_str_mv |
Attribution-NonCommercial-NoDerivs 3.0 Brazil http://creativecommons.org/licenses/by-nc-nd/3.0/br/ |
eu_rights_str_mv |
openAccess |
dc.publisher.none.fl_str_mv |
Universidade Federal de Pernambuco |
dc.publisher.program.fl_str_mv |
Programa de Pos Graduacao em Ciencia da Computacao |
dc.publisher.initials.fl_str_mv |
UFPE |
dc.publisher.country.fl_str_mv |
Brasil |
publisher.none.fl_str_mv |
Universidade Federal de Pernambuco |
dc.source.none.fl_str_mv |
reponame:Repositório Institucional da UFPE instname:Universidade Federal de Pernambuco (UFPE) instacron:UFPE |
instname_str |
Universidade Federal de Pernambuco (UFPE) |
instacron_str |
UFPE |
institution |
UFPE |
reponame_str |
Repositório Institucional da UFPE |
collection |
Repositório Institucional da UFPE |
bitstream.url.fl_str_mv |
https://repositorio.ufpe.br/bitstream/123456789/43483/1/DISSERTA%c3%87%c3%83O%20Saulo%20C%c3%a9sar%20Rodrigues%20Pereira%20Sobrinho.pdf https://repositorio.ufpe.br/bitstream/123456789/43483/2/license_rdf https://repositorio.ufpe.br/bitstream/123456789/43483/3/license.txt https://repositorio.ufpe.br/bitstream/123456789/43483/4/DISSERTA%c3%87%c3%83O%20Saulo%20C%c3%a9sar%20Rodrigues%20Pereira%20Sobrinho.pdf.txt https://repositorio.ufpe.br/bitstream/123456789/43483/5/DISSERTA%c3%87%c3%83O%20Saulo%20C%c3%a9sar%20Rodrigues%20Pereira%20Sobrinho.pdf.jpg |
bitstream.checksum.fl_str_mv |
507fe1419ecd61fc73d3ea2a96b0a682 e39d27027a6cc9cb039ad269a5db8e34 6928b9260b07fb2755249a5ca9903395 0f9f1a39795579d44f1b00ac780c8d88 5d298ddaa317789f547f8141c1705116 |
bitstream.checksumAlgorithm.fl_str_mv |
MD5 MD5 MD5 MD5 MD5 |
repository.name.fl_str_mv |
Repositório Institucional da UFPE - Universidade Federal de Pernambuco (UFPE) |
repository.mail.fl_str_mv |
attena@ufpe.br |
_version_ |
1802310662974603264 |