FERAtt : new architecture learning for facial expression characterization
Autor(a) principal: | |
---|---|
Data de Publicação: | 2019 |
Tipo de documento: | Tese |
Idioma: | por |
Título da fonte: | Repositório Institucional da UFPE |
Texto Completo: | https://repositorio.ufpe.br/handle/123456789/36907 |
Resumo: | Affective computing is a branch of artificial intelligence responsible for the development of equipment and systems capable of interpreting, recognizing and processing human motions. The automatic understanding of human behavior is of great interest since it allows the creation of new human-machine interfaces. Within this behavior, facial expressions are the most convenient because of the wide range of emotions that can be transmitted. The human face conveys a large part of our emotional behavior. We use facial expressions to demonstrate our emotional states and to communicate our interactions. In addition, we express and read emotions through the expressions of faces without effort. However, automatic understanding of facial expressions is a task not yet solved from the computational point of view, especially in the presence of highly variable expression, artifacts, and poses. Currently, obtaining a semantic representation of expressions is a challenge for the affective computing community. This work promotes the field of facial expression recognition by providing new tools for the representation analysis of expression in static images. First, we present an analysis of the methods of extracting characteristics and methods of combining classifiers based on sparse representation applied to the facial expression recognition problem. We propose a system of multi-classifiers based on trainable combination rules for this problem. Second, we present a study of the main deep neural networks architectures applied in this problem. A comparative analysis allows to determine the best models of deep learning for the classification of facial expressions. Third, we propose a new supervised and semi-supervised representation approach based on metric learning. This type of approach allows us to obtain semantic representations of the facial expressions that are evaluated in this work. We propose a new loss function that generates Gaussian structures in the embedded space of facial expressions. Lastly, we propose FERAtt, a new end-to-end network architecture for facial expression recognition with an attention model. The FERAtt neuralnet focuses attention in the human face and uses a Gaussian space representation for expression recognition. We devise this architecture based on two fundamental complementary components: (1) facial image correction and attention and (2) facial expression representation and classification. |
id |
UFPE_5d9d7ee3eb2499cbe3a9ff7df601ca34 |
---|---|
oai_identifier_str |
oai:repositorio.ufpe.br:123456789/36907 |
network_acronym_str |
UFPE |
network_name_str |
Repositório Institucional da UFPE |
repository_id_str |
2221 |
spelling |
FERNÁNDEZ, Pedro Diamel Marrerohttp://lattes.cnpq.br/8806516920946189http://lattes.cnpq.br/3084134533707587REN, Tsang Ing2020-03-09T20:51:47Z2020-03-09T20:51:47Z2019-08-12FERNÁNDEZ, Pedro Diamel Marrero. FERAtt: new architecture learning for facial expression characterization. 2019. Tese (Doutorado em Ciência da Computação) – Universidade Federal de Pernambuco, Recife, 2019.https://repositorio.ufpe.br/handle/123456789/36907Affective computing is a branch of artificial intelligence responsible for the development of equipment and systems capable of interpreting, recognizing and processing human motions. The automatic understanding of human behavior is of great interest since it allows the creation of new human-machine interfaces. Within this behavior, facial expressions are the most convenient because of the wide range of emotions that can be transmitted. The human face conveys a large part of our emotional behavior. We use facial expressions to demonstrate our emotional states and to communicate our interactions. In addition, we express and read emotions through the expressions of faces without effort. However, automatic understanding of facial expressions is a task not yet solved from the computational point of view, especially in the presence of highly variable expression, artifacts, and poses. Currently, obtaining a semantic representation of expressions is a challenge for the affective computing community. This work promotes the field of facial expression recognition by providing new tools for the representation analysis of expression in static images. First, we present an analysis of the methods of extracting characteristics and methods of combining classifiers based on sparse representation applied to the facial expression recognition problem. We propose a system of multi-classifiers based on trainable combination rules for this problem. Second, we present a study of the main deep neural networks architectures applied in this problem. A comparative analysis allows to determine the best models of deep learning for the classification of facial expressions. Third, we propose a new supervised and semi-supervised representation approach based on metric learning. This type of approach allows us to obtain semantic representations of the facial expressions that are evaluated in this work. We propose a new loss function that generates Gaussian structures in the embedded space of facial expressions. Lastly, we propose FERAtt, a new end-to-end network architecture for facial expression recognition with an attention model. The FERAtt neuralnet focuses attention in the human face and uses a Gaussian space representation for expression recognition. We devise this architecture based on two fundamental complementary components: (1) facial image correction and attention and (2) facial expression representation and classification.FACEPEComputação afetiva é um ramo da inteligência artificial responsável pelo desenvolvimento de equipamentos e sistemas capazes de interpretar, reconhecer e processar emoções humanas. A compreensão automática do comportamento humano é de grande interesse, já que permitiria a criação de novas interfaces homem-máquina. O rosto humano transmite uma grande parte do nosso comportamento emocional. Usamos expressões faciais para demonstrar emoções e para melhorar nossas interações sem esforço, devido a que as expressões são um reflexo incorporado a nosso mecanismo de comunicação. No entanto, a compreensão automática das expressões faciais é uma tarefa ainda não solucionada do ponto de vista computacional, especialmente na presença de expressão altamente variável, artefatos e poses. Atualmente, obter uma representação semântica de expressões faciais é um desafio para a comunidade de computação afetiva. Este trabalho promove o campo do reconhecimento da expressão facial, fornecendo novas ferramentas para a análise de expressão em imagens estáticas a partir do estudo da representação no espaço de características. Em primeiro lugar, apresentamos uma revisão dos principais métodos de extração de características e dos métodos de combinação de classificadores com base em representação escassa que são aplicadas aos problemas de reconhecimento de expressão facial. Propomos um sistema de multi-classificadores baseado em regras de combinação treináveis para a classificação das expressões faciais. Em segundo lugar, apresentamos um estudo das principais arquiteturas de redes neurais profundas aplicadas neste problema. Uma análise comparativa nos permite determinar os melhores modelos de aprendizagem profunda para a classificação das expressões. Em terceiro lugar, propomos uma nova abordagem supervisionada e semi-supervisionada de representação baseada na aprendizagem por métrica. Este tipo de abordagem nos permite obter representações semânticas das expressões faciais que são avaliadas neste trabalho. Propomos uma nova função de perda que geram estruturas Gaussianas no espaço de representação. Finalmente, propomos FERAtt, uma nova arquitetura de rede ponta-a-ponta para o reconhecimento de expressões faciais com um modelo de atenção. A rede FERAtt, concentra a atenção no rostro humano e usa uma representação do espaço Gaussiano para reconhecimento de expressão. Concebemos essa arquitetura com base em dois componentes fundamentais: (1) correção e atenção à imagem facial; e (2) representação e classificação da expressão facial.porUniversidade Federal de PernambucoPrograma de Pos Graduacao em Ciencia da ComputacaoUFPEBrasilAttribution-NonCommercial-NoDerivs 3.0 Brazilhttp://creativecommons.org/licenses/by-nc-nd/3.0/br/info:eu-repo/semantics/openAccessInteligência computacionalExpressões faciaisAprendizagem profundaFERAtt : new architecture learning for facial expression characterizationinfo:eu-repo/semantics/publishedVersioninfo:eu-repo/semantics/doctoralThesisdoutoradoreponame:Repositório Institucional da UFPEinstname:Universidade Federal de Pernambuco (UFPE)instacron:UFPELICENSElicense.txtlicense.txttext/plain; charset=utf-82310https://repositorio.ufpe.br/bitstream/123456789/36907/3/license.txtbd573a5ca8288eb7272482765f819534MD53ORIGINALTESE Pedro Diamel Marrero Fernandez.pdfTESE Pedro Diamel Marrero Fernandez.pdfapplication/pdf11044774https://repositorio.ufpe.br/bitstream/123456789/36907/1/TESE%20Pedro%20Diamel%20Marrero%20Fernandez.pdf3d95849f7a9c0c903e37a1bd9d9881e2MD51CC-LICENSElicense_rdflicense_rdfapplication/rdf+xml; charset=utf-8811https://repositorio.ufpe.br/bitstream/123456789/36907/2/license_rdfe39d27027a6cc9cb039ad269a5db8e34MD52TEXTTESE Pedro Diamel Marrero Fernandez.pdf.txtTESE Pedro Diamel Marrero Fernandez.pdf.txtExtracted texttext/plain240133https://repositorio.ufpe.br/bitstream/123456789/36907/4/TESE%20Pedro%20Diamel%20Marrero%20Fernandez.pdf.txtb3a62e155bd7510ab38925791dbbba15MD54THUMBNAILTESE Pedro Diamel Marrero Fernandez.pdf.jpgTESE Pedro Diamel Marrero Fernandez.pdf.jpgGenerated Thumbnailimage/jpeg1204https://repositorio.ufpe.br/bitstream/123456789/36907/5/TESE%20Pedro%20Diamel%20Marrero%20Fernandez.pdf.jpgd8ebad0bb65855f07ecdb20b94062b55MD55123456789/369072020-03-10 02:12:47.699oai:repositorio.ufpe.br:123456789/36907TGljZW7Dp2EgZGUgRGlzdHJpYnVpw6fDo28gTsOjbyBFeGNsdXNpdmEKClRvZG8gZGVwb3NpdGFudGUgZGUgbWF0ZXJpYWwgbm8gUmVwb3NpdMOzcmlvIEluc3RpdHVjaW9uYWwgKFJJKSBkZXZlIGNvbmNlZGVyLCDDoCBVbml2ZXJzaWRhZGUgRmVkZXJhbCBkZSBQZXJuYW1idWNvIChVRlBFKSwgdW1hIExpY2Vuw6dhIGRlIERpc3RyaWJ1acOnw6NvIE7Do28gRXhjbHVzaXZhIHBhcmEgbWFudGVyIGUgdG9ybmFyIGFjZXNzw612ZWlzIG9zIHNldXMgZG9jdW1lbnRvcywgZW0gZm9ybWF0byBkaWdpdGFsLCBuZXN0ZSByZXBvc2l0w7NyaW8uCgpDb20gYSBjb25jZXNzw6NvIGRlc3RhIGxpY2Vuw6dhIG7Do28gZXhjbHVzaXZhLCBvIGRlcG9zaXRhbnRlIG1hbnTDqW0gdG9kb3Mgb3MgZGlyZWl0b3MgZGUgYXV0b3IuCl9fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fXwoKTGljZW7Dp2EgZGUgRGlzdHJpYnVpw6fDo28gTsOjbyBFeGNsdXNpdmEKCkFvIGNvbmNvcmRhciBjb20gZXN0YSBsaWNlbsOnYSBlIGFjZWl0w6EtbGEsIHZvY8OqIChhdXRvciBvdSBkZXRlbnRvciBkb3MgZGlyZWl0b3MgYXV0b3JhaXMpOgoKYSkgRGVjbGFyYSBxdWUgY29uaGVjZSBhIHBvbMOtdGljYSBkZSBjb3B5cmlnaHQgZGEgZWRpdG9yYSBkbyBzZXUgZG9jdW1lbnRvOwpiKSBEZWNsYXJhIHF1ZSBjb25oZWNlIGUgYWNlaXRhIGFzIERpcmV0cml6ZXMgcGFyYSBvIFJlcG9zaXTDs3JpbyBJbnN0aXR1Y2lvbmFsIGRhIFVGUEU7CmMpIENvbmNlZGUgw6AgVUZQRSBvIGRpcmVpdG8gbsOjbyBleGNsdXNpdm8gZGUgYXJxdWl2YXIsIHJlcHJvZHV6aXIsIGNvbnZlcnRlciAoY29tbyBkZWZpbmlkbyBhIHNlZ3VpciksIGNvbXVuaWNhciBlL291IGRpc3RyaWJ1aXIsIG5vIFJJLCBvIGRvY3VtZW50byBlbnRyZWd1ZSAoaW5jbHVpbmRvIG8gcmVzdW1vL2Fic3RyYWN0KSBlbSBmb3JtYXRvIGRpZ2l0YWwgb3UgcG9yIG91dHJvIG1laW87CmQpIERlY2xhcmEgcXVlIGF1dG9yaXphIGEgVUZQRSBhIGFycXVpdmFyIG1haXMgZGUgdW1hIGPDs3BpYSBkZXN0ZSBkb2N1bWVudG8gZSBjb252ZXJ0w6otbG8sIHNlbSBhbHRlcmFyIG8gc2V1IGNvbnRlw7pkbywgcGFyYSBxdWFscXVlciBmb3JtYXRvIGRlIGZpY2hlaXJvLCBtZWlvIG91IHN1cG9ydGUsIHBhcmEgZWZlaXRvcyBkZSBzZWd1cmFuw6dhLCBwcmVzZXJ2YcOnw6NvIChiYWNrdXApIGUgYWNlc3NvOwplKSBEZWNsYXJhIHF1ZSBvIGRvY3VtZW50byBzdWJtZXRpZG8gw6kgbyBzZXUgdHJhYmFsaG8gb3JpZ2luYWwgZSBxdWUgZGV0w6ltIG8gZGlyZWl0byBkZSBjb25jZWRlciBhIHRlcmNlaXJvcyBvcyBkaXJlaXRvcyBjb250aWRvcyBuZXN0YSBsaWNlbsOnYS4gRGVjbGFyYSB0YW1iw6ltIHF1ZSBhIGVudHJlZ2EgZG8gZG9jdW1lbnRvIG7Do28gaW5mcmluZ2Ugb3MgZGlyZWl0b3MgZGUgb3V0cmEgcGVzc29hIG91IGVudGlkYWRlOwpmKSBEZWNsYXJhIHF1ZSwgbm8gY2FzbyBkbyBkb2N1bWVudG8gc3VibWV0aWRvIGNvbnRlciBtYXRlcmlhbCBkbyBxdWFsIG7Do28gZGV0w6ltIG9zIGRpcmVpdG9zIGRlCmF1dG9yLCBvYnRldmUgYSBhdXRvcml6YcOnw6NvIGlycmVzdHJpdGEgZG8gcmVzcGVjdGl2byBkZXRlbnRvciBkZXNzZXMgZGlyZWl0b3MgcGFyYSBjZWRlciDDoApVRlBFIG9zIGRpcmVpdG9zIHJlcXVlcmlkb3MgcG9yIGVzdGEgTGljZW7Dp2EgZSBhdXRvcml6YXIgYSB1bml2ZXJzaWRhZGUgYSB1dGlsaXrDoS1sb3MgbGVnYWxtZW50ZS4gRGVjbGFyYSB0YW1iw6ltIHF1ZSBlc3NlIG1hdGVyaWFsIGN1am9zIGRpcmVpdG9zIHPDo28gZGUgdGVyY2Vpcm9zIGVzdMOhIGNsYXJhbWVudGUgaWRlbnRpZmljYWRvIGUgcmVjb25oZWNpZG8gbm8gdGV4dG8gb3UgY29udGXDumRvIGRvIGRvY3VtZW50byBlbnRyZWd1ZTsKZykgU2UgbyBkb2N1bWVudG8gZW50cmVndWUgw6kgYmFzZWFkbyBlbSB0cmFiYWxobyBmaW5hbmNpYWRvIG91IGFwb2lhZG8gcG9yIG91dHJhIGluc3RpdHVpw6fDo28gcXVlIG7Do28gYSBVRlBFLCBkZWNsYXJhIHF1ZSBjdW1wcml1IHF1YWlzcXVlciBvYnJpZ2HDp8O1ZXMgZXhpZ2lkYXMgcGVsbyByZXNwZWN0aXZvIGNvbnRyYXRvIG91IGFjb3Jkby4KCkEgVUZQRSBpZGVudGlmaWNhcsOhIGNsYXJhbWVudGUgbyhzKSBub21lKHMpIGRvKHMpIGF1dG9yIChlcykgZG9zIGRpcmVpdG9zIGRvIGRvY3VtZW50byBlbnRyZWd1ZSBlIG7Do28gZmFyw6EgcXVhbHF1ZXIgYWx0ZXJhw6fDo28sIHBhcmEgYWzDqW0gZG8gcHJldmlzdG8gbmEgYWzDrW5lYSBjKS4KRepositório InstitucionalPUBhttps://repositorio.ufpe.br/oai/requestattena@ufpe.bropendoar:22212020-03-10T05:12:47Repositório Institucional da UFPE - Universidade Federal de Pernambuco (UFPE)false |
dc.title.pt_BR.fl_str_mv |
FERAtt : new architecture learning for facial expression characterization |
title |
FERAtt : new architecture learning for facial expression characterization |
spellingShingle |
FERAtt : new architecture learning for facial expression characterization FERNÁNDEZ, Pedro Diamel Marrero Inteligência computacional Expressões faciais Aprendizagem profunda |
title_short |
FERAtt : new architecture learning for facial expression characterization |
title_full |
FERAtt : new architecture learning for facial expression characterization |
title_fullStr |
FERAtt : new architecture learning for facial expression characterization |
title_full_unstemmed |
FERAtt : new architecture learning for facial expression characterization |
title_sort |
FERAtt : new architecture learning for facial expression characterization |
author |
FERNÁNDEZ, Pedro Diamel Marrero |
author_facet |
FERNÁNDEZ, Pedro Diamel Marrero |
author_role |
author |
dc.contributor.authorLattes.pt_BR.fl_str_mv |
http://lattes.cnpq.br/8806516920946189 |
dc.contributor.advisorLattes.pt_BR.fl_str_mv |
http://lattes.cnpq.br/3084134533707587 |
dc.contributor.author.fl_str_mv |
FERNÁNDEZ, Pedro Diamel Marrero |
dc.contributor.advisor1.fl_str_mv |
REN, Tsang Ing |
contributor_str_mv |
REN, Tsang Ing |
dc.subject.por.fl_str_mv |
Inteligência computacional Expressões faciais Aprendizagem profunda |
topic |
Inteligência computacional Expressões faciais Aprendizagem profunda |
description |
Affective computing is a branch of artificial intelligence responsible for the development of equipment and systems capable of interpreting, recognizing and processing human motions. The automatic understanding of human behavior is of great interest since it allows the creation of new human-machine interfaces. Within this behavior, facial expressions are the most convenient because of the wide range of emotions that can be transmitted. The human face conveys a large part of our emotional behavior. We use facial expressions to demonstrate our emotional states and to communicate our interactions. In addition, we express and read emotions through the expressions of faces without effort. However, automatic understanding of facial expressions is a task not yet solved from the computational point of view, especially in the presence of highly variable expression, artifacts, and poses. Currently, obtaining a semantic representation of expressions is a challenge for the affective computing community. This work promotes the field of facial expression recognition by providing new tools for the representation analysis of expression in static images. First, we present an analysis of the methods of extracting characteristics and methods of combining classifiers based on sparse representation applied to the facial expression recognition problem. We propose a system of multi-classifiers based on trainable combination rules for this problem. Second, we present a study of the main deep neural networks architectures applied in this problem. A comparative analysis allows to determine the best models of deep learning for the classification of facial expressions. Third, we propose a new supervised and semi-supervised representation approach based on metric learning. This type of approach allows us to obtain semantic representations of the facial expressions that are evaluated in this work. We propose a new loss function that generates Gaussian structures in the embedded space of facial expressions. Lastly, we propose FERAtt, a new end-to-end network architecture for facial expression recognition with an attention model. The FERAtt neuralnet focuses attention in the human face and uses a Gaussian space representation for expression recognition. We devise this architecture based on two fundamental complementary components: (1) facial image correction and attention and (2) facial expression representation and classification. |
publishDate |
2019 |
dc.date.issued.fl_str_mv |
2019-08-12 |
dc.date.accessioned.fl_str_mv |
2020-03-09T20:51:47Z |
dc.date.available.fl_str_mv |
2020-03-09T20:51:47Z |
dc.type.status.fl_str_mv |
info:eu-repo/semantics/publishedVersion |
dc.type.driver.fl_str_mv |
info:eu-repo/semantics/doctoralThesis |
format |
doctoralThesis |
status_str |
publishedVersion |
dc.identifier.citation.fl_str_mv |
FERNÁNDEZ, Pedro Diamel Marrero. FERAtt: new architecture learning for facial expression characterization. 2019. Tese (Doutorado em Ciência da Computação) – Universidade Federal de Pernambuco, Recife, 2019. |
dc.identifier.uri.fl_str_mv |
https://repositorio.ufpe.br/handle/123456789/36907 |
identifier_str_mv |
FERNÁNDEZ, Pedro Diamel Marrero. FERAtt: new architecture learning for facial expression characterization. 2019. Tese (Doutorado em Ciência da Computação) – Universidade Federal de Pernambuco, Recife, 2019. |
url |
https://repositorio.ufpe.br/handle/123456789/36907 |
dc.language.iso.fl_str_mv |
por |
language |
por |
dc.rights.driver.fl_str_mv |
Attribution-NonCommercial-NoDerivs 3.0 Brazil http://creativecommons.org/licenses/by-nc-nd/3.0/br/ info:eu-repo/semantics/openAccess |
rights_invalid_str_mv |
Attribution-NonCommercial-NoDerivs 3.0 Brazil http://creativecommons.org/licenses/by-nc-nd/3.0/br/ |
eu_rights_str_mv |
openAccess |
dc.publisher.none.fl_str_mv |
Universidade Federal de Pernambuco |
dc.publisher.program.fl_str_mv |
Programa de Pos Graduacao em Ciencia da Computacao |
dc.publisher.initials.fl_str_mv |
UFPE |
dc.publisher.country.fl_str_mv |
Brasil |
publisher.none.fl_str_mv |
Universidade Federal de Pernambuco |
dc.source.none.fl_str_mv |
reponame:Repositório Institucional da UFPE instname:Universidade Federal de Pernambuco (UFPE) instacron:UFPE |
instname_str |
Universidade Federal de Pernambuco (UFPE) |
instacron_str |
UFPE |
institution |
UFPE |
reponame_str |
Repositório Institucional da UFPE |
collection |
Repositório Institucional da UFPE |
bitstream.url.fl_str_mv |
https://repositorio.ufpe.br/bitstream/123456789/36907/3/license.txt https://repositorio.ufpe.br/bitstream/123456789/36907/1/TESE%20Pedro%20Diamel%20Marrero%20Fernandez.pdf https://repositorio.ufpe.br/bitstream/123456789/36907/2/license_rdf https://repositorio.ufpe.br/bitstream/123456789/36907/4/TESE%20Pedro%20Diamel%20Marrero%20Fernandez.pdf.txt https://repositorio.ufpe.br/bitstream/123456789/36907/5/TESE%20Pedro%20Diamel%20Marrero%20Fernandez.pdf.jpg |
bitstream.checksum.fl_str_mv |
bd573a5ca8288eb7272482765f819534 3d95849f7a9c0c903e37a1bd9d9881e2 e39d27027a6cc9cb039ad269a5db8e34 b3a62e155bd7510ab38925791dbbba15 d8ebad0bb65855f07ecdb20b94062b55 |
bitstream.checksumAlgorithm.fl_str_mv |
MD5 MD5 MD5 MD5 MD5 |
repository.name.fl_str_mv |
Repositório Institucional da UFPE - Universidade Federal de Pernambuco (UFPE) |
repository.mail.fl_str_mv |
attena@ufpe.br |
_version_ |
1802310899006963712 |