Learning nonlinear differentiable models for signals and systems: with applications
Autor(a) principal: | |
---|---|
Data de Publicação: | 2020 |
Tipo de documento: | Tese |
Idioma: | eng |
Título da fonte: | Repositório Institucional da UFMG |
Texto Completo: | http://hdl.handle.net/1843/33922 |
Resumo: | Building empirical models from data is of fundamental importance in engineering, and pushing the boundaries of current engineering technology requires us to model and understand nonlinear systems. In this thesis, nonlinear differentiable models and its applications are studied. This class of models has gained traction in machine learning tasks with the introduction of deep learning. Indeed, deep models of stacked differentiable components have recently achieved super-human performance on several tasks, including computer games, image classification, and medical diagnosis. The application of nonlinear differentiable models is studied for modeling signals and systems both for engineering and machine learning applications. One central question is the role of recurrence and the pros and cons of recurrent and feedforward models. The question is approached from more than one angle: 1) by studying the effect of recurrence in neural networks in terms of robustness to noise, computational cost, and convergence; 2) by analyzing the smoothness of the cost function in nonlinear system identification problems and its relation to the model internal dynamics – and proposing the use of a technique called multiple shooting for improving the cost-function smoothness; and, 3) by investigating the interplay between the internal dynamics, the attractors and the expressiveness of the model in deep recurrent neural networks. The more applied part of the thesis consists of the use of deep neural networks to solve complex tasks and to model nonlinear behavior from real data. Data from the Telehealth Center of Minas Gerais is used to train a deep neural network capable of identifying abnormalities in the electrocardiogram exam with performance superior to the medical residents in the studied scenario. Also, a deep neural network is used for modeling an electronic oscillator and an F-16 aircraft using data from ground vibration experiments, obtaining competitive results in both cases. |
id |
UFMG_5d381463eca2d81e2b2d92a513c28737 |
---|---|
oai_identifier_str |
oai:repositorio.ufmg.br:1843/33922 |
network_acronym_str |
UFMG |
network_name_str |
Repositório Institucional da UFMG |
repository_id_str |
|
spelling |
Luis Antonio Aguirrehttp://lattes.cnpq.br/6682146998710900Thomas B. SchonEduardo Mazoni Andrade Marçal MendesFrederico Gadelha GuimarãesGuilherme de Alencar BarretoLeandro dos Santos CoelhoMaarten Schoukenshttp://lattes.cnpq.br/0898576944135254Antônio Horta Ribeiro2020-08-07T19:21:43Z2020-08-07T19:21:43Z2020-03-03http://hdl.handle.net/1843/339220000-0003-3632-8529Building empirical models from data is of fundamental importance in engineering, and pushing the boundaries of current engineering technology requires us to model and understand nonlinear systems. In this thesis, nonlinear differentiable models and its applications are studied. This class of models has gained traction in machine learning tasks with the introduction of deep learning. Indeed, deep models of stacked differentiable components have recently achieved super-human performance on several tasks, including computer games, image classification, and medical diagnosis. The application of nonlinear differentiable models is studied for modeling signals and systems both for engineering and machine learning applications. One central question is the role of recurrence and the pros and cons of recurrent and feedforward models. The question is approached from more than one angle: 1) by studying the effect of recurrence in neural networks in terms of robustness to noise, computational cost, and convergence; 2) by analyzing the smoothness of the cost function in nonlinear system identification problems and its relation to the model internal dynamics – and proposing the use of a technique called multiple shooting for improving the cost-function smoothness; and, 3) by investigating the interplay between the internal dynamics, the attractors and the expressiveness of the model in deep recurrent neural networks. The more applied part of the thesis consists of the use of deep neural networks to solve complex tasks and to model nonlinear behavior from real data. Data from the Telehealth Center of Minas Gerais is used to train a deep neural network capable of identifying abnormalities in the electrocardiogram exam with performance superior to the medical residents in the studied scenario. Also, a deep neural network is used for modeling an electronic oscillator and an F-16 aircraft using data from ground vibration experiments, obtaining competitive results in both cases.Construir modelos empíricos a partir de dados é de fundamental importância em engenharia e, além disso, o entendimento e a capacidade de modelar sistemas não lineares são necessários para o desenvolvimento de tecnologias de fronteira. Nesse trabalho, modelos diferenciáveis não lineares e suas aplicações são estudados. Esta classe de modelos tem ganhado força na área de aprendizado de máquina com a introdução do aprendizado profundo. De fato, modelos profundos de componentes diferenciáveis alcançaram, recentemente, desempenho superior ao humano em diversas tarefas, incluindo a competição em jogos digitais, classificação de imagens e diagnóstico de exames médicos. A aplicação de modelos não lineares diferenciáveis é estudada para modelar sinais e sistemas, tanto no contexto de aplicações em engenharia quanto no contexto de aprendizado de máquina. Uma questão central é o papel da recorrência, e os prós e os contras de modelos recorrentes. A questão é abordada de mais de um ângulo: 1) estudando o efeito da recorrência em redes neurais em termos da robustez a ruído, custo computacional e convergência; 2) analisando a suavidade da função de custo na identificação de sistemas não lineares e a relação com a dinâmica interna do modelo – e propondo o uso da técnica de múltiplos tiros para melhorar a suavidade da função custo; e, 3) investigando a relação entre dinâmica interna, atractores e expressividade do modelo em redes neurais recorrentes. A parte mais aplicada desta tese consiste no uso de redes neurais profundas para resolver tarefas complexas e modelar comportamento não linear a partir de dados reais. Dados do Centro de Telessaúde do estado de Minas Gerais são usados para treinar uma rede neural capaz de identificar abnormalidades no eletrocardiograma com desempenho superior ao de residentes de medicina no cenário estudado. Além disso, uma rede neural profunda é usada para modelar um oscilador eletrônico e uma aeronave F-16 usando dados de um ensaio de vibrações, obtendo resultados competitivos nos dois casos.CNPq - Conselho Nacional de Desenvolvimento Científico e TecnológicoCAPES - Coordenação de Aperfeiçoamento de Pessoal de Nível SuperiorengUniversidade Federal de Minas GeraisPrograma de Pós-Graduação em Engenharia ElétricaUFMGBrasilENG - DEPARTAMENTO DE ENGENHARIA ELÉTRICAENG - DEPARTAMENTO DE ENGENHARIA ELETRÔNICAhttp://creativecommons.org/licenses/by-nc-nd/3.0/pt/info:eu-repo/semantics/openAccessEngenharia elétricaAprendizado do computadorAprendizado profundoIdentificação de sistemasSistemas não linearesNonlinear systemsDifferentiable modelsDeep learningSystem identificationMachine learningLearning nonlinear differentiable models for signals and systems: with applicationsAprendendo modelos não-lineares diferenciáveis para sinais e sistemas: com aplicaçõesinfo:eu-repo/semantics/publishedVersioninfo:eu-repo/semantics/doctoralThesisreponame:Repositório Institucional da UFMGinstname:Universidade Federal de Minas Gerais (UFMG)instacron:UFMGORIGINALphd-antonio.pdfphd-antonio.pdfapplication/pdf17978007https://repositorio.ufmg.br/bitstream/1843/33922/4/phd-antonio.pdf52153790069a12a8f3afdababae12694MD54CC-LICENSElicense_rdflicense_rdfapplication/rdf+xml; charset=utf-8811https://repositorio.ufmg.br/bitstream/1843/33922/5/license_rdfcfd6801dba008cb6adbd9838b81582abMD55LICENSElicense.txtlicense.txttext/plain; charset=utf-82119https://repositorio.ufmg.br/bitstream/1843/33922/6/license.txt34badce4be7e31e3adb4575ae96af679MD561843/339222020-08-07 16:21:43.137oai:repositorio.ufmg.br:1843/33922TElDRU7Dh0EgREUgRElTVFJJQlVJw4fDg08gTsODTy1FWENMVVNJVkEgRE8gUkVQT1NJVMOTUklPIElOU1RJVFVDSU9OQUwgREEgVUZNRwoKQ29tIGEgYXByZXNlbnRhw6fDo28gZGVzdGEgbGljZW7Dp2EsIHZvY8OqIChvIGF1dG9yIChlcykgb3UgbyB0aXR1bGFyIGRvcyBkaXJlaXRvcyBkZSBhdXRvcikgY29uY2VkZSBhbyBSZXBvc2l0w7NyaW8gSW5zdGl0dWNpb25hbCBkYSBVRk1HIChSSS1VRk1HKSBvIGRpcmVpdG8gbsOjbyBleGNsdXNpdm8gZSBpcnJldm9nw6F2ZWwgZGUgcmVwcm9kdXppciBlL291IGRpc3RyaWJ1aXIgYSBzdWEgcHVibGljYcOnw6NvIChpbmNsdWluZG8gbyByZXN1bW8pIHBvciB0b2RvIG8gbXVuZG8gbm8gZm9ybWF0byBpbXByZXNzbyBlIGVsZXRyw7RuaWNvIGUgZW0gcXVhbHF1ZXIgbWVpbywgaW5jbHVpbmRvIG9zIGZvcm1hdG9zIMOhdWRpbyBvdSB2w61kZW8uCgpWb2PDqiBkZWNsYXJhIHF1ZSBjb25oZWNlIGEgcG9sw610aWNhIGRlIGNvcHlyaWdodCBkYSBlZGl0b3JhIGRvIHNldSBkb2N1bWVudG8gZSBxdWUgY29uaGVjZSBlIGFjZWl0YSBhcyBEaXJldHJpemVzIGRvIFJJLVVGTUcuCgpWb2PDqiBjb25jb3JkYSBxdWUgbyBSZXBvc2l0w7NyaW8gSW5zdGl0dWNpb25hbCBkYSBVRk1HIHBvZGUsIHNlbSBhbHRlcmFyIG8gY29udGXDumRvLCB0cmFuc3BvciBhIHN1YSBwdWJsaWNhw6fDo28gcGFyYSBxdWFscXVlciBtZWlvIG91IGZvcm1hdG8gcGFyYSBmaW5zIGRlIHByZXNlcnZhw6fDo28uCgpWb2PDqiB0YW1iw6ltIGNvbmNvcmRhIHF1ZSBvIFJlcG9zaXTDs3JpbyBJbnN0aXR1Y2lvbmFsIGRhIFVGTUcgcG9kZSBtYW50ZXIgbWFpcyBkZSB1bWEgY8OzcGlhIGRlIHN1YSBwdWJsaWNhw6fDo28gcGFyYSBmaW5zIGRlIHNlZ3VyYW7Dp2EsIGJhY2stdXAgZSBwcmVzZXJ2YcOnw6NvLgoKVm9jw6ogZGVjbGFyYSBxdWUgYSBzdWEgcHVibGljYcOnw6NvIMOpIG9yaWdpbmFsIGUgcXVlIHZvY8OqIHRlbSBvIHBvZGVyIGRlIGNvbmNlZGVyIG9zIGRpcmVpdG9zIGNvbnRpZG9zIG5lc3RhIGxpY2Vuw6dhLiBWb2PDqiB0YW1iw6ltIGRlY2xhcmEgcXVlIG8gZGVww7NzaXRvIGRlIHN1YSBwdWJsaWNhw6fDo28gbsOjbywgcXVlIHNlamEgZGUgc2V1IGNvbmhlY2ltZW50bywgaW5mcmluZ2UgZGlyZWl0b3MgYXV0b3JhaXMgZGUgbmluZ3XDqW0uCgpDYXNvIGEgc3VhIHB1YmxpY2HDp8OjbyBjb250ZW5oYSBtYXRlcmlhbCBxdWUgdm9jw6ogbsOjbyBwb3NzdWkgYSB0aXR1bGFyaWRhZGUgZG9zIGRpcmVpdG9zIGF1dG9yYWlzLCB2b2PDqiBkZWNsYXJhIHF1ZSBvYnRldmUgYSBwZXJtaXNzw6NvIGlycmVzdHJpdGEgZG8gZGV0ZW50b3IgZG9zIGRpcmVpdG9zIGF1dG9yYWlzIHBhcmEgY29uY2VkZXIgYW8gUmVwb3NpdMOzcmlvIEluc3RpdHVjaW9uYWwgZGEgVUZNRyBvcyBkaXJlaXRvcyBhcHJlc2VudGFkb3MgbmVzdGEgbGljZW7Dp2EsIGUgcXVlIGVzc2UgbWF0ZXJpYWwgZGUgcHJvcHJpZWRhZGUgZGUgdGVyY2Vpcm9zIGVzdMOhIGNsYXJhbWVudGUgaWRlbnRpZmljYWRvIGUgcmVjb25oZWNpZG8gbm8gdGV4dG8gb3Ugbm8gY29udGXDumRvIGRhIHB1YmxpY2HDp8OjbyBvcmEgZGVwb3NpdGFkYS4KCkNBU08gQSBQVUJMSUNBw4fDg08gT1JBIERFUE9TSVRBREEgVEVOSEEgU0lETyBSRVNVTFRBRE8gREUgVU0gUEFUUk9Dw41OSU8gT1UgQVBPSU8gREUgVU1BIEFHw4pOQ0lBIERFIEZPTUVOVE8gT1UgT1VUUk8gT1JHQU5JU01PLCBWT0PDiiBERUNMQVJBIFFVRSBSRVNQRUlUT1UgVE9ET1MgRSBRVUFJU1FVRVIgRElSRUlUT1MgREUgUkVWSVPDg08gQ09NTyBUQU1Cw4lNIEFTIERFTUFJUyBPQlJJR0HDh8OVRVMgRVhJR0lEQVMgUE9SIENPTlRSQVRPIE9VIEFDT1JETy4KCk8gUmVwb3NpdMOzcmlvIEluc3RpdHVjaW9uYWwgZGEgVUZNRyBzZSBjb21wcm9tZXRlIGEgaWRlbnRpZmljYXIgY2xhcmFtZW50ZSBvIHNldSBub21lKHMpIG91IG8ocykgbm9tZXMocykgZG8ocykgZGV0ZW50b3IoZXMpIGRvcyBkaXJlaXRvcyBhdXRvcmFpcyBkYSBwdWJsaWNhw6fDo28sIGUgbsOjbyBmYXLDoSBxdWFscXVlciBhbHRlcmHDp8OjbywgYWzDqW0gZGFxdWVsYXMgY29uY2VkaWRhcyBwb3IgZXN0YSBsaWNlbsOnYS4KCg==Repositório de PublicaçõesPUBhttps://repositorio.ufmg.br/oaiopendoar:2020-08-07T19:21:43Repositório Institucional da UFMG - Universidade Federal de Minas Gerais (UFMG)false |
dc.title.pt_BR.fl_str_mv |
Learning nonlinear differentiable models for signals and systems: with applications |
dc.title.alternative.pt_BR.fl_str_mv |
Aprendendo modelos não-lineares diferenciáveis para sinais e sistemas: com aplicações |
title |
Learning nonlinear differentiable models for signals and systems: with applications |
spellingShingle |
Learning nonlinear differentiable models for signals and systems: with applications Antônio Horta Ribeiro Nonlinear systems Differentiable models Deep learning System identification Machine learning Engenharia elétrica Aprendizado do computador Aprendizado profundo Identificação de sistemas Sistemas não lineares |
title_short |
Learning nonlinear differentiable models for signals and systems: with applications |
title_full |
Learning nonlinear differentiable models for signals and systems: with applications |
title_fullStr |
Learning nonlinear differentiable models for signals and systems: with applications |
title_full_unstemmed |
Learning nonlinear differentiable models for signals and systems: with applications |
title_sort |
Learning nonlinear differentiable models for signals and systems: with applications |
author |
Antônio Horta Ribeiro |
author_facet |
Antônio Horta Ribeiro |
author_role |
author |
dc.contributor.advisor1.fl_str_mv |
Luis Antonio Aguirre |
dc.contributor.advisor1Lattes.fl_str_mv |
http://lattes.cnpq.br/6682146998710900 |
dc.contributor.advisor-co1.fl_str_mv |
Thomas B. Schon |
dc.contributor.referee1.fl_str_mv |
Eduardo Mazoni Andrade Marçal Mendes |
dc.contributor.referee2.fl_str_mv |
Frederico Gadelha Guimarães |
dc.contributor.referee3.fl_str_mv |
Guilherme de Alencar Barreto |
dc.contributor.referee4.fl_str_mv |
Leandro dos Santos Coelho |
dc.contributor.referee5.fl_str_mv |
Maarten Schoukens |
dc.contributor.authorLattes.fl_str_mv |
http://lattes.cnpq.br/0898576944135254 |
dc.contributor.author.fl_str_mv |
Antônio Horta Ribeiro |
contributor_str_mv |
Luis Antonio Aguirre Thomas B. Schon Eduardo Mazoni Andrade Marçal Mendes Frederico Gadelha Guimarães Guilherme de Alencar Barreto Leandro dos Santos Coelho Maarten Schoukens |
dc.subject.por.fl_str_mv |
Nonlinear systems Differentiable models Deep learning System identification Machine learning |
topic |
Nonlinear systems Differentiable models Deep learning System identification Machine learning Engenharia elétrica Aprendizado do computador Aprendizado profundo Identificação de sistemas Sistemas não lineares |
dc.subject.other.pt_BR.fl_str_mv |
Engenharia elétrica Aprendizado do computador Aprendizado profundo Identificação de sistemas Sistemas não lineares |
description |
Building empirical models from data is of fundamental importance in engineering, and pushing the boundaries of current engineering technology requires us to model and understand nonlinear systems. In this thesis, nonlinear differentiable models and its applications are studied. This class of models has gained traction in machine learning tasks with the introduction of deep learning. Indeed, deep models of stacked differentiable components have recently achieved super-human performance on several tasks, including computer games, image classification, and medical diagnosis. The application of nonlinear differentiable models is studied for modeling signals and systems both for engineering and machine learning applications. One central question is the role of recurrence and the pros and cons of recurrent and feedforward models. The question is approached from more than one angle: 1) by studying the effect of recurrence in neural networks in terms of robustness to noise, computational cost, and convergence; 2) by analyzing the smoothness of the cost function in nonlinear system identification problems and its relation to the model internal dynamics – and proposing the use of a technique called multiple shooting for improving the cost-function smoothness; and, 3) by investigating the interplay between the internal dynamics, the attractors and the expressiveness of the model in deep recurrent neural networks. The more applied part of the thesis consists of the use of deep neural networks to solve complex tasks and to model nonlinear behavior from real data. Data from the Telehealth Center of Minas Gerais is used to train a deep neural network capable of identifying abnormalities in the electrocardiogram exam with performance superior to the medical residents in the studied scenario. Also, a deep neural network is used for modeling an electronic oscillator and an F-16 aircraft using data from ground vibration experiments, obtaining competitive results in both cases. |
publishDate |
2020 |
dc.date.accessioned.fl_str_mv |
2020-08-07T19:21:43Z |
dc.date.available.fl_str_mv |
2020-08-07T19:21:43Z |
dc.date.issued.fl_str_mv |
2020-03-03 |
dc.type.status.fl_str_mv |
info:eu-repo/semantics/publishedVersion |
dc.type.driver.fl_str_mv |
info:eu-repo/semantics/doctoralThesis |
format |
doctoralThesis |
status_str |
publishedVersion |
dc.identifier.uri.fl_str_mv |
http://hdl.handle.net/1843/33922 |
dc.identifier.orcid.pt_BR.fl_str_mv |
0000-0003-3632-8529 |
url |
http://hdl.handle.net/1843/33922 |
identifier_str_mv |
0000-0003-3632-8529 |
dc.language.iso.fl_str_mv |
eng |
language |
eng |
dc.rights.driver.fl_str_mv |
http://creativecommons.org/licenses/by-nc-nd/3.0/pt/ info:eu-repo/semantics/openAccess |
rights_invalid_str_mv |
http://creativecommons.org/licenses/by-nc-nd/3.0/pt/ |
eu_rights_str_mv |
openAccess |
dc.publisher.none.fl_str_mv |
Universidade Federal de Minas Gerais |
dc.publisher.program.fl_str_mv |
Programa de Pós-Graduação em Engenharia Elétrica |
dc.publisher.initials.fl_str_mv |
UFMG |
dc.publisher.country.fl_str_mv |
Brasil |
dc.publisher.department.fl_str_mv |
ENG - DEPARTAMENTO DE ENGENHARIA ELÉTRICA ENG - DEPARTAMENTO DE ENGENHARIA ELETRÔNICA |
publisher.none.fl_str_mv |
Universidade Federal de Minas Gerais |
dc.source.none.fl_str_mv |
reponame:Repositório Institucional da UFMG instname:Universidade Federal de Minas Gerais (UFMG) instacron:UFMG |
instname_str |
Universidade Federal de Minas Gerais (UFMG) |
instacron_str |
UFMG |
institution |
UFMG |
reponame_str |
Repositório Institucional da UFMG |
collection |
Repositório Institucional da UFMG |
bitstream.url.fl_str_mv |
https://repositorio.ufmg.br/bitstream/1843/33922/4/phd-antonio.pdf https://repositorio.ufmg.br/bitstream/1843/33922/5/license_rdf https://repositorio.ufmg.br/bitstream/1843/33922/6/license.txt |
bitstream.checksum.fl_str_mv |
52153790069a12a8f3afdababae12694 cfd6801dba008cb6adbd9838b81582ab 34badce4be7e31e3adb4575ae96af679 |
bitstream.checksumAlgorithm.fl_str_mv |
MD5 MD5 MD5 |
repository.name.fl_str_mv |
Repositório Institucional da UFMG - Universidade Federal de Minas Gerais (UFMG) |
repository.mail.fl_str_mv |
|
_version_ |
1803589283518349312 |