Fitness landscape analysis of convolutional neural network architectures for image classification

Detalhes bibliográficos
Autor(a) principal: Rodrigues, Nuno M.
Data de Publicação: 2022
Outros Autores: Malan, Katherine M., Ochoa, Gabriela, Vanneschi, Leonardo, Silva, Sara
Tipo de documento: Artigo
Idioma: eng
Título da fonte: Repositório Científico de Acesso Aberto de Portugal (Repositórios Cientìficos)
Texto Completo: http://hdl.handle.net/10362/142593
Resumo: Rodrigues, N. M., Malan, K. M., Ochoa, G., Vanneschi, L., & Silva, S. (2022). Fitness landscape analysis of convolutional neural network architectures for image classification. Information Sciences, 609(September), 711-726. https://doi.org/10.1016/j.ins.2022.07.040. ----- Funding: The work of K.M. Malan was supported by the National Research Foundation, South Africa, under Grant 120837. This work was partially supported by FCT, Portugal, through funding of the LASIGE Research Unit (UIDB/00408/2020 and UIDP/00408/2020); projects GADgET (DSAIPA/ DS/ 0022/ 2018), BINDER (PTDC/ CCI-INF/ 29168/ 2017), AICE (DSAIPA/ DS/ 0113/ 2019); Nuno Rodrigues was supported by PhD Grant 2021/05322/BD.
id RCAP_4e1ded10f950c55d7541070ce5935c51
oai_identifier_str oai:run.unl.pt:10362/142593
network_acronym_str RCAP
network_name_str Repositório Científico de Acesso Aberto de Portugal (Repositórios Cientìficos)
repository_id_str 7160
spelling Fitness landscape analysis of convolutional neural network architectures for image classificationNeural architecture searchConvolutional neural networksFitness distance correlationLocal optima networksLoss landscapesTheoretical Computer ScienceSoftwareControl and Systems EngineeringComputer Science ApplicationsInformation Systems and ManagementArtificial IntelligenceRodrigues, N. M., Malan, K. M., Ochoa, G., Vanneschi, L., & Silva, S. (2022). Fitness landscape analysis of convolutional neural network architectures for image classification. Information Sciences, 609(September), 711-726. https://doi.org/10.1016/j.ins.2022.07.040. ----- Funding: The work of K.M. Malan was supported by the National Research Foundation, South Africa, under Grant 120837. This work was partially supported by FCT, Portugal, through funding of the LASIGE Research Unit (UIDB/00408/2020 and UIDP/00408/2020); projects GADgET (DSAIPA/ DS/ 0022/ 2018), BINDER (PTDC/ CCI-INF/ 29168/ 2017), AICE (DSAIPA/ DS/ 0113/ 2019); Nuno Rodrigues was supported by PhD Grant 2021/05322/BD.The global structure of the hyperparameter spaces of neural networks is not well understood and it is therefore not clear which hyperparameter search algorithm will be most effective. In this paper we analyze the landscapes of convolutional neural network architecture search spaces to provide insight into appropriate search algorithms for these spaces. Using a classical fitness landscape analysis approach (fitness distance correlation) and a more recent tool (local optima networks) we study the global structure of these spaces. Our analysis on six image classification datasets reveals that the landscapes are multi-modal, but with relatively few local optima from which it is not hard to escape with a simple perturbation operator. This led us to explore the performance of iterated local search, which we found to more effectively search the training landscapes than three evolutionary algorithm variants. Evolutionary algorithms, however, outperformed iterated local search in terms of generalization on problems with larger discrepancies between the training and testing landscapes.NOVA Information Management School (NOVA IMS)Information Management Research Center (MagIC) - NOVA Information Management SchoolRUNRodrigues, Nuno M.Malan, Katherine M.Ochoa, GabrielaVanneschi, LeonardoSilva, Sara2022-07-28T22:24:00Z2022-09-012022-09-01T00:00:00Zinfo:eu-repo/semantics/publishedVersioninfo:eu-repo/semantics/article16application/pdfhttp://hdl.handle.net/10362/142593eng0020-0255PURE: 45396324https://doi.org/10.1016/j.ins.2022.07.040info:eu-repo/semantics/openAccessreponame:Repositório Científico de Acesso Aberto de Portugal (Repositórios Cientìficos)instname:Agência para a Sociedade do Conhecimento (UMIC) - FCT - Sociedade da Informaçãoinstacron:RCAAP2024-03-11T05:20:22Zoai:run.unl.pt:10362/142593Portal AgregadorONGhttps://www.rcaap.pt/oai/openaireopendoar:71602024-03-20T03:50:24.577974Repositório Científico de Acesso Aberto de Portugal (Repositórios Cientìficos) - Agência para a Sociedade do Conhecimento (UMIC) - FCT - Sociedade da Informaçãofalse
dc.title.none.fl_str_mv Fitness landscape analysis of convolutional neural network architectures for image classification
title Fitness landscape analysis of convolutional neural network architectures for image classification
spellingShingle Fitness landscape analysis of convolutional neural network architectures for image classification
Rodrigues, Nuno M.
Neural architecture search
Convolutional neural networks
Fitness distance correlation
Local optima networks
Loss landscapes
Theoretical Computer Science
Software
Control and Systems Engineering
Computer Science Applications
Information Systems and Management
Artificial Intelligence
title_short Fitness landscape analysis of convolutional neural network architectures for image classification
title_full Fitness landscape analysis of convolutional neural network architectures for image classification
title_fullStr Fitness landscape analysis of convolutional neural network architectures for image classification
title_full_unstemmed Fitness landscape analysis of convolutional neural network architectures for image classification
title_sort Fitness landscape analysis of convolutional neural network architectures for image classification
author Rodrigues, Nuno M.
author_facet Rodrigues, Nuno M.
Malan, Katherine M.
Ochoa, Gabriela
Vanneschi, Leonardo
Silva, Sara
author_role author
author2 Malan, Katherine M.
Ochoa, Gabriela
Vanneschi, Leonardo
Silva, Sara
author2_role author
author
author
author
dc.contributor.none.fl_str_mv NOVA Information Management School (NOVA IMS)
Information Management Research Center (MagIC) - NOVA Information Management School
RUN
dc.contributor.author.fl_str_mv Rodrigues, Nuno M.
Malan, Katherine M.
Ochoa, Gabriela
Vanneschi, Leonardo
Silva, Sara
dc.subject.por.fl_str_mv Neural architecture search
Convolutional neural networks
Fitness distance correlation
Local optima networks
Loss landscapes
Theoretical Computer Science
Software
Control and Systems Engineering
Computer Science Applications
Information Systems and Management
Artificial Intelligence
topic Neural architecture search
Convolutional neural networks
Fitness distance correlation
Local optima networks
Loss landscapes
Theoretical Computer Science
Software
Control and Systems Engineering
Computer Science Applications
Information Systems and Management
Artificial Intelligence
description Rodrigues, N. M., Malan, K. M., Ochoa, G., Vanneschi, L., & Silva, S. (2022). Fitness landscape analysis of convolutional neural network architectures for image classification. Information Sciences, 609(September), 711-726. https://doi.org/10.1016/j.ins.2022.07.040. ----- Funding: The work of K.M. Malan was supported by the National Research Foundation, South Africa, under Grant 120837. This work was partially supported by FCT, Portugal, through funding of the LASIGE Research Unit (UIDB/00408/2020 and UIDP/00408/2020); projects GADgET (DSAIPA/ DS/ 0022/ 2018), BINDER (PTDC/ CCI-INF/ 29168/ 2017), AICE (DSAIPA/ DS/ 0113/ 2019); Nuno Rodrigues was supported by PhD Grant 2021/05322/BD.
publishDate 2022
dc.date.none.fl_str_mv 2022-07-28T22:24:00Z
2022-09-01
2022-09-01T00:00:00Z
dc.type.status.fl_str_mv info:eu-repo/semantics/publishedVersion
dc.type.driver.fl_str_mv info:eu-repo/semantics/article
format article
status_str publishedVersion
dc.identifier.uri.fl_str_mv http://hdl.handle.net/10362/142593
url http://hdl.handle.net/10362/142593
dc.language.iso.fl_str_mv eng
language eng
dc.relation.none.fl_str_mv 0020-0255
PURE: 45396324
https://doi.org/10.1016/j.ins.2022.07.040
dc.rights.driver.fl_str_mv info:eu-repo/semantics/openAccess
eu_rights_str_mv openAccess
dc.format.none.fl_str_mv 16
application/pdf
dc.source.none.fl_str_mv reponame:Repositório Científico de Acesso Aberto de Portugal (Repositórios Cientìficos)
instname:Agência para a Sociedade do Conhecimento (UMIC) - FCT - Sociedade da Informação
instacron:RCAAP
instname_str Agência para a Sociedade do Conhecimento (UMIC) - FCT - Sociedade da Informação
instacron_str RCAAP
institution RCAAP
reponame_str Repositório Científico de Acesso Aberto de Portugal (Repositórios Cientìficos)
collection Repositório Científico de Acesso Aberto de Portugal (Repositórios Cientìficos)
repository.name.fl_str_mv Repositório Científico de Acesso Aberto de Portugal (Repositórios Cientìficos) - Agência para a Sociedade do Conhecimento (UMIC) - FCT - Sociedade da Informação
repository.mail.fl_str_mv
_version_ 1799138100737212416

Registros relacionados