Smoothing and worst-case complexity for direct-search methods in nonsmooth optimization
Autor(a) principal: | |
---|---|
Data de Publicação: | 2013 |
Outros Autores: | |
Tipo de documento: | Artigo |
Idioma: | eng |
Título da fonte: | Repositório Científico de Acesso Aberto de Portugal (Repositórios Cientìficos) |
Texto Completo: | http://hdl.handle.net/10316/45705 https://doi.org/10.1093/imanum/drs027 |
Resumo: | In the context of the derivative-free optimization of a smooth objective function, it has been shown that the worst-case complexity of direct-search methods is of the same order as that of the steepest descent for derivative-based optimization; more precisely, the number of iterations needed to reduce the norm of the gradient of the objective function below a certain threshold is proportional to the inverse of the threshold squared. Motivated by the lack of such a result in the nonsmooth case, we propose, analyse, and test a class of smoothing direct-search methods for the unconstrained optimization of nonsmooth functions. Given a parameterized family of smoothing functions for the nonsmooth objective function dependent on a smoothing parameter, this class of methods consists of applying a direct-search algorithm for a fixed value of the smoothing parameter until the step size is relatively small, after which the smoothing parameter is reduced and the process is repeated. One can show that the worst-case complexity (or cost) of this procedure is roughly one order of magnitude worse than the one for direct search or steepest descent on smooth functions. The class of smoothing direct-search methods is also shown to enjoy asymptotic global convergence properties. Some preliminary numerical experiments indicate that this approach leads to better values of the objective function, in some cases pushing the optimization further, apparently without an additional cost in the number of function evaluations. |
id |
RCAP_22b6543855e33a95cb7dab2a471f0a5e |
---|---|
oai_identifier_str |
oai:estudogeral.uc.pt:10316/45705 |
network_acronym_str |
RCAP |
network_name_str |
Repositório Científico de Acesso Aberto de Portugal (Repositórios Cientìficos) |
repository_id_str |
7160 |
spelling |
Smoothing and worst-case complexity for direct-search methods in nonsmooth optimizationIn the context of the derivative-free optimization of a smooth objective function, it has been shown that the worst-case complexity of direct-search methods is of the same order as that of the steepest descent for derivative-based optimization; more precisely, the number of iterations needed to reduce the norm of the gradient of the objective function below a certain threshold is proportional to the inverse of the threshold squared. Motivated by the lack of such a result in the nonsmooth case, we propose, analyse, and test a class of smoothing direct-search methods for the unconstrained optimization of nonsmooth functions. Given a parameterized family of smoothing functions for the nonsmooth objective function dependent on a smoothing parameter, this class of methods consists of applying a direct-search algorithm for a fixed value of the smoothing parameter until the step size is relatively small, after which the smoothing parameter is reduced and the process is repeated. One can show that the worst-case complexity (or cost) of this procedure is roughly one order of magnitude worse than the one for direct search or steepest descent on smooth functions. The class of smoothing direct-search methods is also shown to enjoy asymptotic global convergence properties. Some preliminary numerical experiments indicate that this approach leads to better values of the objective function, in some cases pushing the optimization further, apparently without an additional cost in the number of function evaluations.Oxford University Press2013info:eu-repo/semantics/publishedVersioninfo:eu-repo/semantics/articlehttp://hdl.handle.net/10316/45705http://hdl.handle.net/10316/45705https://doi.org/10.1093/imanum/drs027enghttps://doi.org/10.1093/imanum/drs027Garmanjani, RohollahVicente, Luís Nunesinfo:eu-repo/semantics/openAccessreponame:Repositório Científico de Acesso Aberto de Portugal (Repositórios Cientìficos)instname:Agência para a Sociedade do Conhecimento (UMIC) - FCT - Sociedade da Informaçãoinstacron:RCAAP2020-05-25T12:11:58Zoai:estudogeral.uc.pt:10316/45705Portal AgregadorONGhttps://www.rcaap.pt/oai/openaireopendoar:71602024-03-19T20:53:25.720617Repositório Científico de Acesso Aberto de Portugal (Repositórios Cientìficos) - Agência para a Sociedade do Conhecimento (UMIC) - FCT - Sociedade da Informaçãofalse |
dc.title.none.fl_str_mv |
Smoothing and worst-case complexity for direct-search methods in nonsmooth optimization |
title |
Smoothing and worst-case complexity for direct-search methods in nonsmooth optimization |
spellingShingle |
Smoothing and worst-case complexity for direct-search methods in nonsmooth optimization Garmanjani, Rohollah |
title_short |
Smoothing and worst-case complexity for direct-search methods in nonsmooth optimization |
title_full |
Smoothing and worst-case complexity for direct-search methods in nonsmooth optimization |
title_fullStr |
Smoothing and worst-case complexity for direct-search methods in nonsmooth optimization |
title_full_unstemmed |
Smoothing and worst-case complexity for direct-search methods in nonsmooth optimization |
title_sort |
Smoothing and worst-case complexity for direct-search methods in nonsmooth optimization |
author |
Garmanjani, Rohollah |
author_facet |
Garmanjani, Rohollah Vicente, Luís Nunes |
author_role |
author |
author2 |
Vicente, Luís Nunes |
author2_role |
author |
dc.contributor.author.fl_str_mv |
Garmanjani, Rohollah Vicente, Luís Nunes |
description |
In the context of the derivative-free optimization of a smooth objective function, it has been shown that the worst-case complexity of direct-search methods is of the same order as that of the steepest descent for derivative-based optimization; more precisely, the number of iterations needed to reduce the norm of the gradient of the objective function below a certain threshold is proportional to the inverse of the threshold squared. Motivated by the lack of such a result in the nonsmooth case, we propose, analyse, and test a class of smoothing direct-search methods for the unconstrained optimization of nonsmooth functions. Given a parameterized family of smoothing functions for the nonsmooth objective function dependent on a smoothing parameter, this class of methods consists of applying a direct-search algorithm for a fixed value of the smoothing parameter until the step size is relatively small, after which the smoothing parameter is reduced and the process is repeated. One can show that the worst-case complexity (or cost) of this procedure is roughly one order of magnitude worse than the one for direct search or steepest descent on smooth functions. The class of smoothing direct-search methods is also shown to enjoy asymptotic global convergence properties. Some preliminary numerical experiments indicate that this approach leads to better values of the objective function, in some cases pushing the optimization further, apparently without an additional cost in the number of function evaluations. |
publishDate |
2013 |
dc.date.none.fl_str_mv |
2013 |
dc.type.status.fl_str_mv |
info:eu-repo/semantics/publishedVersion |
dc.type.driver.fl_str_mv |
info:eu-repo/semantics/article |
format |
article |
status_str |
publishedVersion |
dc.identifier.uri.fl_str_mv |
http://hdl.handle.net/10316/45705 http://hdl.handle.net/10316/45705 https://doi.org/10.1093/imanum/drs027 |
url |
http://hdl.handle.net/10316/45705 https://doi.org/10.1093/imanum/drs027 |
dc.language.iso.fl_str_mv |
eng |
language |
eng |
dc.relation.none.fl_str_mv |
https://doi.org/10.1093/imanum/drs027 |
dc.rights.driver.fl_str_mv |
info:eu-repo/semantics/openAccess |
eu_rights_str_mv |
openAccess |
dc.publisher.none.fl_str_mv |
Oxford University Press |
publisher.none.fl_str_mv |
Oxford University Press |
dc.source.none.fl_str_mv |
reponame:Repositório Científico de Acesso Aberto de Portugal (Repositórios Cientìficos) instname:Agência para a Sociedade do Conhecimento (UMIC) - FCT - Sociedade da Informação instacron:RCAAP |
instname_str |
Agência para a Sociedade do Conhecimento (UMIC) - FCT - Sociedade da Informação |
instacron_str |
RCAAP |
institution |
RCAAP |
reponame_str |
Repositório Científico de Acesso Aberto de Portugal (Repositórios Cientìficos) |
collection |
Repositório Científico de Acesso Aberto de Portugal (Repositórios Cientìficos) |
repository.name.fl_str_mv |
Repositório Científico de Acesso Aberto de Portugal (Repositórios Cientìficos) - Agência para a Sociedade do Conhecimento (UMIC) - FCT - Sociedade da Informação |
repository.mail.fl_str_mv |
|
_version_ |
1799133821160914944 |