Transferring knowledge as heuristics in reinforcement learning: A case-based approach
Autor(a) principal: | |
---|---|
Data de Publicação: | 2015 |
Outros Autores: | , , , |
Tipo de documento: | Artigo |
Título da fonte: | Biblioteca Digital de Teses e Dissertações da FEI |
Texto Completo: | https://repositorio.fei.edu.br/handle/FEI/1217 |
Resumo: | © 2015 Elsevier B.V.Abstract The goal of this paper is to propose and analyse a transfer learning meta-algorithm that allows the implementation of distinct methods using heuristics to accelerate a Reinforcement Learning procedure in one domain (the target) that are obtained from another (simpler) domain (the source domain). This meta-algorithm works in three stages: first, it uses a Reinforcement Learning step to learn a task on the source domain, storing the knowledge thus obtained in a case base; second, it does an unsupervised mapping of the source-domain actions to the target-domain actions; and, third, the case base obtained in the first stage is used as heuristics to speed up the learning process in the target domain. A set of empirical evaluations were conducted in two target domains: the 3D mountain car (using a learned case base from a 2D simulation) and stability learning for a humanoid robot in the Robocup 3D Soccer Simulator (that uses knowledge learned from the Acrobot domain). The results attest that our transfer learning algorithm outperforms recent heuristically-accelerated reinforcement learning and transfer learning algorithms. |
id |
FEI_98e0b9a80975e583aaea6708f7ce37b7 |
---|---|
oai_identifier_str |
oai:repositorio.fei.edu.br:FEI/1217 |
network_acronym_str |
FEI |
network_name_str |
Biblioteca Digital de Teses e Dissertações da FEI |
repository_id_str |
|
spelling |
Bianchi R.A.C.Celiberto L.A.Santos P.E.Matsuura J.P.Lopez De Mantaras R.2019-08-19T23:45:20Z2019-08-19T23:45:20Z2015BIANCHI, REINALDO A.C.; JUNIOR, LUIZ A. CELIBERTO; Santos, Paulo E.; MATSUURA, JACKSON P.; LÓPEZ DE MÀNTARAS, RAMÓN. Transferring knowledge as heuristics in reinforcement learning: a case-based approach. Artificial Intelligence (General Ed.), v. 226, p. 102-121, 2015.0004-3702https://repositorio.fei.edu.br/handle/FEI/121710.1016/j.artint.2015.05.008© 2015 Elsevier B.V.Abstract The goal of this paper is to propose and analyse a transfer learning meta-algorithm that allows the implementation of distinct methods using heuristics to accelerate a Reinforcement Learning procedure in one domain (the target) that are obtained from another (simpler) domain (the source domain). This meta-algorithm works in three stages: first, it uses a Reinforcement Learning step to learn a task on the source domain, storing the knowledge thus obtained in a case base; second, it does an unsupervised mapping of the source-domain actions to the target-domain actions; and, third, the case base obtained in the first stage is used as heuristics to speed up the learning process in the target domain. A set of empirical evaluations were conducted in two target domains: the 3D mountain car (using a learned case base from a 2D simulation) and stability learning for a humanoid robot in the Robocup 3D Soccer Simulator (that uses knowledge learned from the Acrobot domain). The results attest that our transfer learning algorithm outperforms recent heuristically-accelerated reinforcement learning and transfer learning algorithms.226102121Artificial IntelligenceTransferring knowledge as heuristics in reinforcement learning: A case-based approachinfo:eu-repo/semantics/publishedVersioninfo:eu-repo/semantics/articleCase-based reasoningReinforcement learningTransfer learninginfo:eu-repo/semantics/openAccessreponame:Biblioteca Digital de Teses e Dissertações da FEIinstname:Centro Universitário da Fundação Educacional Inaciana (FEI)instacron:FEI492-s2.0-849309602332D simulationsCase-based approachEmpirical evaluationsHumanoid robotLearning processMeta-algorithmsTarget domainTransfer learninghttps://www.scopus.com/inward/record.uri?partnerID=HzOxMe3b&scp=84930960233&origin=inward2022-06-01FEI/12172022-06-01 03:07:35.802Biblioteca Digital de Teses e Dissertaçõeshttp://sofia.fei.edu.br/pergamum/biblioteca/PRI |
dc.title.none.fl_str_mv |
Transferring knowledge as heuristics in reinforcement learning: A case-based approach |
title |
Transferring knowledge as heuristics in reinforcement learning: A case-based approach |
spellingShingle |
Transferring knowledge as heuristics in reinforcement learning: A case-based approach Bianchi R.A.C. Case-based reasoning Reinforcement learning Transfer learning |
title_short |
Transferring knowledge as heuristics in reinforcement learning: A case-based approach |
title_full |
Transferring knowledge as heuristics in reinforcement learning: A case-based approach |
title_fullStr |
Transferring knowledge as heuristics in reinforcement learning: A case-based approach |
title_full_unstemmed |
Transferring knowledge as heuristics in reinforcement learning: A case-based approach |
title_sort |
Transferring knowledge as heuristics in reinforcement learning: A case-based approach |
author |
Bianchi R.A.C. |
author_facet |
Bianchi R.A.C. Celiberto L.A. Santos P.E. Matsuura J.P. Lopez De Mantaras R. |
author_role |
author |
author2 |
Celiberto L.A. Santos P.E. Matsuura J.P. Lopez De Mantaras R. |
author2_role |
author author author author |
dc.contributor.author.fl_str_mv |
Bianchi R.A.C. Celiberto L.A. Santos P.E. Matsuura J.P. Lopez De Mantaras R. |
dc.subject.eng.fl_str_mv |
Case-based reasoning Reinforcement learning Transfer learning |
topic |
Case-based reasoning Reinforcement learning Transfer learning |
description |
© 2015 Elsevier B.V.Abstract The goal of this paper is to propose and analyse a transfer learning meta-algorithm that allows the implementation of distinct methods using heuristics to accelerate a Reinforcement Learning procedure in one domain (the target) that are obtained from another (simpler) domain (the source domain). This meta-algorithm works in three stages: first, it uses a Reinforcement Learning step to learn a task on the source domain, storing the knowledge thus obtained in a case base; second, it does an unsupervised mapping of the source-domain actions to the target-domain actions; and, third, the case base obtained in the first stage is used as heuristics to speed up the learning process in the target domain. A set of empirical evaluations were conducted in two target domains: the 3D mountain car (using a learned case base from a 2D simulation) and stability learning for a humanoid robot in the Robocup 3D Soccer Simulator (that uses knowledge learned from the Acrobot domain). The results attest that our transfer learning algorithm outperforms recent heuristically-accelerated reinforcement learning and transfer learning algorithms. |
publishDate |
2015 |
dc.date.issued.fl_str_mv |
2015 |
dc.date.accessioned.fl_str_mv |
2019-08-19T23:45:20Z |
dc.date.available.fl_str_mv |
2019-08-19T23:45:20Z |
dc.type.status.fl_str_mv |
info:eu-repo/semantics/publishedVersion |
dc.type.driver.fl_str_mv |
info:eu-repo/semantics/article |
format |
article |
status_str |
publishedVersion |
dc.identifier.citation.fl_str_mv |
BIANCHI, REINALDO A.C.; JUNIOR, LUIZ A. CELIBERTO; Santos, Paulo E.; MATSUURA, JACKSON P.; LÓPEZ DE MÀNTARAS, RAMÓN. Transferring knowledge as heuristics in reinforcement learning: a case-based approach. Artificial Intelligence (General Ed.), v. 226, p. 102-121, 2015. |
dc.identifier.uri.fl_str_mv |
https://repositorio.fei.edu.br/handle/FEI/1217 |
dc.identifier.issn.none.fl_str_mv |
0004-3702 |
dc.identifier.doi.none.fl_str_mv |
10.1016/j.artint.2015.05.008 |
identifier_str_mv |
BIANCHI, REINALDO A.C.; JUNIOR, LUIZ A. CELIBERTO; Santos, Paulo E.; MATSUURA, JACKSON P.; LÓPEZ DE MÀNTARAS, RAMÓN. Transferring knowledge as heuristics in reinforcement learning: a case-based approach. Artificial Intelligence (General Ed.), v. 226, p. 102-121, 2015. 0004-3702 10.1016/j.artint.2015.05.008 |
url |
https://repositorio.fei.edu.br/handle/FEI/1217 |
dc.relation.ispartof.none.fl_str_mv |
Artificial Intelligence |
dc.rights.driver.fl_str_mv |
info:eu-repo/semantics/openAccess |
eu_rights_str_mv |
openAccess |
dc.source.none.fl_str_mv |
reponame:Biblioteca Digital de Teses e Dissertações da FEI instname:Centro Universitário da Fundação Educacional Inaciana (FEI) instacron:FEI |
instname_str |
Centro Universitário da Fundação Educacional Inaciana (FEI) |
instacron_str |
FEI |
institution |
FEI |
reponame_str |
Biblioteca Digital de Teses e Dissertações da FEI |
collection |
Biblioteca Digital de Teses e Dissertações da FEI |
repository.name.fl_str_mv |
|
repository.mail.fl_str_mv |
|
_version_ |
1734750992840261632 |