Managing Workflows on top of a Cloud Computing Orchestrator for using heterogeneous environments on e-Science

Detalhes bibliográficos
Autor(a) principal: Carrión, Abel
Data de Publicação: 2018
Outros Autores: Caballer, Miguel, Blanquer, Ignacio, Kotowski, Nelson, Jardim, Rodrigo, Dàvila, Alberto Martin Rivera
Tipo de documento: Artigo
Idioma: eng
Título da fonte: Repositório Institucional da FIOCRUZ (ARCA)
Texto Completo: https://www.arca.fiocruz.br/handle/icict/26682
Resumo: Versão preprint do autor: O autor pode arquivar a versão preprint (i.e. antes do peer-review)
id CRUZ_c66400b6fc99aafea7cb22d04145b07f
oai_identifier_str oai:www.arca.fiocruz.br:icict/26682
network_acronym_str CRUZ
network_name_str Repositório Institucional da FIOCRUZ (ARCA)
repository_id_str 2135
spelling Carrión, AbelCaballer, MiguelBlanquer, IgnacioKotowski, NelsonJardim, RodrigoDàvila, Alberto Martin Rivera2018-05-30T12:41:30Z2018-05-30T12:41:30Z2018CARRIÓN, Abel et al. Managing Workflows on top of a Cloud Computing Orchestrator for using heterogeneous environments on e-Science. International Journal of Web and Grid Services, v. 13, n. 4, p. 1-38, Mar. 2018.https://www.arca.fiocruz.br/handle/icict/26682Versão preprint do autor: O autor pode arquivar a versão preprint (i.e. antes do peer-review)Universitat Politécnica de Valéncia. Instituto de Instrumentación para Imagen Molecular. Centro Mixto CSIC. Valencia, Spain.Universitat Politécnica de Valéncia. Instituto de Instrumentación para Imagen Molecular. Centro Mixto CSIC. Valencia, Spain.Universitat Politécnica de Valéncia. Instituto de Instrumentación para Imagen Molecular. Centro Mixto CSIC. Valencia, Spain.Fundação Oswaldo Cruz. Instituto Oswaldo Cruz. Laboratório de Biologia Computacional e de Sistemas. Rio de Janeiro, RJ, Brasil.Fundação Oswaldo Cruz. Instituto Oswaldo Cruz. Laboratório de Biologia Computacional e de Sistemas. Rio de Janeiro, RJ, Brasil. .Fundação Oswaldo Cruz. Instituto Oswaldo Cruz. Laboratório de Biologia Computacional e de Sistemas. Rio de Janeiro, RJ, Brasil.Scientific Workflows (SWFs) are widely used to model processes in e-Science. SWFs are executed by means of Workflow Management Systems (WMSs), which orchestrate the workload on top of computing infrastructures. The advent of cloud computing infrastructures has opened the door of using ondemand infrastructures to complement or even replace local infrastructures. However, new issues have arisen, such as the integration of hybrid resources or the compromise between infrastructure reutilization and elasticity. In this article we present an ad-hoc solution for managing workflows exploiting the capabilities of cloud orchestrators to deploy resources on demand according to the workload and to combine heterogeneous cloud providers (such as on-premise clouds and public clouds) and traditional infrastructures (clusters) to minimize costs and response time. The work does not propose yet another WMS, but demonstrates the benefits of the integration of cloud orchestration when running complex workflows. The article shows several configuration experiments from a realistic comparative genomics workflow called Orthosearch, to migrate memory-intensive workload to public infrastructures while keeping other blocks of the experiment running locally. The article computes running time and cost suggesting best practices.engElsevierManaging Workflows on top of a Cloud Computing Orchestrator for using heterogeneous environments on e-Scienceinfo:eu-repo/semantics/publishedVersioninfo:eu-repo/semantics/articleWorkflowWorkflow Management SystemsCloud OrchestratorMulti-plataforme-ScienceCloud ComputingComparative genomicsNãoNão publicadoinfo:eu-repo/semantics/openAccessreponame:Repositório Institucional da FIOCRUZ (ARCA)instname:Fundação Oswaldo Cruz (FIOCRUZ)instacron:FIOCRUZLICENSElicense.txtlicense.txttext/plain; charset=utf-82991https://www.arca.fiocruz.br/bitstream/icict/26682/1/license.txt5a560609d32a3863062d77ff32785d58MD51ORIGINALnelson_kotowski_etal_IOC_2017.pdfnelson_kotowski_etal_IOC_2017.pdfapplication/pdf2177013https://www.arca.fiocruz.br/bitstream/icict/26682/2/nelson_kotowski_etal_IOC_2017.pdf27b8179b23c0f97566e075d8aa6939fbMD52TEXTnelson_kotowski_etal_IOC_2017.pdf.txtnelson_kotowski_etal_IOC_2017.pdf.txtExtracted texttext/plain65611https://www.arca.fiocruz.br/bitstream/icict/26682/3/nelson_kotowski_etal_IOC_2017.pdf.txt238d5f042da94ebc98400eca75ce03f2MD53icict/266822018-12-30 20:17:23.679oai:www.arca.fiocruz.br:icict/26682Q0VTU8ODTyBOw4NPIEVYQ0xVU0lWQSBERSBESVJFSVRPUyBBVVRPUkFJUwoKQW8gYWNlaXRhciBvcyBURVJNT1MgZSBDT05EScOHw5VFUyBkZXN0YSBDRVNTw4NPLCBvIEFVVE9SIGUvb3UgVElUVUxBUiBkZSBkaXJlaXRvcwphdXRvcmFpcyBzb2JyZSBhIE9CUkEgZGUgcXVlIHRyYXRhIGVzdGUgZG9jdW1lbnRvOgoKKDEpIENFREUgZSBUUkFOU0ZFUkUsIHRvdGFsIGUgZ3JhdHVpdGFtZW50ZSwgw6AgRklPQ1JVWiAtIEZVTkRBw4fDg08gT1NXQUxETyBDUlVaLCBlbQpjYXLDoXRlciBwZXJtYW5lbnRlLCBpcnJldm9nw6F2ZWwgZSBOw4NPIEVYQ0xVU0lWTywgdG9kb3Mgb3MgZGlyZWl0b3MgcGF0cmltb25pYWlzIE7Dg08KQ09NRVJDSUFJUyBkZSB1dGlsaXphw6fDo28gZGEgT0JSQSBhcnTDrXN0aWNhIGUvb3UgY2llbnTDrWZpY2EgaW5kaWNhZGEgYWNpbWEsIGluY2x1c2l2ZSBvcyBkaXJlaXRvcwpkZSB2b3ogZSBpbWFnZW0gdmluY3VsYWRvcyDDoCBPQlJBLCBkdXJhbnRlIHRvZG8gbyBwcmF6byBkZSBkdXJhw6fDo28gZG9zIGRpcmVpdG9zIGF1dG9yYWlzLCBlbQpxdWFscXVlciBpZGlvbWEgZSBlbSB0b2RvcyBvcyBwYcOtc2VzOwoKKDIpIEFDRUlUQSBxdWUgYSBjZXNzw6NvIHRvdGFsIG7Do28gZXhjbHVzaXZhLCBwZXJtYW5lbnRlIGUgaXJyZXZvZ8OhdmVsIGRvcyBkaXJlaXRvcyBhdXRvcmFpcwpwYXRyaW1vbmlhaXMgbsOjbyBjb21lcmNpYWlzIGRlIHV0aWxpemHDp8OjbyBkZSBxdWUgdHJhdGEgZXN0ZSBkb2N1bWVudG8gaW5jbHVpLCBleGVtcGxpZmljYXRpdmFtZW50ZSwKb3MgZGlyZWl0b3MgZGUgZGlzcG9uaWJpbGl6YcOnw6NvIGUgY29tdW5pY2HDp8OjbyBww7pibGljYSBkYSBPQlJBLCBlbSBxdWFscXVlciBtZWlvIG91IHZlw61jdWxvLAppbmNsdXNpdmUgZW0gUmVwb3NpdMOzcmlvcyBEaWdpdGFpcywgYmVtIGNvbW8gb3MgZGlyZWl0b3MgZGUgcmVwcm9kdcOnw6NvLCBleGliacOnw6NvLCBleGVjdcOnw6NvLApkZWNsYW1hw6fDo28sIHJlY2l0YcOnw6NvLCBleHBvc2nDp8OjbywgYXJxdWl2YW1lbnRvLCBpbmNsdXPDo28gZW0gYmFuY28gZGUgZGFkb3MsIHByZXNlcnZhw6fDo28sIGRpZnVzw6NvLApkaXN0cmlidWnDp8OjbywgZGl2dWxnYcOnw6NvLCBlbXByw6lzdGltbywgdHJhZHXDp8OjbywgZHVibGFnZW0sIGxlZ2VuZGFnZW0sIGluY2x1c8OjbyBlbSBub3ZhcyBvYnJhcyBvdQpjb2xldMOibmVhcywgcmV1dGlsaXphw6fDo28sIGVkacOnw6NvLCBwcm9kdcOnw6NvIGRlIG1hdGVyaWFsIGRpZMOhdGljbyBlIGN1cnNvcyBvdSBxdWFscXVlciBmb3JtYSBkZQp1dGlsaXphw6fDo28gbsOjbyBjb21lcmNpYWw7CgooMykgUkVDT05IRUNFIHF1ZSBhIGNlc3PDo28gYXF1aSBlc3BlY2lmaWNhZGEgY29uY2VkZSDDoCBGSU9DUlVaIC0gRlVOREHDh8ODTyBPU1dBTERPCkNSVVogbyBkaXJlaXRvIGRlIGF1dG9yaXphciBxdWFscXVlciBwZXNzb2Eg4oCTIGbDrXNpY2Egb3UganVyw61kaWNhLCBww7pibGljYSBvdSBwcml2YWRhLCBuYWNpb25hbCBvdQplc3RyYW5nZWlyYSDigJMgYSBhY2Vzc2FyIGUgdXRpbGl6YXIgYW1wbGFtZW50ZSBhIE9CUkEsIHNlbSBleGNsdXNpdmlkYWRlLCBwYXJhIHF1YWlzcXVlcgpmaW5hbGlkYWRlcyBuw6NvIGNvbWVyY2lhaXM7CgooNCkgREVDTEFSQSBxdWUgYSBvYnJhIMOpIGNyaWHDp8OjbyBvcmlnaW5hbCBlIHF1ZSDDqSBvIHRpdHVsYXIgZG9zIGRpcmVpdG9zIGFxdWkgY2VkaWRvcyBlIGF1dG9yaXphZG9zLApyZXNwb25zYWJpbGl6YW5kby1zZSBpbnRlZ3JhbG1lbnRlIHBlbG8gY29udGXDumRvIGUgb3V0cm9zIGVsZW1lbnRvcyBxdWUgZmF6ZW0gcGFydGUgZGEgT0JSQSwKaW5jbHVzaXZlIG9zIGRpcmVpdG9zIGRlIHZveiBlIGltYWdlbSB2aW5jdWxhZG9zIMOgIE9CUkEsIG9icmlnYW5kby1zZSBhIGluZGVuaXphciB0ZXJjZWlyb3MgcG9yCmRhbm9zLCBiZW0gY29tbyBpbmRlbml6YXIgZSByZXNzYXJjaXIgYSBGSU9DUlVaIC0gRlVOREHDh8ODTyBPU1dBTERPIENSVVogZGUKZXZlbnR1YWlzIGRlc3Blc2FzIHF1ZSB2aWVyZW0gYSBzdXBvcnRhciwgZW0gcmF6w6NvIGRlIHF1YWxxdWVyIG9mZW5zYSBhIGRpcmVpdG9zIGF1dG9yYWlzIG91CmRpcmVpdG9zIGRlIHZveiBvdSBpbWFnZW0sIHByaW5jaXBhbG1lbnRlIG5vIHF1ZSBkaXogcmVzcGVpdG8gYSBwbMOhZ2lvIGUgdmlvbGHDp8O1ZXMgZGUgZGlyZWl0b3M7CgooNSkgQUZJUk1BIHF1ZSBjb25oZWNlIGEgUG9sw610aWNhIEluc3RpdHVjaW9uYWwgZGUgQWNlc3NvIEFiZXJ0byBkYSBGSU9DUlVaIC0gRlVOREHDh8ODTwpPU1dBTERPIENSVVogZSBhcyBkaXJldHJpemVzIHBhcmEgbyBmdW5jaW9uYW1lbnRvIGRvIHJlcG9zaXTDs3JpbyBpbnN0aXR1Y2lvbmFsIEFSQ0EuCgpBIFBvbMOtdGljYSBJbnN0aXR1Y2lvbmFsIGRlIEFjZXNzbyBBYmVydG8gZGEgRklPQ1JVWiAtIEZVTkRBw4fDg08gT1NXQUxETyBDUlVaIHJlc2VydmEKZXhjbHVzaXZhbWVudGUgYW8gQVVUT1Igb3MgZGlyZWl0b3MgbW9yYWlzIGUgb3MgdXNvcyBjb21lcmNpYWlzIHNvYnJlIGFzIG9icmFzIGRlIHN1YSBhdXRvcmlhCmUvb3UgdGl0dWxhcmlkYWRlLCBzZW5kbyBvcyB0ZXJjZWlyb3MgdXN1w6FyaW9zIHJlc3BvbnPDoXZlaXMgcGVsYSBhdHJpYnVpw6fDo28gZGUgYXV0b3JpYSBlIG1hbnV0ZW7Dp8OjbwpkYSBpbnRlZ3JpZGFkZSBkYSBPQlJBIGVtIHF1YWxxdWVyIHV0aWxpemHDp8Ojby4KCkEgUG9sw610aWNhIEluc3RpdHVjaW9uYWwgZGUgQWNlc3NvIEFiZXJ0byBkYSBGSU9DUlVaIC0gRlVOREHDh8ODTyBPU1dBTERPIENSVVoKcmVzcGVpdGEgb3MgY29udHJhdG9zIGUgYWNvcmRvcyBwcmVleGlzdGVudGVzIGRvcyBBdXRvcmVzIGNvbSB0ZXJjZWlyb3MsIGNhYmVuZG8gYW9zIEF1dG9yZXMKaW5mb3JtYXIgw6AgSW5zdGl0dWnDp8OjbyBhcyBjb25kacOnw7VlcyBlIG91dHJhcyByZXN0cmnDp8O1ZXMgaW1wb3N0YXMgcG9yIGVzdGVzIGluc3RydW1lbnRvcy4KRepositório InstitucionalPUBhttps://www.arca.fiocruz.br/oai/requestrepositorio.arca@fiocruz.bropendoar:21352018-12-30T22:17:23Repositório Institucional da FIOCRUZ (ARCA) - Fundação Oswaldo Cruz (FIOCRUZ)false
dc.title.pt_BR.fl_str_mv Managing Workflows on top of a Cloud Computing Orchestrator for using heterogeneous environments on e-Science
title Managing Workflows on top of a Cloud Computing Orchestrator for using heterogeneous environments on e-Science
spellingShingle Managing Workflows on top of a Cloud Computing Orchestrator for using heterogeneous environments on e-Science
Carrión, Abel
Workflow
Workflow Management Systems
Cloud Orchestrator
Multi-plataform
e-Science
Cloud Computing
Comparative genomics
title_short Managing Workflows on top of a Cloud Computing Orchestrator for using heterogeneous environments on e-Science
title_full Managing Workflows on top of a Cloud Computing Orchestrator for using heterogeneous environments on e-Science
title_fullStr Managing Workflows on top of a Cloud Computing Orchestrator for using heterogeneous environments on e-Science
title_full_unstemmed Managing Workflows on top of a Cloud Computing Orchestrator for using heterogeneous environments on e-Science
title_sort Managing Workflows on top of a Cloud Computing Orchestrator for using heterogeneous environments on e-Science
author Carrión, Abel
author_facet Carrión, Abel
Caballer, Miguel
Blanquer, Ignacio
Kotowski, Nelson
Jardim, Rodrigo
Dàvila, Alberto Martin Rivera
author_role author
author2 Caballer, Miguel
Blanquer, Ignacio
Kotowski, Nelson
Jardim, Rodrigo
Dàvila, Alberto Martin Rivera
author2_role author
author
author
author
author
dc.contributor.author.fl_str_mv Carrión, Abel
Caballer, Miguel
Blanquer, Ignacio
Kotowski, Nelson
Jardim, Rodrigo
Dàvila, Alberto Martin Rivera
dc.subject.en.pt_BR.fl_str_mv Workflow
Workflow Management Systems
Cloud Orchestrator
Multi-plataform
e-Science
Cloud Computing
Comparative genomics
topic Workflow
Workflow Management Systems
Cloud Orchestrator
Multi-plataform
e-Science
Cloud Computing
Comparative genomics
description Versão preprint do autor: O autor pode arquivar a versão preprint (i.e. antes do peer-review)
publishDate 2018
dc.date.accessioned.fl_str_mv 2018-05-30T12:41:30Z
dc.date.available.fl_str_mv 2018-05-30T12:41:30Z
dc.date.issued.fl_str_mv 2018
dc.type.status.fl_str_mv info:eu-repo/semantics/publishedVersion
dc.type.driver.fl_str_mv info:eu-repo/semantics/article
format article
status_str publishedVersion
dc.identifier.citation.fl_str_mv CARRIÓN, Abel et al. Managing Workflows on top of a Cloud Computing Orchestrator for using heterogeneous environments on e-Science. International Journal of Web and Grid Services, v. 13, n. 4, p. 1-38, Mar. 2018.
dc.identifier.uri.fl_str_mv https://www.arca.fiocruz.br/handle/icict/26682
identifier_str_mv CARRIÓN, Abel et al. Managing Workflows on top of a Cloud Computing Orchestrator for using heterogeneous environments on e-Science. International Journal of Web and Grid Services, v. 13, n. 4, p. 1-38, Mar. 2018.
url https://www.arca.fiocruz.br/handle/icict/26682
dc.language.iso.fl_str_mv eng
language eng
dc.rights.driver.fl_str_mv info:eu-repo/semantics/openAccess
eu_rights_str_mv openAccess
dc.publisher.none.fl_str_mv Elsevier
publisher.none.fl_str_mv Elsevier
dc.source.none.fl_str_mv reponame:Repositório Institucional da FIOCRUZ (ARCA)
instname:Fundação Oswaldo Cruz (FIOCRUZ)
instacron:FIOCRUZ
instname_str Fundação Oswaldo Cruz (FIOCRUZ)
instacron_str FIOCRUZ
institution FIOCRUZ
reponame_str Repositório Institucional da FIOCRUZ (ARCA)
collection Repositório Institucional da FIOCRUZ (ARCA)
bitstream.url.fl_str_mv https://www.arca.fiocruz.br/bitstream/icict/26682/1/license.txt
https://www.arca.fiocruz.br/bitstream/icict/26682/2/nelson_kotowski_etal_IOC_2017.pdf
https://www.arca.fiocruz.br/bitstream/icict/26682/3/nelson_kotowski_etal_IOC_2017.pdf.txt
bitstream.checksum.fl_str_mv 5a560609d32a3863062d77ff32785d58
27b8179b23c0f97566e075d8aa6939fb
238d5f042da94ebc98400eca75ce03f2
bitstream.checksumAlgorithm.fl_str_mv MD5
MD5
MD5
repository.name.fl_str_mv Repositório Institucional da FIOCRUZ (ARCA) - Fundação Oswaldo Cruz (FIOCRUZ)
repository.mail.fl_str_mv repositorio.arca@fiocruz.br
_version_ 1798324739672899584

Registros relacionados