Stability and scalability of the CMS Global Pool: Pushing HTCondor and glideinWMS to new limits
Autor(a) principal: | |
---|---|
Data de Publicação: | 2017 |
Outros Autores: | , , , , , , , , , , |
Tipo de documento: | Artigo de conferência |
Idioma: | eng |
Título da fonte: | Repositório Institucional da UNESP |
Texto Completo: | http://dx.doi.org/10.1088/1742-6596/898/5/052031 http://hdl.handle.net/11449/220987 |
Resumo: | The CMS Global Pool, based on HTCondor and glideinWMS, is the main computing resource provisioning system for all CMS workflows, including analysis, Monte Carlo production, and detector data reprocessing activities. The total resources at Tier-1 and Tier-2 grid sites pledged to CMS exceed 100,000 CPU cores, while another 50,000 to 100,000 CPU cores are available opportunistically, pushing the needs of the Global Pool to higher scales each year. These resources are becoming more diverse in their accessibility and configuration over time. Furthermore, the challenge of stably running at higher and higher scales while introducing new modes of operation such as multi-core pilots, as well as the chaotic nature of physics analysis workflows, places huge strains on the submission infrastructure. This paper details some of the most important challenges to scalability and stability that the CMS Global Pool has faced since the beginning of the LHC Run II and how they were overcome. |
id |
UNSP_f3750f6c629892a19b3ef87e84086ac4 |
---|---|
oai_identifier_str |
oai:repositorio.unesp.br:11449/220987 |
network_acronym_str |
UNSP |
network_name_str |
Repositório Institucional da UNESP |
repository_id_str |
2946 |
spelling |
Stability and scalability of the CMS Global Pool: Pushing HTCondor and glideinWMS to new limitsThe CMS Global Pool, based on HTCondor and glideinWMS, is the main computing resource provisioning system for all CMS workflows, including analysis, Monte Carlo production, and detector data reprocessing activities. The total resources at Tier-1 and Tier-2 grid sites pledged to CMS exceed 100,000 CPU cores, while another 50,000 to 100,000 CPU cores are available opportunistically, pushing the needs of the Global Pool to higher scales each year. These resources are becoming more diverse in their accessibility and configuration over time. Furthermore, the challenge of stably running at higher and higher scales while introducing new modes of operation such as multi-core pilots, as well as the chaotic nature of physics analysis workflows, places huge strains on the submission infrastructure. This paper details some of the most important challenges to scalability and stability that the CMS Global Pool has faced since the beginning of the LHC Run II and how they were overcome.U.S. Department of EnergyNational Science FoundationCalifornia Institute of TechnologyUniversity of NebraskaFermi National Accelerator LaboratoryUniversity of Notre DameNational Centre for Physics Quaid-I-Azam UniversityUniversity of California San DiegoUniversidade Estadual PaulistaPort d'Informació CientíficaCentro de Investigaciones Energéeticas Medioambientales y Tecnológicas CIEMATUniversidade Estadual PaulistaCalifornia Institute of TechnologyUniversity of NebraskaFermi National Accelerator LaboratoryUniversity of Notre DameQuaid-I-Azam UniversityUniversity of California San DiegoUniversidade Estadual Paulista (UNESP)Port d'Informació CientíficaCIEMATBalcas, J.Bockelman, B.Hufnagel, D.Anampa, K HurtadoKhan, F AftabLarson, K.Letts, J.Marra Da Silva, J. [UNESP]Mascheroni, M.Mason, D.Yzquierdo, A. Perez-CaleroTiradani, A.2022-04-28T19:07:12Z2022-04-28T19:07:12Z2017-11-23info:eu-repo/semantics/publishedVersioninfo:eu-repo/semantics/conferenceObjecthttp://dx.doi.org/10.1088/1742-6596/898/5/052031Journal of Physics: Conference Series, v. 898, n. 5, 2017.1742-65961742-6588http://hdl.handle.net/11449/22098710.1088/1742-6596/898/5/0520312-s2.0-85038444697Scopusreponame:Repositório Institucional da UNESPinstname:Universidade Estadual Paulista (UNESP)instacron:UNESPengJournal of Physics: Conference Seriesinfo:eu-repo/semantics/openAccess2022-04-28T19:07:12Zoai:repositorio.unesp.br:11449/220987Repositório InstitucionalPUBhttp://repositorio.unesp.br/oai/requestopendoar:29462022-04-28T19:07:12Repositório Institucional da UNESP - Universidade Estadual Paulista (UNESP)false |
dc.title.none.fl_str_mv |
Stability and scalability of the CMS Global Pool: Pushing HTCondor and glideinWMS to new limits |
title |
Stability and scalability of the CMS Global Pool: Pushing HTCondor and glideinWMS to new limits |
spellingShingle |
Stability and scalability of the CMS Global Pool: Pushing HTCondor and glideinWMS to new limits Balcas, J. |
title_short |
Stability and scalability of the CMS Global Pool: Pushing HTCondor and glideinWMS to new limits |
title_full |
Stability and scalability of the CMS Global Pool: Pushing HTCondor and glideinWMS to new limits |
title_fullStr |
Stability and scalability of the CMS Global Pool: Pushing HTCondor and glideinWMS to new limits |
title_full_unstemmed |
Stability and scalability of the CMS Global Pool: Pushing HTCondor and glideinWMS to new limits |
title_sort |
Stability and scalability of the CMS Global Pool: Pushing HTCondor and glideinWMS to new limits |
author |
Balcas, J. |
author_facet |
Balcas, J. Bockelman, B. Hufnagel, D. Anampa, K Hurtado Khan, F Aftab Larson, K. Letts, J. Marra Da Silva, J. [UNESP] Mascheroni, M. Mason, D. Yzquierdo, A. Perez-Calero Tiradani, A. |
author_role |
author |
author2 |
Bockelman, B. Hufnagel, D. Anampa, K Hurtado Khan, F Aftab Larson, K. Letts, J. Marra Da Silva, J. [UNESP] Mascheroni, M. Mason, D. Yzquierdo, A. Perez-Calero Tiradani, A. |
author2_role |
author author author author author author author author author author author |
dc.contributor.none.fl_str_mv |
California Institute of Technology University of Nebraska Fermi National Accelerator Laboratory University of Notre Dame Quaid-I-Azam University University of California San Diego Universidade Estadual Paulista (UNESP) Port d'Informació Científica CIEMAT |
dc.contributor.author.fl_str_mv |
Balcas, J. Bockelman, B. Hufnagel, D. Anampa, K Hurtado Khan, F Aftab Larson, K. Letts, J. Marra Da Silva, J. [UNESP] Mascheroni, M. Mason, D. Yzquierdo, A. Perez-Calero Tiradani, A. |
description |
The CMS Global Pool, based on HTCondor and glideinWMS, is the main computing resource provisioning system for all CMS workflows, including analysis, Monte Carlo production, and detector data reprocessing activities. The total resources at Tier-1 and Tier-2 grid sites pledged to CMS exceed 100,000 CPU cores, while another 50,000 to 100,000 CPU cores are available opportunistically, pushing the needs of the Global Pool to higher scales each year. These resources are becoming more diverse in their accessibility and configuration over time. Furthermore, the challenge of stably running at higher and higher scales while introducing new modes of operation such as multi-core pilots, as well as the chaotic nature of physics analysis workflows, places huge strains on the submission infrastructure. This paper details some of the most important challenges to scalability and stability that the CMS Global Pool has faced since the beginning of the LHC Run II and how they were overcome. |
publishDate |
2017 |
dc.date.none.fl_str_mv |
2017-11-23 2022-04-28T19:07:12Z 2022-04-28T19:07:12Z |
dc.type.status.fl_str_mv |
info:eu-repo/semantics/publishedVersion |
dc.type.driver.fl_str_mv |
info:eu-repo/semantics/conferenceObject |
format |
conferenceObject |
status_str |
publishedVersion |
dc.identifier.uri.fl_str_mv |
http://dx.doi.org/10.1088/1742-6596/898/5/052031 Journal of Physics: Conference Series, v. 898, n. 5, 2017. 1742-6596 1742-6588 http://hdl.handle.net/11449/220987 10.1088/1742-6596/898/5/052031 2-s2.0-85038444697 |
url |
http://dx.doi.org/10.1088/1742-6596/898/5/052031 http://hdl.handle.net/11449/220987 |
identifier_str_mv |
Journal of Physics: Conference Series, v. 898, n. 5, 2017. 1742-6596 1742-6588 10.1088/1742-6596/898/5/052031 2-s2.0-85038444697 |
dc.language.iso.fl_str_mv |
eng |
language |
eng |
dc.relation.none.fl_str_mv |
Journal of Physics: Conference Series |
dc.rights.driver.fl_str_mv |
info:eu-repo/semantics/openAccess |
eu_rights_str_mv |
openAccess |
dc.source.none.fl_str_mv |
Scopus reponame:Repositório Institucional da UNESP instname:Universidade Estadual Paulista (UNESP) instacron:UNESP |
instname_str |
Universidade Estadual Paulista (UNESP) |
instacron_str |
UNESP |
institution |
UNESP |
reponame_str |
Repositório Institucional da UNESP |
collection |
Repositório Institucional da UNESP |
repository.name.fl_str_mv |
Repositório Institucional da UNESP - Universidade Estadual Paulista (UNESP) |
repository.mail.fl_str_mv |
|
_version_ |
1792961484961087488 |