Evaluating generative models in high energy physics
Autor(a) principal: | |
---|---|
Data de Publicação: | 2023 |
Outros Autores: | , , , , , |
Tipo de documento: | Artigo |
Idioma: | eng |
Título da fonte: | Repositório Institucional da UNESP |
Texto Completo: | http://dx.doi.org/10.1103/PhysRevD.107.076017 http://hdl.handle.net/11449/247328 |
Resumo: | There has been a recent explosion in research into machine-learning-based generative modeling to tackle computational challenges for simulations in high energy physics (HEP). In order to use such alternative simulators in practice, we need well-defined metrics to compare different generative models and evaluate their discrepancy from the true distributions. We present the first systematic review and investigation into evaluation metrics and their sensitivity to failure modes of generative models, using the framework of two-sample goodness-of-fit testing, and their relevance and viability for HEP. Inspired by previous work in both physics and computer vision, we propose two new metrics, the Fréchet and kernel physics distances (FPD and KPD, respectively) and perform a variety of experiments measuring their performance on simple Gaussian-distributed and simulated high energy jet datasets. We find FPD, in particular, to be the most sensitive metric to all alternative jet distributions tested and recommend its adoption, along with the KPD and Wasserstein distances between individual feature distributions, for evaluating generative models in HEP. We finally demonstrate the efficacy of these proposed metrics in evaluating and comparing a novel attention-based generative adversarial particle transformer to the state-of-the-art message-passing generative adversarial network jet simulation model. The code for our proposed metrics is provided in the open source jetnet python library. |
id |
UNSP_c57dc39e3493efb2f42e26c754a9f533 |
---|---|
oai_identifier_str |
oai:repositorio.unesp.br:11449/247328 |
network_acronym_str |
UNSP |
network_name_str |
Repositório Institucional da UNESP |
repository_id_str |
2946 |
spelling |
Evaluating generative models in high energy physicsThere has been a recent explosion in research into machine-learning-based generative modeling to tackle computational challenges for simulations in high energy physics (HEP). In order to use such alternative simulators in practice, we need well-defined metrics to compare different generative models and evaluate their discrepancy from the true distributions. We present the first systematic review and investigation into evaluation metrics and their sensitivity to failure modes of generative models, using the framework of two-sample goodness-of-fit testing, and their relevance and viability for HEP. Inspired by previous work in both physics and computer vision, we propose two new metrics, the Fréchet and kernel physics distances (FPD and KPD, respectively) and perform a variety of experiments measuring their performance on simple Gaussian-distributed and simulated high energy jet datasets. We find FPD, in particular, to be the most sensitive metric to all alternative jet distributions tested and recommend its adoption, along with the KPD and Wasserstein distances between individual feature distributions, for evaluating generative models in HEP. We finally demonstrate the efficacy of these proposed metrics in evaluating and comparing a novel attention-based generative adversarial particle transformer to the state-of-the-art message-passing generative adversarial network jet simulation model. The code for our proposed metrics is provided in the open source jetnet python library.University of California San DiegoEuropean Center for Nuclear Research (CERN)Universidade Estadual Paulista, SPFermilabUniversidade Estadual Paulista, SPUniversity of California San DiegoEuropean Center for Nuclear Research (CERN)Universidade Estadual Paulista (UNESP)FermilabKansal, RaghavLi, AnniDuarte, JavierChernyavskaya, NadezdaPierini, MaurizioOrzari, Breno [UNESP]Tomei, Thiago [UNESP]2023-07-29T13:13:02Z2023-07-29T13:13:02Z2023-04-01info:eu-repo/semantics/publishedVersioninfo:eu-repo/semantics/articlehttp://dx.doi.org/10.1103/PhysRevD.107.076017Physical Review D, v. 107, n. 7, 2023.2470-00292470-0010http://hdl.handle.net/11449/24732810.1103/PhysRevD.107.0760172-s2.0-85158863233Scopusreponame:Repositório Institucional da UNESPinstname:Universidade Estadual Paulista (UNESP)instacron:UNESPengPhysical Review Dinfo:eu-repo/semantics/openAccess2023-07-29T13:13:02Zoai:repositorio.unesp.br:11449/247328Repositório InstitucionalPUBhttp://repositorio.unesp.br/oai/requestopendoar:29462023-07-29T13:13:02Repositório Institucional da UNESP - Universidade Estadual Paulista (UNESP)false |
dc.title.none.fl_str_mv |
Evaluating generative models in high energy physics |
title |
Evaluating generative models in high energy physics |
spellingShingle |
Evaluating generative models in high energy physics Kansal, Raghav |
title_short |
Evaluating generative models in high energy physics |
title_full |
Evaluating generative models in high energy physics |
title_fullStr |
Evaluating generative models in high energy physics |
title_full_unstemmed |
Evaluating generative models in high energy physics |
title_sort |
Evaluating generative models in high energy physics |
author |
Kansal, Raghav |
author_facet |
Kansal, Raghav Li, Anni Duarte, Javier Chernyavskaya, Nadezda Pierini, Maurizio Orzari, Breno [UNESP] Tomei, Thiago [UNESP] |
author_role |
author |
author2 |
Li, Anni Duarte, Javier Chernyavskaya, Nadezda Pierini, Maurizio Orzari, Breno [UNESP] Tomei, Thiago [UNESP] |
author2_role |
author author author author author author |
dc.contributor.none.fl_str_mv |
University of California San Diego European Center for Nuclear Research (CERN) Universidade Estadual Paulista (UNESP) Fermilab |
dc.contributor.author.fl_str_mv |
Kansal, Raghav Li, Anni Duarte, Javier Chernyavskaya, Nadezda Pierini, Maurizio Orzari, Breno [UNESP] Tomei, Thiago [UNESP] |
description |
There has been a recent explosion in research into machine-learning-based generative modeling to tackle computational challenges for simulations in high energy physics (HEP). In order to use such alternative simulators in practice, we need well-defined metrics to compare different generative models and evaluate their discrepancy from the true distributions. We present the first systematic review and investigation into evaluation metrics and their sensitivity to failure modes of generative models, using the framework of two-sample goodness-of-fit testing, and their relevance and viability for HEP. Inspired by previous work in both physics and computer vision, we propose two new metrics, the Fréchet and kernel physics distances (FPD and KPD, respectively) and perform a variety of experiments measuring their performance on simple Gaussian-distributed and simulated high energy jet datasets. We find FPD, in particular, to be the most sensitive metric to all alternative jet distributions tested and recommend its adoption, along with the KPD and Wasserstein distances between individual feature distributions, for evaluating generative models in HEP. We finally demonstrate the efficacy of these proposed metrics in evaluating and comparing a novel attention-based generative adversarial particle transformer to the state-of-the-art message-passing generative adversarial network jet simulation model. The code for our proposed metrics is provided in the open source jetnet python library. |
publishDate |
2023 |
dc.date.none.fl_str_mv |
2023-07-29T13:13:02Z 2023-07-29T13:13:02Z 2023-04-01 |
dc.type.status.fl_str_mv |
info:eu-repo/semantics/publishedVersion |
dc.type.driver.fl_str_mv |
info:eu-repo/semantics/article |
format |
article |
status_str |
publishedVersion |
dc.identifier.uri.fl_str_mv |
http://dx.doi.org/10.1103/PhysRevD.107.076017 Physical Review D, v. 107, n. 7, 2023. 2470-0029 2470-0010 http://hdl.handle.net/11449/247328 10.1103/PhysRevD.107.076017 2-s2.0-85158863233 |
url |
http://dx.doi.org/10.1103/PhysRevD.107.076017 http://hdl.handle.net/11449/247328 |
identifier_str_mv |
Physical Review D, v. 107, n. 7, 2023. 2470-0029 2470-0010 10.1103/PhysRevD.107.076017 2-s2.0-85158863233 |
dc.language.iso.fl_str_mv |
eng |
language |
eng |
dc.relation.none.fl_str_mv |
Physical Review D |
dc.rights.driver.fl_str_mv |
info:eu-repo/semantics/openAccess |
eu_rights_str_mv |
openAccess |
dc.source.none.fl_str_mv |
Scopus reponame:Repositório Institucional da UNESP instname:Universidade Estadual Paulista (UNESP) instacron:UNESP |
instname_str |
Universidade Estadual Paulista (UNESP) |
instacron_str |
UNESP |
institution |
UNESP |
reponame_str |
Repositório Institucional da UNESP |
collection |
Repositório Institucional da UNESP |
repository.name.fl_str_mv |
Repositório Institucional da UNESP - Universidade Estadual Paulista (UNESP) |
repository.mail.fl_str_mv |
|
_version_ |
1803046657552547840 |