Learning Curves Prediction for a Transformers-based Model

Detalhes bibliográficos
Autor(a) principal: Cruz, Francisco
Data de Publicação: 2023
Outros Autores: Castelli, Mauro
Tipo de documento: Artigo
Idioma: eng
Título da fonte: Repositório Científico de Acesso Aberto de Portugal (Repositórios Cientìficos)
Texto Completo: http://hdl.handle.net/10362/159492
Resumo: Cruz, F., & Castelli, M. (2023). Learning Curves Prediction for a Transformers-based Model. Emerging Science Journal, 7(5), 1491-1500. https://doi.org/10.28991/ESJ-2023-07-05-03
id RCAP_708bea98fe14104073029567e7ac8473
oai_identifier_str oai:run.unl.pt:10362/159492
network_acronym_str RCAP
network_name_str Repositório Científico de Acesso Aberto de Portugal (Repositórios Cientìficos)
repository_id_str 7160
spelling Learning Curves Prediction for a Transformers-based ModelDataset SizeDocument Data ExtractionFine-TuningLearning CurvesTransformersGeneralCruz, F., & Castelli, M. (2023). Learning Curves Prediction for a Transformers-based Model. Emerging Science Journal, 7(5), 1491-1500. https://doi.org/10.28991/ESJ-2023-07-05-03One of the main challenges when training or fine-tuning a machine learning model concerns the number of observations necessary to achieve satisfactory performance. While, in general, more training observations result in a better-performing model, collecting more data can be time-consuming, expensive, or even impossible. For this reason, investigating the relationship between the dataset's size and the performance of a machine learning model is fundamental to deciding, with a certain likelihood, the minimum number of observations that are necessary to ensure a satisfactory-performing model is obtained as a result of the training process. The learning curve represents the relationship between the dataset’s size and the performance of the model and is especially useful when choosing a model for a specific task or planning the annotation work of a dataset. Thus, the purpose of this paper is to find the functions that best fit the learning curves of a Transformers-based model (LayoutLM) when fine-tuned to extract information from invoices. Two new datasets of invoices are made available for such a task. Combined with a third dataset already available online, 22 sub-datasets are defined, and their learning curves are plotted based on cross-validation results. The functions are fit using a non-linear least squares technique. The results show that both a biasymptotic and a Morgan-Mercer-Flodin function fit the learning curves extremely well. Also, an empirical relation is presented to predict the learning curve from a single parameter that may be easily obtained in the early stage of the annotation process.NOVA Information Management School (NOVA IMS)Information Management Research Center (MagIC) - NOVA Information Management SchoolRUNCruz, FranciscoCastelli, Mauro2023-11-02T22:08:59Z2023-10-012023-10-01T00:00:00Zinfo:eu-repo/semantics/publishedVersioninfo:eu-repo/semantics/article10application/pdfhttp://hdl.handle.net/10362/159492eng2610-9182PURE: 63539763https://doi.org/10.28991/ESJ-2023-07-05-03info:eu-repo/semantics/openAccessreponame:Repositório Científico de Acesso Aberto de Portugal (Repositórios Cientìficos)instname:Agência para a Sociedade do Conhecimento (UMIC) - FCT - Sociedade da Informaçãoinstacron:RCAAP2024-03-11T05:41:53Zoai:run.unl.pt:10362/159492Portal AgregadorONGhttps://www.rcaap.pt/oai/openaireopendoar:71602024-03-20T03:57:34.100523Repositório Científico de Acesso Aberto de Portugal (Repositórios Cientìficos) - Agência para a Sociedade do Conhecimento (UMIC) - FCT - Sociedade da Informaçãofalse
dc.title.none.fl_str_mv Learning Curves Prediction for a Transformers-based Model
title Learning Curves Prediction for a Transformers-based Model
spellingShingle Learning Curves Prediction for a Transformers-based Model
Cruz, Francisco
Dataset Size
Document Data Extraction
Fine-Tuning
Learning Curves
Transformers
General
title_short Learning Curves Prediction for a Transformers-based Model
title_full Learning Curves Prediction for a Transformers-based Model
title_fullStr Learning Curves Prediction for a Transformers-based Model
title_full_unstemmed Learning Curves Prediction for a Transformers-based Model
title_sort Learning Curves Prediction for a Transformers-based Model
author Cruz, Francisco
author_facet Cruz, Francisco
Castelli, Mauro
author_role author
author2 Castelli, Mauro
author2_role author
dc.contributor.none.fl_str_mv NOVA Information Management School (NOVA IMS)
Information Management Research Center (MagIC) - NOVA Information Management School
RUN
dc.contributor.author.fl_str_mv Cruz, Francisco
Castelli, Mauro
dc.subject.por.fl_str_mv Dataset Size
Document Data Extraction
Fine-Tuning
Learning Curves
Transformers
General
topic Dataset Size
Document Data Extraction
Fine-Tuning
Learning Curves
Transformers
General
description Cruz, F., & Castelli, M. (2023). Learning Curves Prediction for a Transformers-based Model. Emerging Science Journal, 7(5), 1491-1500. https://doi.org/10.28991/ESJ-2023-07-05-03
publishDate 2023
dc.date.none.fl_str_mv 2023-11-02T22:08:59Z
2023-10-01
2023-10-01T00:00:00Z
dc.type.status.fl_str_mv info:eu-repo/semantics/publishedVersion
dc.type.driver.fl_str_mv info:eu-repo/semantics/article
format article
status_str publishedVersion
dc.identifier.uri.fl_str_mv http://hdl.handle.net/10362/159492
url http://hdl.handle.net/10362/159492
dc.language.iso.fl_str_mv eng
language eng
dc.relation.none.fl_str_mv 2610-9182
PURE: 63539763
https://doi.org/10.28991/ESJ-2023-07-05-03
dc.rights.driver.fl_str_mv info:eu-repo/semantics/openAccess
eu_rights_str_mv openAccess
dc.format.none.fl_str_mv 10
application/pdf
dc.source.none.fl_str_mv reponame:Repositório Científico de Acesso Aberto de Portugal (Repositórios Cientìficos)
instname:Agência para a Sociedade do Conhecimento (UMIC) - FCT - Sociedade da Informação
instacron:RCAAP
instname_str Agência para a Sociedade do Conhecimento (UMIC) - FCT - Sociedade da Informação
instacron_str RCAAP
institution RCAAP
reponame_str Repositório Científico de Acesso Aberto de Portugal (Repositórios Cientìficos)
collection Repositório Científico de Acesso Aberto de Portugal (Repositórios Cientìficos)
repository.name.fl_str_mv Repositório Científico de Acesso Aberto de Portugal (Repositórios Cientìficos) - Agência para a Sociedade do Conhecimento (UMIC) - FCT - Sociedade da Informação
repository.mail.fl_str_mv
_version_ 1799138158074396672

Registros relacionados