The Fine tuning of Language models for automation of Humor Detection
Autor(a) principal: | |
---|---|
Data de Publicação: | 2021 |
Outros Autores: | |
Tipo de documento: | Artigo |
Idioma: | eng |
Título da fonte: | INFOCOMP: Jornal de Ciência da Computação |
Texto Completo: | https://infocomp.dcc.ufla.br/index.php/infocomp/article/view/1612 |
Resumo: | In this paper, we propose a method that showcases a novel approach for humor identification using ALBERT and automation of best fit loss function identification and also the Optimiser identification. We have used two configurations of ALBERT, Albert-base and Albert-large. Using different hyper-parameters, we compare their results to obtain the best results for the binary classification problem of detecting texts that are humorous and those that are not humorous. We also determine the best optimizer and loss function that can be used to achieve state-of-the-art performance. The proposed system has been evaluated using metrics that include accuracy, precision, recall, F1-score, and the amount of time required. Among multiple loss functions, Adafactor on Albert-base model have shown promising results with 99\% of accuracy. Paper also talks about comparison of the proposed approach with other language models like BERT, ROBERTa to see a steep decline of 1/3rd in the time taken to train the model on 160K sentences. |
id |
UFLA-5_5fcbe450cc90883f9e57f1998e73dc0f |
---|---|
oai_identifier_str |
oai:infocomp.dcc.ufla.br:article/1612 |
network_acronym_str |
UFLA-5 |
network_name_str |
INFOCOMP: Jornal de Ciência da Computação |
repository_id_str |
|
spelling |
The Fine tuning of Language models for automation of Humor DetectionIn this paper, we propose a method that showcases a novel approach for humor identification using ALBERT and automation of best fit loss function identification and also the Optimiser identification. We have used two configurations of ALBERT, Albert-base and Albert-large. Using different hyper-parameters, we compare their results to obtain the best results for the binary classification problem of detecting texts that are humorous and those that are not humorous. We also determine the best optimizer and loss function that can be used to achieve state-of-the-art performance. The proposed system has been evaluated using metrics that include accuracy, precision, recall, F1-score, and the amount of time required. Among multiple loss functions, Adafactor on Albert-base model have shown promising results with 99\% of accuracy. Paper also talks about comparison of the proposed approach with other language models like BERT, ROBERTa to see a steep decline of 1/3rd in the time taken to train the model on 160K sentences.Editora da UFLA2021-12-01info:eu-repo/semantics/articleinfo:eu-repo/semantics/publishedVersionapplication/pdfhttps://infocomp.dcc.ufla.br/index.php/infocomp/article/view/1612INFOCOMP Journal of Computer Science; Vol. 20 No. 2 (2021): December 20211982-33631807-4545reponame:INFOCOMP: Jornal de Ciência da Computaçãoinstname:Universidade Federal de Lavras (UFLA)instacron:UFLAenghttps://infocomp.dcc.ufla.br/index.php/infocomp/article/view/1612/570Copyright (c) 2021 Tavishee Chauhan, Hemant Palivelainfo:eu-repo/semantics/openAccessChauhan, TavisheePalivela, Hemant2021-12-01T17:16:52Zoai:infocomp.dcc.ufla.br:article/1612Revistahttps://infocomp.dcc.ufla.br/index.php/infocompPUBhttps://infocomp.dcc.ufla.br/index.php/infocomp/oaiinfocomp@dcc.ufla.br||apfreire@dcc.ufla.br1982-33631807-4545opendoar:2024-05-21T19:54:47.168979INFOCOMP: Jornal de Ciência da Computação - Universidade Federal de Lavras (UFLA)true |
dc.title.none.fl_str_mv |
The Fine tuning of Language models for automation of Humor Detection |
title |
The Fine tuning of Language models for automation of Humor Detection |
spellingShingle |
The Fine tuning of Language models for automation of Humor Detection Chauhan, Tavishee |
title_short |
The Fine tuning of Language models for automation of Humor Detection |
title_full |
The Fine tuning of Language models for automation of Humor Detection |
title_fullStr |
The Fine tuning of Language models for automation of Humor Detection |
title_full_unstemmed |
The Fine tuning of Language models for automation of Humor Detection |
title_sort |
The Fine tuning of Language models for automation of Humor Detection |
author |
Chauhan, Tavishee |
author_facet |
Chauhan, Tavishee Palivela, Hemant |
author_role |
author |
author2 |
Palivela, Hemant |
author2_role |
author |
dc.contributor.author.fl_str_mv |
Chauhan, Tavishee Palivela, Hemant |
description |
In this paper, we propose a method that showcases a novel approach for humor identification using ALBERT and automation of best fit loss function identification and also the Optimiser identification. We have used two configurations of ALBERT, Albert-base and Albert-large. Using different hyper-parameters, we compare their results to obtain the best results for the binary classification problem of detecting texts that are humorous and those that are not humorous. We also determine the best optimizer and loss function that can be used to achieve state-of-the-art performance. The proposed system has been evaluated using metrics that include accuracy, precision, recall, F1-score, and the amount of time required. Among multiple loss functions, Adafactor on Albert-base model have shown promising results with 99\% of accuracy. Paper also talks about comparison of the proposed approach with other language models like BERT, ROBERTa to see a steep decline of 1/3rd in the time taken to train the model on 160K sentences. |
publishDate |
2021 |
dc.date.none.fl_str_mv |
2021-12-01 |
dc.type.driver.fl_str_mv |
info:eu-repo/semantics/article info:eu-repo/semantics/publishedVersion |
format |
article |
status_str |
publishedVersion |
dc.identifier.uri.fl_str_mv |
https://infocomp.dcc.ufla.br/index.php/infocomp/article/view/1612 |
url |
https://infocomp.dcc.ufla.br/index.php/infocomp/article/view/1612 |
dc.language.iso.fl_str_mv |
eng |
language |
eng |
dc.relation.none.fl_str_mv |
https://infocomp.dcc.ufla.br/index.php/infocomp/article/view/1612/570 |
dc.rights.driver.fl_str_mv |
Copyright (c) 2021 Tavishee Chauhan, Hemant Palivela info:eu-repo/semantics/openAccess |
rights_invalid_str_mv |
Copyright (c) 2021 Tavishee Chauhan, Hemant Palivela |
eu_rights_str_mv |
openAccess |
dc.format.none.fl_str_mv |
application/pdf |
dc.publisher.none.fl_str_mv |
Editora da UFLA |
publisher.none.fl_str_mv |
Editora da UFLA |
dc.source.none.fl_str_mv |
INFOCOMP Journal of Computer Science; Vol. 20 No. 2 (2021): December 2021 1982-3363 1807-4545 reponame:INFOCOMP: Jornal de Ciência da Computação instname:Universidade Federal de Lavras (UFLA) instacron:UFLA |
instname_str |
Universidade Federal de Lavras (UFLA) |
instacron_str |
UFLA |
institution |
UFLA |
reponame_str |
INFOCOMP: Jornal de Ciência da Computação |
collection |
INFOCOMP: Jornal de Ciência da Computação |
repository.name.fl_str_mv |
INFOCOMP: Jornal de Ciência da Computação - Universidade Federal de Lavras (UFLA) |
repository.mail.fl_str_mv |
infocomp@dcc.ufla.br||apfreire@dcc.ufla.br |
_version_ |
1799874742670327808 |