PetroBERT: A Domain Adaptation Language Model for Oil and Gas Applications in Portuguese

• Autor(a) principal: Rodrigues, Rafael B. M. [UNESP]
• Data de Publicação: 2022
• Outros Autores: Privatto, Pedro I. M. [UNESP], de Sousa, Gustavo José [UNESP], Murari, Rafael P. [UNESP], Afonso, Luis C. S. [UNESP], Papa, João P. [UNESP], Pedronette, Daniel C. G. [UNESP], Guilherme, Ivan R. [UNESP], Perrout, Stephan R., Riente, Aliel F.
• Tipo de documento: Artigo de conferência
• Idioma: eng
• Título da fonte: Repositório Institucional da UNESP
• Texto Completo: http://dx.doi.org/10.1007/978-3-030-98305-5_10; http://hdl.handle.net/11449/234320
• topic: BERT; Domain adaption; Oil and gas

This work proposes the PetroBERT, which is a BERT-based model adapted to the oil and gas exploration domain in Portuguese. PetroBERT was pre-trained using the Petrolês corpus and a private daily drilling report corpus over BERT multilingual and BERTimbau. The proposed model was evaluated in the NER and sentence classification tasks and achieved interesting results, which shows its potential for such a domain. To the best of our knowledge, this is the first BERT-based model to the oil and gas context.