Detecção de tópicos em documentos usando agrupamento de vetores de palavras

• Autor(a) principal: Miranda, Guilherme Raiol de
• Data de Publicação: 2020
• Tipo de documento: Dissertação
• Idioma: por
• Título da fonte: Biblioteca Digital de Teses e Dissertações do Mackenzie
• Texto Completo: https://dspace.mackenzie.br/handle/10899/28616
• topic: processamendo de língua natural; detecção de tópicos; word2vec; CNPQ::ENGENHARIAS

With the exponential increase of texts generated each year, the demand for Natural Language Processing techniques has been increasing, both by companies and by the Academy. Automatic topic detection in documents is one of the most challenging and useful tasks for information discovery and document summarization. Traditional topic detection techniques, such as the Latent Dirichlet Allocation (LDA) and the Non-Negative Matrix Factorization (NMF), originally did not produce good results for large databases nor use syntactic and semantic information to generate topics. Recently, word vectorization techniques, suchasWord2Vec, proved to have a good computational performance in large data sets and to be effective in representing words as word vectors in a distributed way, maintaining syntactic and semantic information. This dissertation proposes the verification of the following research hypothesis: Is Word2Vec capable of providing enough information to generate interpretable topics? For validation, a method, named Word2Topic, with two approaches for the generation of topics was proposed: direct grouping of word vectors and grouping after dimensionality reduction. The method was applied in two benchmark datasets and was compared with the traditional algorithms by a topic interpretability metric. It was observed in the results that the techniques proposed in one of the data bases generated set sof interpretable words or similar morphological classes. The topics obtained were similar to those of NMF, while LDA was unable to generate interpretable topics. It was not possible to fully validate the research hypothesis because the results of the second dataset did not lead to have the same interpretability or generated morphologically similar words.