Embedded representations for item descriptions in unsupervised tasks

• Autor(a) principal: Pedro Paulo Valadares Brum
• Data de Publicação: 2021
• Tipo de documento: Dissertação
• Idioma: eng
• Título da fonte: Repositório Institucional da UFMG
• Texto Completo: http://hdl.handle.net/1843/42299
• topic: Text representation; Text clustering; Word embeddings; Representação de texto; Agrupamento de texto; Vetores de palavras; Computação – Teses; Representação documentária – Teses; Agrupamento de texto – Teses; Processamento da linguagem natural (Computação) - Teses

Most machine learning algorithms require a fixed-size vector as input. This makes the area of text representation a challenging one in Natural Language Processing (NLP) tasks, and its results are highly dependent on the target application. For NLP tasks, this fixed-size vector usually represents a sentence or a paragraph. However, building text representations capable of capturing semantic and context-specific information is not a simple task. In this work, we propose a methodology to solve a real-world problem: the identification of unique objects from public procurement stored in the databases of the Federal Public Ministry of Minas Gerais. These scenarios pose challenges that go beyond those commonly known in the text representation area, as we want to group descriptions of products or services. These descriptions in general do not follow the grammatical structure of a sentence in the Portuguese language, as they are mostly formed by nouns, adjectives, and quantities, the latter describing the quantity of items purchased/contracted or the unit of measure that describes the item. Within the proposed framework, we emphasize the text representation problem for unsupervised algorithms. We propose a simple information extraction strategy to improve the quality of sentence vectors, focusing on specific terms such as numbers and nouns, and present a modification of the BERT siamese network, which can be used in an unsupervised way to generate embeddings that carry semantic and syntactic information from descriptions. We also identify numerical terms and measurement units as the two main components in this context, and show that a simple method of standardizing numbers has a significant effect on the results. Experimental results show improvements from the proposed framework in relation to state-of-the-art methods.