Exploiting semantic similarity for improved text representation

• Autor(a) principal: Victor Silva Rodrigues
• Data de Publicação: 2018
• Tipo de documento: Dissertação
• Idioma: eng
• Título da fonte: Repositório Institucional da UFMG
• Texto Completo: http://hdl.handle.net/1843/39134
• topic: Text classification; Hyperwords; Bag-of-Words; Word embeddings; Classificação automática de documentos; Hyper-palavras; Saco-de-Palavras; Vetores de palavras; Computação – Teses.; Indexação automática – Teses.; Processamento da linguagem natural (Computação) – Teses

Automatic Document Classification is a key technique to help extracting useful information from the huge amount of textual data produced daily on the Web and inside organizations. Recently, Word Embeddings (e.g., Word2Vec) have been proposed for representing terms as vectors whose similarities should correlate with semantic relatedness. There has also been some research on how to use Word Embeddings to improve text classification. Nevertheless, current results depend on heavy and careful parameter tuning and still do not consistently outperform Bag-of-Words representation in a variety of scenarios. Since the nearest words of a given Word Embedding are all semantically related to each other, we propose a new method for generating features from clusters of similar Word Embeddings. We refer to these clusters as hyperwords, since they correspond to new semantic concepts, richer than simple words. We propose an adaptation of the TF-IDF weighting scheme for these new features so that they can be used similarly to the original terms, but substituting them. We demonstrate that features generated from hyperwords are significantly more discriminative than those obtained from simple words. We also experiment with the combination of the hyperwords-based representation with a state-of-art pooling technique, obtaining a very robust method. Extensive experiments performed using 24 benchmarks on topic classification and sentiment analysis against state-of-the-art baselines that exploit Word Embedding-based document representations show the superiority of our proposals by large margins, achieving gains up to 18% on topic classification datasets and 16% in sentiment classification datasets over the Bag-of-Words representation.