Junções por similaridade com expressões complexas em ambientes distribuídos

• Autor(a) principal: Oliveira, Diego Junior do Carmo
• Data de Publicação: 2018
• Tipo de documento: Dissertação
• Idioma: por
• Título da fonte: Repositório Institucional da UFG
• dARK ID: ark:/38995/001300000b53b
• Texto Completo: http://repositorio.bc.ufg.br/tede/handle/tede/8928
• topic: Junção por similaridade; Sistemas distribuídos; Apache spark; Big data; Similarity joins; Distributed platforms; CIENCIAS EXATAS E DA TERRA::CIENCIA DA COMPUTACAO

A recurrent problem that degrades the quality of the information in databases is the presence of duplicates, i.e., multiple representations of the same real-world entity. Despite being computationally expensive, the use of similarity operations is fundamental to identify duplicates. Furthermore, real-world data is typically composed of different attributes and each attribute represents a distinct type of information. The application of complex similarity expressions is important in this context because they allow considering the importance of each attribute in the similarity evaluation. However, due to a large amount of data present in Big Data applications, it has become crucial to perform these operations in parallel and distributed processing environments. In order to solve such problems of great relevance to organizations, this work proposes a novel strategy to identify duplicates in textual data by using similarity joins with complex expressions in a distributed environment.