Fast and Scalable Outlier Detection with Metric Access Methods

• Autor(a) principal: Bispo Junior, Altamir Gomes
• Data de Publicação: 2019
• Tipo de documento: Dissertação
• Idioma: eng
• Título da fonte: Biblioteca Digital de Teses e Dissertações da USP
• Texto Completo: http://www.teses.usp.br/teses/disponiveis/55/55134/tde-04102019-154943/
• topic: Applied computational sciences; Ciência computacional aplicada; Complex data; Dados complexos; Data mining; Detecção de outliers não-supervisionada; Métodos de acesso métrico; Metric access methods; Mineração de dados; Unsupervised outlier detection

It is well-known that the existing theoretical models for outlier detection make assumptions that may not reflect the true nature of outliers in every real application. This dissertation describes an empirical study performed on unsupervised outlier detection using 8 algorithms from the state-of-the-art and 8 datasets that refer to a variety of real-world tasks of practical relevance, such as spotting cyberattacks, clinical pathologies and abnormalities occurring in nature. We present our lowdown on the results obtained, pointing out to the strengths and weaknesses of each technique from the application specialists point of view, which is a shift from the designer-based point of view that is commonly adopted. Many of the techniques had unfeasibly high runtime requirements or failed to spot what the specialists consider as outliers in their own data. To tackle this issue, we propose MetricABOD: a novel ABOD-based algorithm that makes the analysis up to thousands of times faster, still being in average 26% more accurate than the most accurate related work. This improvement is tantamount to practical outlier detection in many real-world applications for which the existing methods present unstable accuracy or unfeasible runtime requirements. Finally, we studied two collections of text data to show that our MetricABOD works also for adimensional, purely metric data.