Otimização de desempenho de algoritmo para detecção de outliers em séries temporais

• Autor(a) principal: Lima, Beatriz Ferreira de
• Data de Publicação: 2022
• Tipo de documento: Trabalho de conclusão de curso
• Idioma: por
• Título da fonte: Repositório Institucional da UNESP
• Texto Completo: http://hdl.handle.net/11449/216225
• topic: Apache spark; Ciência da computação; Banco de dados; Séries temporais; Algoritmos paralelos; Limpeza de dados; Detecção de outliers

Time series have been present in several areas of great economic value, such as the stock market and industry. In circumstances of Big Data characteristics, data can be processed in real time or near real time, and sources capable of generating high volumes of data, which impose the preparation phase to need to perform cleaning of these data effectively and efficiently when dealing with these requirements. There are several algorithms that can be used to carry out this process, but they may have limitations such as low yield, data distortion, high processing time, among others. One of the problems to be addressed in data preparation is the detection of outliers, data that can reflect distortions and that can imply additional costs in the data cleaning phase. Thus, this work aimed to propose an algorithm that performs the detection of outliers and, subsequently, the cleaning of outliers in an effective and efficient way, which sought to maintain the completeness of the information obtained through time series data. From the tests carried out with the algorithm, it was possible to verify a significant reduction in processing time, up to 70%, without altering the original data.