Retail monitoring and alerting system for observability

• Autor(a) principal: Silva, Diogo Manuel Pais
• Data de Publicação: 2023
• Tipo de documento: Dissertação
• Idioma: eng
• Título da fonte: Repositório Científico de Acesso Aberto de Portugal (Repositórios Cientìficos)
• Texto Completo: http://hdl.handle.net/10773/41046
• topic: Observability; Alerting; Time Series Database; Streaming; ETL; Kafka; InfluxDB; Oracle Retail

Oracle Retail implementations are supported by a somewhat complex infrastructure composed of various services. Retail Consult offers consultancy to clients with these systems, providing 24/7 support. Problems can arise in any of these infrastructure components with varying levels of priority, affecting the system’s availability. Manual monitoring of various clients’ systems is not possible due to the number of clients, systems, information dispersion and human resources available. Thus, automating the collection of key metrics from the various systems is essential. By constantly monitoring key metrics of the most important infrastructure components (the OS, WebLogic, and Database), alerts can be set for critical issues, improving the reaction time and efficacy of the solution. These metrics can also provide insight into the systems’ current and previous states, which can help debugging problems that arise. In this dissertation we propose a monitoring solution supported on a ETL (Extract, Transform, Load) pipeline based on the Unified Data Infrastructure 2.0, which pulls the desired metrics from the monitored system, sends them to a data broker (Kafka) and then loads them on a time series database (InfluxDB). Grafana was used to support metric visualization. It also allows customized analytics. With our system the metrics can be provided with low latency for visualization by an end user. The system also provides alerts and daily reports sent via Microsoft Teams and email, respectively. The system pilot was deployed in a production environment of a client, which had a very positive impact in the target infrastructure, helping spot problems in terms of Database concurrency, bugs, and resource usage, improving the quality of service provided by Retail Consult.