Distributed Job Scheduler

• Autor(a) principal: Carvalho, Jorge Miguel de Sousa
• Data de Publicação: 2022
• Tipo de documento: Dissertação
• Idioma: eng
• Título da fonte: Repositório Científico de Acesso Aberto de Portugal (Repositórios Cientìficos)
• Texto Completo: http://hdl.handle.net/10400.22/20727
• topic: Job Scheduler; Distributed Systems; Microservices; Databases; Reliability; Fault Tolerance; Sistema de agendamento de tarefas; Sistemas distribuídos; Micro-serviços; Bases de dados; Confiabilidade; Tolerância a falhas

Since Cron was released for Unix operating systems in 1975, it became a useful tool for making developers and system administrators’ life easier by programming tasks to be launched autonomously. Although Cron is a simple and powerful tool, it has some problems associated with it, such as lack of visibility, and complexity, because scheduling tasks using crontab’s notation can sometimes be difficult. As times wore on, new approaches of job scheduling systems emerged, most of them providing a user friendly interface to manage jobs/tasks scheduling and reports or statistics about job’s execution. Every day Jumia dispatches millions of marketing campaigns which include emails, newsletters, push notifications, SMS, and other types of channels to engage its customers to visit the e-commerce online store and other Jumia applications. In Jumia Marketing and Digital Services team’s systems a job scheduler is also used, it’s called Eye Of Sauron (EOS). EOS is very useful, however it wasn’t designed very well when it begun and, nowadays, it’s considered a problem for Jumia’s business because it’s not reliable. It’s Eye Of Sauron’s duty to trigger the dispatch process for all the marketing campaigns for Jumia’s users, so it needs to be well designed and provide trust to Jumia’s business stakeholders. With this project the problems from the original service were addressed. A new distributed job scheduler named Eye of Sauron v2 was designed and developed. It is composed by several components that are capable of being scaled horizontally and/or vertically. The new system also uses a message broker for asynchronous communication and a relational database as storage solution. The new job scheduler was considered successful because it was evaluated with a quality percentage of eighty-seven points using a Quantitative Evaluation Framework (QEF) model that considers numerous aspects not only related with functionality, but also with user interface and experience.