Orchestration of music emotion recognition services - automating deployment, scaling and management

• Autor(a) principal: Canoso, João Pedro Bernardino
• Data de Publicação: 2021
• Tipo de documento: Dissertação
• Idioma: por
• Título da fonte: Repositório Científico de Acesso Aberto de Portugal (Repositórios Cientìficos)
• Texto Completo: http://hdl.handle.net/10400.26/39066
• topic: Music emotion recognition; Containers; Orchestration; Kubernetes; DevOps; Domínio/Área Científica::Engenharia e Tecnologia::Engenharia Eletrotécnica, Eletrónica e Informática

Every day, thousands of new songs are created and distributed over the internet. These ever-increasing databases introduced the need for automatic search and organization methods, that allow users to better filter and browse such collections. However, fundamental research in the MER field is very academic, with the typical work presenting results in the form classification metrics – how good the approach worked in the tested datasets and providing access to the data and methods. In order to overcome this problem, we built and deployed a platform to orchestrate a distributed, resilient, and scalable, music emotion recognition (MER) application using Kubernetes that can be easily expanded in the future. The solution developed is based on a proof of concept that explored the usage of containers and microservices in MER but had some gaps. We reengineered and expanded it, proposing a properly orchestrated, containerbased solution, and adopting a DevOps development culture with continuous integration (CI) and continuous delivery (CD) that in an automated way, makes it easy for the different teams to focus on developing new blocks separately. At the application level, instead of analyzing the audio signal recurring to only three audio features, the system now combines a large number of audio and lyric (text) features, explores different parts of audio (vocals, accompaniment) in segments (e.g., 30-second segments instead of the full song) and uses properly trained machine learning (ML) classifiers, a contribution by Tiago António. At the orchestration level, it uses Kubernetes with Calico as the networking plugin, providing networking for the containers and pods and Rook with Ceph for the persistent block and file storage. To allow external traffic into the cluster, will use HAproxy as an external ingress controller on an external node, with BIRD providing BGP peering with Calico, allowing the communication between the pods and the external node. ArgoCD was selected as the continuous delivery tool, constantly syncing with a git repository, and thus maintaining the state of the cluster manifests up to date, which allows totally abstracting developers from the infrastructure. A monitoring stack combining Prometheus, Alertmanager and Grafana allows the constant monitoring of running iv applications and cluster status, collecting metrics that can help to understand the state of operations. The administration of the cluster can be carried out in a simplified way using Portainer. The continuous implementation pipelines run on GitHub Actions, integrating software and security tests and automatically build new versions of the containers based on tag releases and publish them on DockerHub. This implementation is fully cloud native and backed only by open source software.