Automatic classification of complaints from public administration

• Autor(a) principal: Caldeira, Francisco Miguel Silva
• Data de Publicação: 2022
• Tipo de documento: Dissertação
• Idioma: eng
• Título da fonte: Repositório Científico de Acesso Aberto de Portugal (Repositórios Cientìficos)
• Texto Completo: http://hdl.handle.net/10071/26805
• topic: Text classification; Natural language processing; Machine learning; BERT; Classificação de texto; Processamento de linguagem natural

Complaint management is a problem faced by many organizations that is both vital to customer satisfaction and retention, while being highly dependent on human resources. This work attempts to tackle a part of the problem, by classifying summaries of complaints using machine learning models in order to better redirect these to the appropriate responders. To solve the aforementioned problem text mining, and more specifically natural language processing, were used alongside machine learning algorithms for automatic classification. The main challenge of this task is related with the diverse set of characteristics real world datasets have, in this case being small and highly imbalanced. This can have a big impact on the performance of the classification models. The dataset analyzed in this work suffers from both of these problems, being relatively small and having labels in different proportions the three most common labels account for around 95% the dataset. In this work, two different techniques are analyzed: multistage classification with for classifying the more common labels first and the remaining on a second step; and, generating new artificial examples for some classes via translation into other languages. The classification models explored were the following: k-NN, SVM, Naïve Bayes, boosting, and Deep Learning approaches, including transformers. Although, in general using summaries leads to better results, we also experimented with the full documents. Using the models trained with the summarized documents the classification of the full documents. Even though the results were not on par with the summarized dataset the experimented presented good results for signaling the most common label of the documents. We conclude that although, as expected, the classes with little representation are hard to classify, the techniques explored helped to boost the performance, especially in the classes with a low number of elements. SVM and Transformer-based models outperformed their peers.