Ensemble learning through Rashomon sets

• Autor(a) principal: Gianlucca Lodron Zuin
• Data de Publicação: 2023
• Tipo de documento: Tese
• Idioma: eng
• Título da fonte: Repositório Institucional da UFMG
• Texto Completo: http://hdl.handle.net/1843/52748; https://orcid.org/0000-0002-0429-3280
• topic: Rashomon Effect; Ensemble Learning; Data Drift

Creating models from previous observations and ensuring effectiveness on new data is the essence of machine learning. Therefore, estimating the generalization error of a trained model is a crucial step. Despite the existence of many capacity measures that approximate the generalization power of trained models, it is still challenging to select models that generalize to future data. In this work, we investigate how models perform in datasets that have different underlying generator functions but constitute co-related tasks. The key motivation is to study the Rashomon Effect, which appears whenever the learning problem admits a set of models that all perform roughly equally well. Many real-world problems are characterized by multiple local structures in the data space and, as a result, the corresponding learning problem has a non-convex error surface with no obvious global minimum, thus implying a multiplicity of performant models, each of them providing a different explanation, which literature suggests to being subject to the Rashomon Effect. Through an empirical study across different datasets, we devise a strategy focusing primarily on model explainability (i.e., feature importance). Our approach to deal with the Rashomon Effect is to stratify, during training, models into groups that are either coherent or contrasting. From these Rashomon groups, we can select models that increase the robustness of the production responses along with means to gauge data drift. We present performance gains on most of the evaluated scenarios by locating these models and creating an ensemble guaranteeing that each constituent covers an independent solution sub-space. We validate our approach by performing a series of experiments in both closed and open-source benchmark suites and give insights into the possible applications by analyzing real-world case studies in which our framework was employed with success. Not only does our approach prove to be superior to state-of-the-art tree-based ensembling techniques, with gains in AUC of up to .20+, but the constituent models are highly explainable and allow for the integration of humans into the decision-making pipeline, thus empowering them.