Domain generalization, invariance and the Time Robust Forest

• Autor(a) principal: Santos, Luis Gustavo Moneda dos
• Data de Publicação: 2021
• Tipo de documento: Dissertação
• Idioma: eng
• Título da fonte: Biblioteca Digital de Teses e Dissertações da USP
• Texto Completo: https://www.teses.usp.br/teses/disponiveis/45/45134/tde-16112021-193305/
• topic: Causal invariance; Domain generalization; Generalização de domínio; Inductive bias; Invariância causal; Viés indutivo

As time passes by, the performance of real-world predictive models degrades due to distributional shifts. Typical countermeasures, such as retraining and online learning, can be costly and difficult to implement in production, especially when business constraints and culture are accounted for. Causality-based approaches aim at identifying invariant mechanisms from data, thus leading to more robust predictors at the possible expense of a decrease in short-term performance. However, most such approaches scale poorly to high dimensions or require extra knowledge such as segmentation of the data in representative environments. In this work, we review the literature on the limitations of Machine Learning in real settings, with a focus on approaches that use causality concepts to improve generalization. Motivated by the shortcomings discussed above, we develop Time Robust Forests (TRF), a new algorithm for inducing decision trees with an inductive bias towards learning time-invariant rules. The algorithm\'s main innovation is to replace the usual information-gain split criterion (or similar) with a new criterion that examines the imbalance among classes induced by the split through time. Experiments with real data show that our approach can improve long-term generalization, thus offering an interesting alternative for dynamical classification problems.