If This Do That: Interpretable Machine Learning Models For High Stakes Decision-Making

Detalhes bibliográficos
Autor(a) principal: Vasconcelos, Lourenço Malheiro Serpa de
Data de Publicação: 2022
Tipo de documento: Dissertação
Idioma: eng
Título da fonte: Repositório Científico de Acesso Aberto de Portugal (Repositórios Cientìficos)
Texto Completo: http://hdl.handle.net/10362/147468
Resumo: In recent years, there has been an increasing trend in the use of black-box machine learning models in high-stakes decision making throughout different domains of society. Even though these models might be very accurate, there is a need to understand the reason behind the decisions, that is, a need to interpret the models. This need can be seen in predicting the waiting time in the Emergency Department of a hospital, where we can not simply trust a model prediction to manage a real health environment without understanding its reasons, thus, making black-box models impracticable in this scenario. Although multiple studies have aimed to explain black-box models, this can give the model even more power and be misleading which is why we should avoid it in highstakes decisions and use an interpretable model instead. This thesis demonstrates that interpretable machine learning models can be as good or even better than state of the art black-box models when predicting the waiting time in the Emergency Department of a hospital. Moreover, we also propose four new rulebased methods. To achieve this, we implemented 12 machine learning models, 6 noninterpretable (black-box) methods (ARIMA, SARIMA, Prophet, LSTM, GRU and the Transformer) and 6 rule-based and interpretable methods (RuleFit, SIRUS, REN, RDLL, RDLE and RDLR), 4 of which are proposed in this work (the REN, RDLL, RDLE and RDLR). The results obtained revealed that although some black-box models can achieve very good predictions of the waiting times in the emergency department, in some scenarios, interpretable models can outperform them.
id RCAP_aaeefbdb143a8abadc0253d79979aeec
oai_identifier_str oai:run.unl.pt:10362/147468
network_acronym_str RCAP
network_name_str Repositório Científico de Acesso Aberto de Portugal (Repositórios Cientìficos)
repository_id_str 7160
spelling If This Do That: Interpretable Machine Learning Models For High Stakes Decision-MakingInterpretabilityMachine LearningTime SeriesInterpretable Machine LearningRule-based AlgorithmsEmergency Department Waiting TimesDomínio/Área Científica::Engenharia e Tecnologia::Engenharia Eletrotécnica, Eletrónica e InformáticaIn recent years, there has been an increasing trend in the use of black-box machine learning models in high-stakes decision making throughout different domains of society. Even though these models might be very accurate, there is a need to understand the reason behind the decisions, that is, a need to interpret the models. This need can be seen in predicting the waiting time in the Emergency Department of a hospital, where we can not simply trust a model prediction to manage a real health environment without understanding its reasons, thus, making black-box models impracticable in this scenario. Although multiple studies have aimed to explain black-box models, this can give the model even more power and be misleading which is why we should avoid it in highstakes decisions and use an interpretable model instead. This thesis demonstrates that interpretable machine learning models can be as good or even better than state of the art black-box models when predicting the waiting time in the Emergency Department of a hospital. Moreover, we also propose four new rulebased methods. To achieve this, we implemented 12 machine learning models, 6 noninterpretable (black-box) methods (ARIMA, SARIMA, Prophet, LSTM, GRU and the Transformer) and 6 rule-based and interpretable methods (RuleFit, SIRUS, REN, RDLL, RDLE and RDLR), 4 of which are proposed in this work (the REN, RDLL, RDLE and RDLR). The results obtained revealed that although some black-box models can achieve very good predictions of the waiting times in the emergency department, in some scenarios, interpretable models can outperform them.Nos últimos anos, tem havido uma tendência crescente na utilização de modelos blackbox de machine learning na tomada de decisões de alto risco em diferentes domínios da sociedade. Embora estes modelos possam ser muito precisos, existe a necessidade de compreender a razão por detrás das decisões, isto é, existe uma necessidade de interpretar os modelos. Esta necessidade pode ser vista na previsão do tempo de espera no Departamento de Emergência de um hospital, onde não podemos simplesmente confiar numa previsão de um modelo para gerir um ambiente de saúde real sem compreender a sua razão, tornando assim os modelos black-box impraticáveis neste cenário. Apesar de terem sido desenvolvidos vários estudos com o objetivo de explicar os modelos black-box, isto pode dar-lhes ainda mais poder e ser enganador. Por estas razões devemos optar por usar um modelo interpretável em vez de modelos black-box. Esta tese demonstra que modelos de aprendizagem automática interpretáveis podem ser tão bons ou até mesmo melhores do que os modelos não interpretáveis considerados state-of-the-art ao prever o tempo de espera no departamento de emergência de um hospital. Além disso, esta tese propõe quatro modelos de aprendizagem automática novos baseados em regras. De forma a atingir estes objetivos, implementámos 12 modelos de aprendizegem automática, 6 dos quais não interpretáveis (black-box) (ARIMA, SARIMA, Prophet, LSTM, GRU e o Transformer) e 6 modelos interpretáveis baseados em regras (RuleFit, SIRUS, REN, RDLL, RDLE e RDLR ), 4 dos quais foram desenvolvidos e propostos nesta dissertação (REN, RDLL, RDLE e RDLR). Os resultados obtidos revelaram que, embora alguns modelos black-box consigam fazer previsões muito boas dos tempos de espera no serviço de urgência, em alguns cenários, os modelos interpretáveis são capazes de os superar.Soares, CláudiaLeite, JoãoGonçalves, RicardoKnorr, MatthiasRUNVasconcelos, Lourenço Malheiro Serpa de2023-01-13T15:40:17Z2022-112022-11-01T00:00:00Zinfo:eu-repo/semantics/publishedVersioninfo:eu-repo/semantics/masterThesisapplication/pdfhttp://hdl.handle.net/10362/147468enginfo:eu-repo/semantics/openAccessreponame:Repositório Científico de Acesso Aberto de Portugal (Repositórios Cientìficos)instname:Agência para a Sociedade do Conhecimento (UMIC) - FCT - Sociedade da Informaçãoinstacron:RCAAP2024-03-11T05:28:37Zoai:run.unl.pt:10362/147468Portal AgregadorONGhttps://www.rcaap.pt/oai/openaireopendoar:71602024-03-20T03:52:56.960353Repositório Científico de Acesso Aberto de Portugal (Repositórios Cientìficos) - Agência para a Sociedade do Conhecimento (UMIC) - FCT - Sociedade da Informaçãofalse
dc.title.none.fl_str_mv If This Do That: Interpretable Machine Learning Models For High Stakes Decision-Making
title If This Do That: Interpretable Machine Learning Models For High Stakes Decision-Making
spellingShingle If This Do That: Interpretable Machine Learning Models For High Stakes Decision-Making
Vasconcelos, Lourenço Malheiro Serpa de
Interpretability
Machine Learning
Time Series
Interpretable Machine Learning
Rule-based Algorithms
Emergency Department Waiting Times
Domínio/Área Científica::Engenharia e Tecnologia::Engenharia Eletrotécnica, Eletrónica e Informática
title_short If This Do That: Interpretable Machine Learning Models For High Stakes Decision-Making
title_full If This Do That: Interpretable Machine Learning Models For High Stakes Decision-Making
title_fullStr If This Do That: Interpretable Machine Learning Models For High Stakes Decision-Making
title_full_unstemmed If This Do That: Interpretable Machine Learning Models For High Stakes Decision-Making
title_sort If This Do That: Interpretable Machine Learning Models For High Stakes Decision-Making
author Vasconcelos, Lourenço Malheiro Serpa de
author_facet Vasconcelos, Lourenço Malheiro Serpa de
author_role author
dc.contributor.none.fl_str_mv Soares, Cláudia
Leite, João
Gonçalves, Ricardo
Knorr, Matthias
RUN
dc.contributor.author.fl_str_mv Vasconcelos, Lourenço Malheiro Serpa de
dc.subject.por.fl_str_mv Interpretability
Machine Learning
Time Series
Interpretable Machine Learning
Rule-based Algorithms
Emergency Department Waiting Times
Domínio/Área Científica::Engenharia e Tecnologia::Engenharia Eletrotécnica, Eletrónica e Informática
topic Interpretability
Machine Learning
Time Series
Interpretable Machine Learning
Rule-based Algorithms
Emergency Department Waiting Times
Domínio/Área Científica::Engenharia e Tecnologia::Engenharia Eletrotécnica, Eletrónica e Informática
description In recent years, there has been an increasing trend in the use of black-box machine learning models in high-stakes decision making throughout different domains of society. Even though these models might be very accurate, there is a need to understand the reason behind the decisions, that is, a need to interpret the models. This need can be seen in predicting the waiting time in the Emergency Department of a hospital, where we can not simply trust a model prediction to manage a real health environment without understanding its reasons, thus, making black-box models impracticable in this scenario. Although multiple studies have aimed to explain black-box models, this can give the model even more power and be misleading which is why we should avoid it in highstakes decisions and use an interpretable model instead. This thesis demonstrates that interpretable machine learning models can be as good or even better than state of the art black-box models when predicting the waiting time in the Emergency Department of a hospital. Moreover, we also propose four new rulebased methods. To achieve this, we implemented 12 machine learning models, 6 noninterpretable (black-box) methods (ARIMA, SARIMA, Prophet, LSTM, GRU and the Transformer) and 6 rule-based and interpretable methods (RuleFit, SIRUS, REN, RDLL, RDLE and RDLR), 4 of which are proposed in this work (the REN, RDLL, RDLE and RDLR). The results obtained revealed that although some black-box models can achieve very good predictions of the waiting times in the emergency department, in some scenarios, interpretable models can outperform them.
publishDate 2022
dc.date.none.fl_str_mv 2022-11
2022-11-01T00:00:00Z
2023-01-13T15:40:17Z
dc.type.status.fl_str_mv info:eu-repo/semantics/publishedVersion
dc.type.driver.fl_str_mv info:eu-repo/semantics/masterThesis
format masterThesis
status_str publishedVersion
dc.identifier.uri.fl_str_mv http://hdl.handle.net/10362/147468
url http://hdl.handle.net/10362/147468
dc.language.iso.fl_str_mv eng
language eng
dc.rights.driver.fl_str_mv info:eu-repo/semantics/openAccess
eu_rights_str_mv openAccess
dc.format.none.fl_str_mv application/pdf
dc.source.none.fl_str_mv reponame:Repositório Científico de Acesso Aberto de Portugal (Repositórios Cientìficos)
instname:Agência para a Sociedade do Conhecimento (UMIC) - FCT - Sociedade da Informação
instacron:RCAAP
instname_str Agência para a Sociedade do Conhecimento (UMIC) - FCT - Sociedade da Informação
instacron_str RCAAP
institution RCAAP
reponame_str Repositório Científico de Acesso Aberto de Portugal (Repositórios Cientìficos)
collection Repositório Científico de Acesso Aberto de Portugal (Repositórios Cientìficos)
repository.name.fl_str_mv Repositório Científico de Acesso Aberto de Portugal (Repositórios Cientìficos) - Agência para a Sociedade do Conhecimento (UMIC) - FCT - Sociedade da Informação
repository.mail.fl_str_mv
_version_ 1799138120824782848