Using a fairness-utility trade-off metric to systematically benchmark non-generative fair adversarial learning strategies
Autor(a) principal: | |
---|---|
Data de Publicação: | 2022 |
Tipo de documento: | Dissertação |
Idioma: | por |
Título da fonte: | Biblioteca Digital de Teses e Dissertações da UFPB |
Texto Completo: | https://repositorio.ufpb.br/jspui/handle/123456789/26323 |
Resumo: | Artificial intelligence systems for decision-making have become increasingly popular in several areas. However, it is possible to identify biased decisions in many applications, which have become a concern for the computer science, artificial intelligence, and law communities. Therefore, researches are proposing solutions to mitigate bias and discrimination in decision-makers. Some explored strategies are based on generative adversarial networks to generate fair data. Others are based on adversarial learning to achieve fairness in machine learning by encoding fairness constraints through an adversarial model. Moreover, it is usual for each proposal to assess its model with a specific metric, making the comparison of current approaches a complex task. Therefore, this work proposes a benchmark procedure with a systematical method to assess the fair machine learning models. In this sense, we define the FU-score metric to evaluate the utility-fairness trade-off, the utility and fairness metrics to compose this assessment, the used dataset and applied data preparation, and the statistical test. We also performed this benchmark evaluation for the non-generative adversarial models, analyzing the literature models from the same metric perspective. This assessment could not indicate a single model which better performs for all datasets. However, we built an understanding of how each model performs on each dataset which statistical confidence. |
id |
UFPB_f71e15f7a5a3a0dbf03860796c19898a |
---|---|
oai_identifier_str |
oai:repositorio.ufpb.br:123456789/26323 |
network_acronym_str |
UFPB |
network_name_str |
Biblioteca Digital de Teses e Dissertações da UFPB |
repository_id_str |
|
spelling |
Using a fairness-utility trade-off metric to systematically benchmark non-generative fair adversarial learning strategiesAprendizado adversárioAprendizado de máquinaBenchmarkTrade-offAdversarial LearningMachine LearningJustiçaFairnessCNPQ::CIENCIAS EXATAS E DA TERRA::CIENCIA DA COMPUTACAOArtificial intelligence systems for decision-making have become increasingly popular in several areas. However, it is possible to identify biased decisions in many applications, which have become a concern for the computer science, artificial intelligence, and law communities. Therefore, researches are proposing solutions to mitigate bias and discrimination in decision-makers. Some explored strategies are based on generative adversarial networks to generate fair data. Others are based on adversarial learning to achieve fairness in machine learning by encoding fairness constraints through an adversarial model. Moreover, it is usual for each proposal to assess its model with a specific metric, making the comparison of current approaches a complex task. Therefore, this work proposes a benchmark procedure with a systematical method to assess the fair machine learning models. In this sense, we define the FU-score metric to evaluate the utility-fairness trade-off, the utility and fairness metrics to compose this assessment, the used dataset and applied data preparation, and the statistical test. We also performed this benchmark evaluation for the non-generative adversarial models, analyzing the literature models from the same metric perspective. This assessment could not indicate a single model which better performs for all datasets. However, we built an understanding of how each model performs on each dataset which statistical confidence.Coordenação de Aperfeiçoamento de Pessoal de Nível Superior - CAPESOs sistemas de inteligência artificial para tomada de decisão têm se tornado cada vez mais populares em diversas áreas. Entretanto, é possível identificar decisões enviesadas em muitas aplicações, que se tornaram uma preocupação para as comunidades de ciência da computação, inteligência artificial e direito. Portanto, as pesquisas vêm propondo soluções para mitigar o viés e a discriminação presente nos tomadores de decisão. Algumas estratégias exploradas são baseadas em redes adversários generativas para gerar dados justos. Outros são baseados no aprendizado adversário para alcançar a justiça no aprendizado de máquina codificando restrições de justiça por meio de um componente adversário. Além disso, é comum que cada proposta avalie seu modelo com uma métrica específica, tornando a comparação das abordagens atuais uma tarefa complexa. Portanto, este trabalho propõe um procedimento de benchmark com um método sistemático para avaliar os modelos de aprendizado de máquina justo. Nesse sentido, definimos a métrica FU-score para avaliar o trade-off de utilidade e justiça, as métricas de utilidade e justiça para compor essa avaliação, o conjunto de dados utilizado e a preparação aplicada e o teste estatístico. Também realizamos esta avaliação de benchmark para os modelos adversários não generativos, analisando os modelos da literatura sob a mesma métrica. Essa avaliação não pôde apontar um único modelo com melhor desempenho para todos os conjuntos de dados. No entanto, construímos um entendimento de como cada modelo funciona em cada conjunto de dados com confiança estatística.Universidade Federal da ParaíbaBrasilInformáticaPrograma de Pós-Graduação em InformáticaUFPBSiebra, Clauirton de Albuquerquehttp://lattes.cnpq.br/7707799028683443Ricarte, Danielle Rousy Diashttp://lattes.cnpq.br/4603035287575568Lima, Luiz Fernando Fonsêca Pinheiro de2023-02-23T16:15:13Z2023-01-112023-02-23T16:15:13Z2022-08-26info:eu-repo/semantics/publishedVersioninfo:eu-repo/semantics/masterThesishttps://repositorio.ufpb.br/jspui/handle/123456789/26323porAttribution-NoDerivs 3.0 Brazilhttp://creativecommons.org/licenses/by-nd/3.0/br/info:eu-repo/semantics/openAccessreponame:Biblioteca Digital de Teses e Dissertações da UFPBinstname:Universidade Federal da Paraíba (UFPB)instacron:UFPB2023-05-22T12:53:41Zoai:repositorio.ufpb.br:123456789/26323Biblioteca Digital de Teses e Dissertaçõeshttps://repositorio.ufpb.br/PUBhttp://tede.biblioteca.ufpb.br:8080/oai/requestdiretoria@ufpb.br|| diretoria@ufpb.bropendoar:2023-05-22T12:53:41Biblioteca Digital de Teses e Dissertações da UFPB - Universidade Federal da Paraíba (UFPB)false |
dc.title.none.fl_str_mv |
Using a fairness-utility trade-off metric to systematically benchmark non-generative fair adversarial learning strategies |
title |
Using a fairness-utility trade-off metric to systematically benchmark non-generative fair adversarial learning strategies |
spellingShingle |
Using a fairness-utility trade-off metric to systematically benchmark non-generative fair adversarial learning strategies Lima, Luiz Fernando Fonsêca Pinheiro de Aprendizado adversário Aprendizado de máquina Benchmark Trade-off Adversarial Learning Machine Learning Justiça Fairness CNPQ::CIENCIAS EXATAS E DA TERRA::CIENCIA DA COMPUTACAO |
title_short |
Using a fairness-utility trade-off metric to systematically benchmark non-generative fair adversarial learning strategies |
title_full |
Using a fairness-utility trade-off metric to systematically benchmark non-generative fair adversarial learning strategies |
title_fullStr |
Using a fairness-utility trade-off metric to systematically benchmark non-generative fair adversarial learning strategies |
title_full_unstemmed |
Using a fairness-utility trade-off metric to systematically benchmark non-generative fair adversarial learning strategies |
title_sort |
Using a fairness-utility trade-off metric to systematically benchmark non-generative fair adversarial learning strategies |
author |
Lima, Luiz Fernando Fonsêca Pinheiro de |
author_facet |
Lima, Luiz Fernando Fonsêca Pinheiro de |
author_role |
author |
dc.contributor.none.fl_str_mv |
Siebra, Clauirton de Albuquerque http://lattes.cnpq.br/7707799028683443 Ricarte, Danielle Rousy Dias http://lattes.cnpq.br/4603035287575568 |
dc.contributor.author.fl_str_mv |
Lima, Luiz Fernando Fonsêca Pinheiro de |
dc.subject.por.fl_str_mv |
Aprendizado adversário Aprendizado de máquina Benchmark Trade-off Adversarial Learning Machine Learning Justiça Fairness CNPQ::CIENCIAS EXATAS E DA TERRA::CIENCIA DA COMPUTACAO |
topic |
Aprendizado adversário Aprendizado de máquina Benchmark Trade-off Adversarial Learning Machine Learning Justiça Fairness CNPQ::CIENCIAS EXATAS E DA TERRA::CIENCIA DA COMPUTACAO |
description |
Artificial intelligence systems for decision-making have become increasingly popular in several areas. However, it is possible to identify biased decisions in many applications, which have become a concern for the computer science, artificial intelligence, and law communities. Therefore, researches are proposing solutions to mitigate bias and discrimination in decision-makers. Some explored strategies are based on generative adversarial networks to generate fair data. Others are based on adversarial learning to achieve fairness in machine learning by encoding fairness constraints through an adversarial model. Moreover, it is usual for each proposal to assess its model with a specific metric, making the comparison of current approaches a complex task. Therefore, this work proposes a benchmark procedure with a systematical method to assess the fair machine learning models. In this sense, we define the FU-score metric to evaluate the utility-fairness trade-off, the utility and fairness metrics to compose this assessment, the used dataset and applied data preparation, and the statistical test. We also performed this benchmark evaluation for the non-generative adversarial models, analyzing the literature models from the same metric perspective. This assessment could not indicate a single model which better performs for all datasets. However, we built an understanding of how each model performs on each dataset which statistical confidence. |
publishDate |
2022 |
dc.date.none.fl_str_mv |
2022-08-26 2023-02-23T16:15:13Z 2023-01-11 2023-02-23T16:15:13Z |
dc.type.status.fl_str_mv |
info:eu-repo/semantics/publishedVersion |
dc.type.driver.fl_str_mv |
info:eu-repo/semantics/masterThesis |
format |
masterThesis |
status_str |
publishedVersion |
dc.identifier.uri.fl_str_mv |
https://repositorio.ufpb.br/jspui/handle/123456789/26323 |
url |
https://repositorio.ufpb.br/jspui/handle/123456789/26323 |
dc.language.iso.fl_str_mv |
por |
language |
por |
dc.rights.driver.fl_str_mv |
Attribution-NoDerivs 3.0 Brazil http://creativecommons.org/licenses/by-nd/3.0/br/ info:eu-repo/semantics/openAccess |
rights_invalid_str_mv |
Attribution-NoDerivs 3.0 Brazil http://creativecommons.org/licenses/by-nd/3.0/br/ |
eu_rights_str_mv |
openAccess |
dc.publisher.none.fl_str_mv |
Universidade Federal da Paraíba Brasil Informática Programa de Pós-Graduação em Informática UFPB |
publisher.none.fl_str_mv |
Universidade Federal da Paraíba Brasil Informática Programa de Pós-Graduação em Informática UFPB |
dc.source.none.fl_str_mv |
reponame:Biblioteca Digital de Teses e Dissertações da UFPB instname:Universidade Federal da Paraíba (UFPB) instacron:UFPB |
instname_str |
Universidade Federal da Paraíba (UFPB) |
instacron_str |
UFPB |
institution |
UFPB |
reponame_str |
Biblioteca Digital de Teses e Dissertações da UFPB |
collection |
Biblioteca Digital de Teses e Dissertações da UFPB |
repository.name.fl_str_mv |
Biblioteca Digital de Teses e Dissertações da UFPB - Universidade Federal da Paraíba (UFPB) |
repository.mail.fl_str_mv |
diretoria@ufpb.br|| diretoria@ufpb.br |
_version_ |
1801843006233378816 |