Assessing how developers review tests on Github
Autor(a) principal: | |
---|---|
Data de Publicação: | 2022 |
Tipo de documento: | Dissertação |
Idioma: | eng |
Título da fonte: | Repositório Institucional da UFMG |
Texto Completo: | http://hdl.handle.net/1843/43919 |
Resumo: | In modern software development, developers are expected to provide proper tests to cover their code changes. However, code contributions are not always attached to good tests. This way, reviewers may request test changes to the contributions. In practice, it is not clear what information is available to guide contributors in implementing the requested test methods during the code review. Therefore, better understanding test review practices would be important to guide both contributors and reviewers. In this dissertation, we propose two studies. First, we provide an empirical study to assess test reviews on GitHub. We find 11,836 test reviews in 5,421 open-source projects, suggesting that this is a common practice. We also detect eight broad categories of recommendations in test reviews: test scope, tool support, test scenarios, goal/purpose, refactoring, bad practices, fixtures, and miscellaneous. Lastly, we find that test reviews with more recommendations are more likely to be solved. In our second study, we propose a tool to assess the quality of individual test methods by relying on mutation testing. This tool extends a state-of-the-art mutation testing framework to analyze test methods and report mutation results at the method level. Finally, based on our results, we discussed implications for researchers and practitioners. |
id |
UFMG_9aa75625ef13ab43359d277d0a44cfd1 |
---|---|
oai_identifier_str |
oai:repositorio.ufmg.br:1843/43919 |
network_acronym_str |
UFMG |
network_name_str |
Repositório Institucional da UFMG |
repository_id_str |
|
spelling |
Andre Hora Cavalcantehttp://lattes.cnpq.br/4957418183504876Marco Tulio de Oliveira ValenteEduardo Magno Lages Figueiredohttp://lattes.cnpq.br/1831616783896700Victor Guerra Veloso2022-08-03T15:44:46Z2022-08-03T15:44:46Z2022-07-13http://hdl.handle.net/1843/439190000-0002-2127-4380In modern software development, developers are expected to provide proper tests to cover their code changes. However, code contributions are not always attached to good tests. This way, reviewers may request test changes to the contributions. In practice, it is not clear what information is available to guide contributors in implementing the requested test methods during the code review. Therefore, better understanding test review practices would be important to guide both contributors and reviewers. In this dissertation, we propose two studies. First, we provide an empirical study to assess test reviews on GitHub. We find 11,836 test reviews in 5,421 open-source projects, suggesting that this is a common practice. We also detect eight broad categories of recommendations in test reviews: test scope, tool support, test scenarios, goal/purpose, refactoring, bad practices, fixtures, and miscellaneous. Lastly, we find that test reviews with more recommendations are more likely to be solved. In our second study, we propose a tool to assess the quality of individual test methods by relying on mutation testing. This tool extends a state-of-the-art mutation testing framework to analyze test methods and report mutation results at the method level. Finally, based on our results, we discussed implications for researchers and practitioners.No desenvolvimento de software moderno, espera-se que desenvolvedores forneçam testes adequados para cobrir suas alterações de código. No entanto, contribuições nem sempre incluem bons testes. Dessa forma, os revisores podem solicitar melhoria de testes nas contribuições. Na prática, não está claro quais informações estão disponíveis para orientar os colaboradores na implementação dos métodos de testes solicitados durante a revisão de código. Portanto, entender melhor as práticas de revisão de testes seria importante para orientar tanto os colaboradores quanto os revisores. Nesta dissertação, propomos dois estudos. Primeiro, fornecemos um estudo empírico para avaliar as revisões de testes no GitHub. Encontramos 11.836 avaliações de teste em 5.421 projetos de código aberto, sugerindo que essa é uma prática comum. Também detectamos oito amplas categorias de recomendações em revisões de testes: escopo do teste, suporte a ferramentas, cenários de teste, objetivo/propósito, refatorações, más práticas, fixtures e miscellaneous. Por fim, descobrimos que as revisões de testes com mais recomendações são mais propensas a serem resolvidas. Em nosso segundo estudo, propomos uma ferramenta para avaliar a qualidade de métodos de testes individuais por meio de testes de mutação. Esta ferramenta estende uma framework de testes de mutação estado-da-arte para analisar métodos de teste e relatar resultados de mutação a nível de método. Finalmente, com base em nossos resultados, discutimos implicações para pesquisadores e profissionais.CNPq - Conselho Nacional de Desenvolvimento Científico e TecnológicoengUniversidade Federal de Minas GeraisPrograma de Pós-Graduação em Ciência da ComputaçãoUFMGBrasilICX - DEPARTAMENTO DE CIÊNCIA DA COMPUTAÇÃOhttp://creativecommons.org/licenses/by-nc/3.0/pt/info:eu-repo/semantics/openAccessComputação – TesesEngenharia de software – TesesSoftware – Testes - TesesSoftware – Controle de qualidade – Teses.Software testingCode ReviewSoftware repository miningSoftware maintenanceMutation TestingCode qualityAssessing how developers review tests on GithubAvaliando comodesenvolvedores revisamtestes no GitHubinfo:eu-repo/semantics/publishedVersioninfo:eu-repo/semantics/masterThesisreponame:Repositório Institucional da UFMGinstname:Universidade Federal de Minas Gerais (UFMG)instacron:UFMGORIGINALMaster_Thesis.pdfMaster_Thesis.pdfapplication/pdf5326425https://repositorio.ufmg.br/bitstream/1843/43919/1/Master_Thesis.pdf2592311ac536389d9143b0e27e8cf3edMD51CC-LICENSElicense_rdflicense_rdfapplication/rdf+xml; charset=utf-8920https://repositorio.ufmg.br/bitstream/1843/43919/2/license_rdf33b8016dc5c4681c1e7a582a4161162cMD52LICENSElicense.txtlicense.txttext/plain; charset=utf-82118https://repositorio.ufmg.br/bitstream/1843/43919/3/license.txtcda590c95a0b51b4d15f60c9642ca272MD531843/439192022-08-03 12:44:47.179oai:repositorio.ufmg.br:1843/43919TElDRU7Dh0EgREUgRElTVFJJQlVJw4fDg08gTsODTy1FWENMVVNJVkEgRE8gUkVQT1NJVMOTUklPIElOU1RJVFVDSU9OQUwgREEgVUZNRwoKQ29tIGEgYXByZXNlbnRhw6fDo28gZGVzdGEgbGljZW7Dp2EsIHZvY8OqIChvIGF1dG9yIChlcykgb3UgbyB0aXR1bGFyIGRvcyBkaXJlaXRvcyBkZSBhdXRvcikgY29uY2VkZSBhbyBSZXBvc2l0w7NyaW8gSW5zdGl0dWNpb25hbCBkYSBVRk1HIChSSS1VRk1HKSBvIGRpcmVpdG8gbsOjbyBleGNsdXNpdm8gZSBpcnJldm9nw6F2ZWwgZGUgcmVwcm9kdXppciBlL291IGRpc3RyaWJ1aXIgYSBzdWEgcHVibGljYcOnw6NvIChpbmNsdWluZG8gbyByZXN1bW8pIHBvciB0b2RvIG8gbXVuZG8gbm8gZm9ybWF0byBpbXByZXNzbyBlIGVsZXRyw7RuaWNvIGUgZW0gcXVhbHF1ZXIgbWVpbywgaW5jbHVpbmRvIG9zIGZvcm1hdG9zIMOhdWRpbyBvdSB2w61kZW8uCgpWb2PDqiBkZWNsYXJhIHF1ZSBjb25oZWNlIGEgcG9sw610aWNhIGRlIGNvcHlyaWdodCBkYSBlZGl0b3JhIGRvIHNldSBkb2N1bWVudG8gZSBxdWUgY29uaGVjZSBlIGFjZWl0YSBhcyBEaXJldHJpemVzIGRvIFJJLVVGTUcuCgpWb2PDqiBjb25jb3JkYSBxdWUgbyBSZXBvc2l0w7NyaW8gSW5zdGl0dWNpb25hbCBkYSBVRk1HIHBvZGUsIHNlbSBhbHRlcmFyIG8gY29udGXDumRvLCB0cmFuc3BvciBhIHN1YSBwdWJsaWNhw6fDo28gcGFyYSBxdWFscXVlciBtZWlvIG91IGZvcm1hdG8gcGFyYSBmaW5zIGRlIHByZXNlcnZhw6fDo28uCgpWb2PDqiB0YW1iw6ltIGNvbmNvcmRhIHF1ZSBvIFJlcG9zaXTDs3JpbyBJbnN0aXR1Y2lvbmFsIGRhIFVGTUcgcG9kZSBtYW50ZXIgbWFpcyBkZSB1bWEgY8OzcGlhIGRlIHN1YSBwdWJsaWNhw6fDo28gcGFyYSBmaW5zIGRlIHNlZ3VyYW7Dp2EsIGJhY2stdXAgZSBwcmVzZXJ2YcOnw6NvLgoKVm9jw6ogZGVjbGFyYSBxdWUgYSBzdWEgcHVibGljYcOnw6NvIMOpIG9yaWdpbmFsIGUgcXVlIHZvY8OqIHRlbSBvIHBvZGVyIGRlIGNvbmNlZGVyIG9zIGRpcmVpdG9zIGNvbnRpZG9zIG5lc3RhIGxpY2Vuw6dhLiBWb2PDqiB0YW1iw6ltIGRlY2xhcmEgcXVlIG8gZGVww7NzaXRvIGRlIHN1YSBwdWJsaWNhw6fDo28gbsOjbywgcXVlIHNlamEgZGUgc2V1IGNvbmhlY2ltZW50bywgaW5mcmluZ2UgZGlyZWl0b3MgYXV0b3JhaXMgZGUgbmluZ3XDqW0uCgpDYXNvIGEgc3VhIHB1YmxpY2HDp8OjbyBjb250ZW5oYSBtYXRlcmlhbCBxdWUgdm9jw6ogbsOjbyBwb3NzdWkgYSB0aXR1bGFyaWRhZGUgZG9zIGRpcmVpdG9zIGF1dG9yYWlzLCB2b2PDqiBkZWNsYXJhIHF1ZSBvYnRldmUgYSBwZXJtaXNzw6NvIGlycmVzdHJpdGEgZG8gZGV0ZW50b3IgZG9zIGRpcmVpdG9zIGF1dG9yYWlzIHBhcmEgY29uY2VkZXIgYW8gUmVwb3NpdMOzcmlvIEluc3RpdHVjaW9uYWwgZGEgVUZNRyBvcyBkaXJlaXRvcyBhcHJlc2VudGFkb3MgbmVzdGEgbGljZW7Dp2EsIGUgcXVlIGVzc2UgbWF0ZXJpYWwgZGUgcHJvcHJpZWRhZGUgZGUgdGVyY2Vpcm9zIGVzdMOhIGNsYXJhbWVudGUgaWRlbnRpZmljYWRvIGUgcmVjb25oZWNpZG8gbm8gdGV4dG8gb3Ugbm8gY29udGXDumRvIGRhIHB1YmxpY2HDp8OjbyBvcmEgZGVwb3NpdGFkYS4KCkNBU08gQSBQVUJMSUNBw4fDg08gT1JBIERFUE9TSVRBREEgVEVOSEEgU0lETyBSRVNVTFRBRE8gREUgVU0gUEFUUk9Dw41OSU8gT1UgQVBPSU8gREUgVU1BIEFHw4pOQ0lBIERFIEZPTUVOVE8gT1UgT1VUUk8gT1JHQU5JU01PLCBWT0PDiiBERUNMQVJBIFFVRSBSRVNQRUlUT1UgVE9ET1MgRSBRVUFJU1FVRVIgRElSRUlUT1MgREUgUkVWSVPDg08gQ09NTyBUQU1Cw4lNIEFTIERFTUFJUyBPQlJJR0HDh8OVRVMgRVhJR0lEQVMgUE9SIENPTlRSQVRPIE9VIEFDT1JETy4KCk8gUmVwb3NpdMOzcmlvIEluc3RpdHVjaW9uYWwgZGEgVUZNRyBzZSBjb21wcm9tZXRlIGEgaWRlbnRpZmljYXIgY2xhcmFtZW50ZSBvIHNldSBub21lKHMpIG91IG8ocykgbm9tZXMocykgZG8ocykgZGV0ZW50b3IoZXMpIGRvcyBkaXJlaXRvcyBhdXRvcmFpcyBkYSBwdWJsaWNhw6fDo28sIGUgbsOjbyBmYXLDoSBxdWFscXVlciBhbHRlcmHDp8OjbywgYWzDqW0gZGFxdWVsYXMgY29uY2VkaWRhcyBwb3IgZXN0YSBsaWNlbsOnYS4KRepositório de PublicaçõesPUBhttps://repositorio.ufmg.br/oaiopendoar:2022-08-03T15:44:47Repositório Institucional da UFMG - Universidade Federal de Minas Gerais (UFMG)false |
dc.title.pt_BR.fl_str_mv |
Assessing how developers review tests on Github |
dc.title.alternative.pt_BR.fl_str_mv |
Avaliando comodesenvolvedores revisamtestes no GitHub |
title |
Assessing how developers review tests on Github |
spellingShingle |
Assessing how developers review tests on Github Victor Guerra Veloso Software testing Code Review Software repository mining Software maintenance Mutation Testing Code quality Computação – Teses Engenharia de software – Teses Software – Testes - Teses Software – Controle de qualidade – Teses. |
title_short |
Assessing how developers review tests on Github |
title_full |
Assessing how developers review tests on Github |
title_fullStr |
Assessing how developers review tests on Github |
title_full_unstemmed |
Assessing how developers review tests on Github |
title_sort |
Assessing how developers review tests on Github |
author |
Victor Guerra Veloso |
author_facet |
Victor Guerra Veloso |
author_role |
author |
dc.contributor.advisor1.fl_str_mv |
Andre Hora Cavalcante |
dc.contributor.advisor1Lattes.fl_str_mv |
http://lattes.cnpq.br/4957418183504876 |
dc.contributor.referee1.fl_str_mv |
Marco Tulio de Oliveira Valente |
dc.contributor.referee2.fl_str_mv |
Eduardo Magno Lages Figueiredo |
dc.contributor.authorLattes.fl_str_mv |
http://lattes.cnpq.br/1831616783896700 |
dc.contributor.author.fl_str_mv |
Victor Guerra Veloso |
contributor_str_mv |
Andre Hora Cavalcante Marco Tulio de Oliveira Valente Eduardo Magno Lages Figueiredo |
dc.subject.por.fl_str_mv |
Software testing Code Review Software repository mining Software maintenance Mutation Testing Code quality |
topic |
Software testing Code Review Software repository mining Software maintenance Mutation Testing Code quality Computação – Teses Engenharia de software – Teses Software – Testes - Teses Software – Controle de qualidade – Teses. |
dc.subject.other.pt_BR.fl_str_mv |
Computação – Teses Engenharia de software – Teses Software – Testes - Teses Software – Controle de qualidade – Teses. |
description |
In modern software development, developers are expected to provide proper tests to cover their code changes. However, code contributions are not always attached to good tests. This way, reviewers may request test changes to the contributions. In practice, it is not clear what information is available to guide contributors in implementing the requested test methods during the code review. Therefore, better understanding test review practices would be important to guide both contributors and reviewers. In this dissertation, we propose two studies. First, we provide an empirical study to assess test reviews on GitHub. We find 11,836 test reviews in 5,421 open-source projects, suggesting that this is a common practice. We also detect eight broad categories of recommendations in test reviews: test scope, tool support, test scenarios, goal/purpose, refactoring, bad practices, fixtures, and miscellaneous. Lastly, we find that test reviews with more recommendations are more likely to be solved. In our second study, we propose a tool to assess the quality of individual test methods by relying on mutation testing. This tool extends a state-of-the-art mutation testing framework to analyze test methods and report mutation results at the method level. Finally, based on our results, we discussed implications for researchers and practitioners. |
publishDate |
2022 |
dc.date.accessioned.fl_str_mv |
2022-08-03T15:44:46Z |
dc.date.available.fl_str_mv |
2022-08-03T15:44:46Z |
dc.date.issued.fl_str_mv |
2022-07-13 |
dc.type.status.fl_str_mv |
info:eu-repo/semantics/publishedVersion |
dc.type.driver.fl_str_mv |
info:eu-repo/semantics/masterThesis |
format |
masterThesis |
status_str |
publishedVersion |
dc.identifier.uri.fl_str_mv |
http://hdl.handle.net/1843/43919 |
dc.identifier.orcid.pt_BR.fl_str_mv |
0000-0002-2127-4380 |
url |
http://hdl.handle.net/1843/43919 |
identifier_str_mv |
0000-0002-2127-4380 |
dc.language.iso.fl_str_mv |
eng |
language |
eng |
dc.rights.driver.fl_str_mv |
http://creativecommons.org/licenses/by-nc/3.0/pt/ info:eu-repo/semantics/openAccess |
rights_invalid_str_mv |
http://creativecommons.org/licenses/by-nc/3.0/pt/ |
eu_rights_str_mv |
openAccess |
dc.publisher.none.fl_str_mv |
Universidade Federal de Minas Gerais |
dc.publisher.program.fl_str_mv |
Programa de Pós-Graduação em Ciência da Computação |
dc.publisher.initials.fl_str_mv |
UFMG |
dc.publisher.country.fl_str_mv |
Brasil |
dc.publisher.department.fl_str_mv |
ICX - DEPARTAMENTO DE CIÊNCIA DA COMPUTAÇÃO |
publisher.none.fl_str_mv |
Universidade Federal de Minas Gerais |
dc.source.none.fl_str_mv |
reponame:Repositório Institucional da UFMG instname:Universidade Federal de Minas Gerais (UFMG) instacron:UFMG |
instname_str |
Universidade Federal de Minas Gerais (UFMG) |
instacron_str |
UFMG |
institution |
UFMG |
reponame_str |
Repositório Institucional da UFMG |
collection |
Repositório Institucional da UFMG |
bitstream.url.fl_str_mv |
https://repositorio.ufmg.br/bitstream/1843/43919/1/Master_Thesis.pdf https://repositorio.ufmg.br/bitstream/1843/43919/2/license_rdf https://repositorio.ufmg.br/bitstream/1843/43919/3/license.txt |
bitstream.checksum.fl_str_mv |
2592311ac536389d9143b0e27e8cf3ed 33b8016dc5c4681c1e7a582a4161162c cda590c95a0b51b4d15f60c9642ca272 |
bitstream.checksumAlgorithm.fl_str_mv |
MD5 MD5 MD5 |
repository.name.fl_str_mv |
Repositório Institucional da UFMG - Universidade Federal de Minas Gerais (UFMG) |
repository.mail.fl_str_mv |
|
_version_ |
1803589498099990528 |