Test suite parallelization in open-source projects: a study on its usage and impact

Detalhes bibliográficos
Autor(a) principal: CÂNDIDO, Jeanderson Barros
Data de Publicação: 2018
Tipo de documento: Dissertação
Idioma: eng
Título da fonte: Repositório Institucional da UFPE
Texto Completo: https://repositorio.ufpe.br/handle/123456789/30520
Resumo: Testing is a costly process but essential in the development process. Complex systems may contain long-running test suites. Dealing with high testing costs remains an important problem in Software Engineering despite being under active research for years. Test suite parallelization is an important approach to address this problem, given the popularity of multi-core processors and native suport from testing frameworks and build systems. This work reports our findings on the usage and impact of test suite parallelization in open-source projects. This study brings to light the benefits and burdens of that approach. It provides recommendations to practitioners and tool developers to speed up test execution. Considering a set of 468 popular Java projects we analyzed, we found that 24% of the projects contain costly test suites but parallelization features still seem underutilized in practice—only 19.1% of costly projects use parallelization. The main reported reason for adoption resistance was the concern to deal with concurrency issues. Results suggest that, on average, developers prefer high predictability than high performance in running tests.
id UFPE_9c0fb78a95603b38808683df122a7b3e
oai_identifier_str oai:repositorio.ufpe.br:123456789/30520
network_acronym_str UFPE
network_name_str Repositório Institucional da UFPE
repository_id_str 2221
spelling CÂNDIDO, Jeanderson Barroshttp://lattes.cnpq.br/2305965106700000http://lattes.cnpq.br/3762670242328435D'AMORIM, Marcelo Bezerra2019-05-07T20:27:00Z2019-05-07T20:27:00Z2018-04-10https://repositorio.ufpe.br/handle/123456789/30520Testing is a costly process but essential in the development process. Complex systems may contain long-running test suites. Dealing with high testing costs remains an important problem in Software Engineering despite being under active research for years. Test suite parallelization is an important approach to address this problem, given the popularity of multi-core processors and native suport from testing frameworks and build systems. This work reports our findings on the usage and impact of test suite parallelization in open-source projects. This study brings to light the benefits and burdens of that approach. It provides recommendations to practitioners and tool developers to speed up test execution. Considering a set of 468 popular Java projects we analyzed, we found that 24% of the projects contain costly test suites but parallelization features still seem underutilized in practice—only 19.1% of costly projects use parallelization. The main reported reason for adoption resistance was the concern to deal with concurrency issues. Results suggest that, on average, developers prefer high predictability than high performance in running tests.FACEPETeste de software é um processo custoso mas essencial no processo de desenvolvimento. Sistemas complexos podem demandar várias horas para executar toda bateria de testes. Por tanto, estratégias para mitigar os custos de teste continuam sendo um tema importante na Engenharia de Software. Paralelismo de suites de teste é uma abordagem importante para lidar com este problema, dada a popularidade de processadores multi-core e suporte nativo de bibliotecas de teste e sistemas de build. Este trabalho reporta nossas descobertas a respeito da utilização e impacto de paralelismo de suites de teste em projetos de código aberto. Este estudo destaca os benefícios e desafios desta abordagem e provê recomendações para programadores e desenvolvedores de ferramentas para acelerar a execução de testes. Considerando um conjunto de 468 projetos populares desenvolvidos em Java, nós vimos que 24% destes projetos possuem suites de testes que demandam um alto custo de tempo de execução, porém o uso de paralelismo de suites de teste ainda é subutilizado na prática — apenas 19.1% destes projetos utilizam paralelismo. O principal motivo de resistência para a adoção desta abordagem é o receio de lidar com problemas de concorrência. Os resultados sugerem que, em média, desenvolvedores preferem alta previsibilidade ao invés de alta performance na execução de testes.engUniversidade Federal de PernambucoPrograma de Pos Graduacao em Ciencia da ComputacaoUFPEBrasilAttribution-NonCommercial-NoDerivs 3.0 Brazilhttp://creativecommons.org/licenses/by-nc-nd/3.0/br/info:eu-repo/semantics/openAccessEngenharia de softwareTeste de softwareParalelismo de testesTest suite parallelization in open-source projects: a study on its usage and impactinfo:eu-repo/semantics/publishedVersioninfo:eu-repo/semantics/masterThesismestradoreponame:Repositório Institucional da UFPEinstname:Universidade Federal de Pernambuco (UFPE)instacron:UFPETHUMBNAILDISSERTAÇÃO Jeanderson Barros Cândido.pdf.jpgDISSERTAÇÃO Jeanderson Barros Cândido.pdf.jpgGenerated Thumbnailimage/jpeg1245https://repositorio.ufpe.br/bitstream/123456789/30520/5/DISSERTA%c3%87%c3%83O%20Jeanderson%20Barros%20C%c3%a2ndido.pdf.jpg99bdef1d2be02953ae11908a06f703b9MD55ORIGINALDISSERTAÇÃO Jeanderson Barros Cândido.pdfDISSERTAÇÃO Jeanderson Barros Cândido.pdfapplication/pdf553677https://repositorio.ufpe.br/bitstream/123456789/30520/1/DISSERTA%c3%87%c3%83O%20Jeanderson%20Barros%20C%c3%a2ndido.pdfb6eeb5380a800bc771456b23c24ab373MD51CC-LICENSElicense_rdflicense_rdfapplication/rdf+xml; charset=utf-8811https://repositorio.ufpe.br/bitstream/123456789/30520/2/license_rdfe39d27027a6cc9cb039ad269a5db8e34MD52LICENSElicense.txtlicense.txttext/plain; charset=utf-82311https://repositorio.ufpe.br/bitstream/123456789/30520/3/license.txt4b8a02c7f2818eaf00dcf2260dd5eb08MD53TEXTDISSERTAÇÃO Jeanderson Barros Cândido.pdf.txtDISSERTAÇÃO Jeanderson Barros Cândido.pdf.txtExtracted texttext/plain95992https://repositorio.ufpe.br/bitstream/123456789/30520/4/DISSERTA%c3%87%c3%83O%20Jeanderson%20Barros%20C%c3%a2ndido.pdf.txtb4e405b497cc091606b73b1892413893MD54123456789/305202019-10-25 10:32:10.344oai:repositorio.ufpe.br:123456789/30520TGljZW7Dp2EgZGUgRGlzdHJpYnVpw6fDo28gTsOjbyBFeGNsdXNpdmEKClRvZG8gZGVwb3NpdGFudGUgZGUgbWF0ZXJpYWwgbm8gUmVwb3NpdMOzcmlvIEluc3RpdHVjaW9uYWwgKFJJKSBkZXZlIGNvbmNlZGVyLCDDoCBVbml2ZXJzaWRhZGUgRmVkZXJhbCBkZSBQZXJuYW1idWNvIChVRlBFKSwgdW1hIExpY2Vuw6dhIGRlIERpc3RyaWJ1acOnw6NvIE7Do28gRXhjbHVzaXZhIHBhcmEgbWFudGVyIGUgdG9ybmFyIGFjZXNzw612ZWlzIG9zIHNldXMgZG9jdW1lbnRvcywgZW0gZm9ybWF0byBkaWdpdGFsLCBuZXN0ZSByZXBvc2l0w7NyaW8uCgpDb20gYSBjb25jZXNzw6NvIGRlc3RhIGxpY2Vuw6dhIG7Do28gZXhjbHVzaXZhLCBvIGRlcG9zaXRhbnRlIG1hbnTDqW0gdG9kb3Mgb3MgZGlyZWl0b3MgZGUgYXV0b3IuCl9fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fXwoKTGljZW7Dp2EgZGUgRGlzdHJpYnVpw6fDo28gTsOjbyBFeGNsdXNpdmEKCkFvIGNvbmNvcmRhciBjb20gZXN0YSBsaWNlbsOnYSBlIGFjZWl0w6EtbGEsIHZvY8OqIChhdXRvciBvdSBkZXRlbnRvciBkb3MgZGlyZWl0b3MgYXV0b3JhaXMpOgoKYSkgRGVjbGFyYSBxdWUgY29uaGVjZSBhIHBvbMOtdGljYSBkZSBjb3B5cmlnaHQgZGEgZWRpdG9yYSBkbyBzZXUgZG9jdW1lbnRvOwpiKSBEZWNsYXJhIHF1ZSBjb25oZWNlIGUgYWNlaXRhIGFzIERpcmV0cml6ZXMgcGFyYSBvIFJlcG9zaXTDs3JpbyBJbnN0aXR1Y2lvbmFsIGRhIFVGUEU7CmMpIENvbmNlZGUgw6AgVUZQRSBvIGRpcmVpdG8gbsOjbyBleGNsdXNpdm8gZGUgYXJxdWl2YXIsIHJlcHJvZHV6aXIsIGNvbnZlcnRlciAoY29tbyBkZWZpbmlkbyBhIHNlZ3VpciksIGNvbXVuaWNhciBlL291IGRpc3RyaWJ1aXIsIG5vIFJJLCBvIGRvY3VtZW50byBlbnRyZWd1ZSAoaW5jbHVpbmRvIG8gcmVzdW1vL2Fic3RyYWN0KSBlbSBmb3JtYXRvIGRpZ2l0YWwgb3UgcG9yIG91dHJvIG1laW87CmQpIERlY2xhcmEgcXVlIGF1dG9yaXphIGEgVUZQRSBhIGFycXVpdmFyIG1haXMgZGUgdW1hIGPDs3BpYSBkZXN0ZSBkb2N1bWVudG8gZSBjb252ZXJ0w6otbG8sIHNlbSBhbHRlcmFyIG8gc2V1IGNvbnRlw7pkbywgcGFyYSBxdWFscXVlciBmb3JtYXRvIGRlIGZpY2hlaXJvLCBtZWlvIG91IHN1cG9ydGUsIHBhcmEgZWZlaXRvcyBkZSBzZWd1cmFuw6dhLCBwcmVzZXJ2YcOnw6NvIChiYWNrdXApIGUgYWNlc3NvOwplKSBEZWNsYXJhIHF1ZSBvIGRvY3VtZW50byBzdWJtZXRpZG8gw6kgbyBzZXUgdHJhYmFsaG8gb3JpZ2luYWwgZSBxdWUgZGV0w6ltIG8gZGlyZWl0byBkZSBjb25jZWRlciBhIHRlcmNlaXJvcyBvcyBkaXJlaXRvcyBjb250aWRvcyBuZXN0YSBsaWNlbsOnYS4gRGVjbGFyYSB0YW1iw6ltIHF1ZSBhIGVudHJlZ2EgZG8gZG9jdW1lbnRvIG7Do28gaW5mcmluZ2Ugb3MgZGlyZWl0b3MgZGUgb3V0cmEgcGVzc29hIG91IGVudGlkYWRlOwpmKSBEZWNsYXJhIHF1ZSwgbm8gY2FzbyBkbyBkb2N1bWVudG8gc3VibWV0aWRvIGNvbnRlciBtYXRlcmlhbCBkbyBxdWFsIG7Do28gZGV0w6ltIG9zIGRpcmVpdG9zIGRlCmF1dG9yLCBvYnRldmUgYSBhdXRvcml6YcOnw6NvIGlycmVzdHJpdGEgZG8gcmVzcGVjdGl2byBkZXRlbnRvciBkZXNzZXMgZGlyZWl0b3MgcGFyYSBjZWRlciDDoApVRlBFIG9zIGRpcmVpdG9zIHJlcXVlcmlkb3MgcG9yIGVzdGEgTGljZW7Dp2EgZSBhdXRvcml6YXIgYSB1bml2ZXJzaWRhZGUgYSB1dGlsaXrDoS1sb3MgbGVnYWxtZW50ZS4gRGVjbGFyYSB0YW1iw6ltIHF1ZSBlc3NlIG1hdGVyaWFsIGN1am9zIGRpcmVpdG9zIHPDo28gZGUgdGVyY2Vpcm9zIGVzdMOhIGNsYXJhbWVudGUgaWRlbnRpZmljYWRvIGUgcmVjb25oZWNpZG8gbm8gdGV4dG8gb3UgY29udGXDumRvIGRvIGRvY3VtZW50byBlbnRyZWd1ZTsKZykgU2UgbyBkb2N1bWVudG8gZW50cmVndWUgw6kgYmFzZWFkbyBlbSB0cmFiYWxobyBmaW5hbmNpYWRvIG91IGFwb2lhZG8gcG9yIG91dHJhIGluc3RpdHVpw6fDo28gcXVlIG7Do28gYSBVRlBFLMKgZGVjbGFyYSBxdWUgY3VtcHJpdSBxdWFpc3F1ZXIgb2JyaWdhw6fDtWVzIGV4aWdpZGFzIHBlbG8gcmVzcGVjdGl2byBjb250cmF0byBvdSBhY29yZG8uCgpBIFVGUEUgaWRlbnRpZmljYXLDoSBjbGFyYW1lbnRlIG8ocykgbm9tZShzKSBkbyhzKSBhdXRvciAoZXMpIGRvcyBkaXJlaXRvcyBkbyBkb2N1bWVudG8gZW50cmVndWUgZSBuw6NvIGZhcsOhIHF1YWxxdWVyIGFsdGVyYcOnw6NvLCBwYXJhIGFsw6ltIGRvIHByZXZpc3RvIG5hIGFsw61uZWEgYykuCg==Repositório InstitucionalPUBhttps://repositorio.ufpe.br/oai/requestattena@ufpe.bropendoar:22212019-10-25T13:32:10Repositório Institucional da UFPE - Universidade Federal de Pernambuco (UFPE)false
dc.title.pt_BR.fl_str_mv Test suite parallelization in open-source projects: a study on its usage and impact
title Test suite parallelization in open-source projects: a study on its usage and impact
spellingShingle Test suite parallelization in open-source projects: a study on its usage and impact
CÂNDIDO, Jeanderson Barros
Engenharia de software
Teste de software
Paralelismo de testes
title_short Test suite parallelization in open-source projects: a study on its usage and impact
title_full Test suite parallelization in open-source projects: a study on its usage and impact
title_fullStr Test suite parallelization in open-source projects: a study on its usage and impact
title_full_unstemmed Test suite parallelization in open-source projects: a study on its usage and impact
title_sort Test suite parallelization in open-source projects: a study on its usage and impact
author CÂNDIDO, Jeanderson Barros
author_facet CÂNDIDO, Jeanderson Barros
author_role author
dc.contributor.authorLattes.pt_BR.fl_str_mv http://lattes.cnpq.br/2305965106700000
dc.contributor.advisorLattes.pt_BR.fl_str_mv http://lattes.cnpq.br/3762670242328435
dc.contributor.author.fl_str_mv CÂNDIDO, Jeanderson Barros
dc.contributor.advisor1.fl_str_mv D'AMORIM, Marcelo Bezerra
contributor_str_mv D'AMORIM, Marcelo Bezerra
dc.subject.por.fl_str_mv Engenharia de software
Teste de software
Paralelismo de testes
topic Engenharia de software
Teste de software
Paralelismo de testes
description Testing is a costly process but essential in the development process. Complex systems may contain long-running test suites. Dealing with high testing costs remains an important problem in Software Engineering despite being under active research for years. Test suite parallelization is an important approach to address this problem, given the popularity of multi-core processors and native suport from testing frameworks and build systems. This work reports our findings on the usage and impact of test suite parallelization in open-source projects. This study brings to light the benefits and burdens of that approach. It provides recommendations to practitioners and tool developers to speed up test execution. Considering a set of 468 popular Java projects we analyzed, we found that 24% of the projects contain costly test suites but parallelization features still seem underutilized in practice—only 19.1% of costly projects use parallelization. The main reported reason for adoption resistance was the concern to deal with concurrency issues. Results suggest that, on average, developers prefer high predictability than high performance in running tests.
publishDate 2018
dc.date.issued.fl_str_mv 2018-04-10
dc.date.accessioned.fl_str_mv 2019-05-07T20:27:00Z
dc.date.available.fl_str_mv 2019-05-07T20:27:00Z
dc.type.status.fl_str_mv info:eu-repo/semantics/publishedVersion
dc.type.driver.fl_str_mv info:eu-repo/semantics/masterThesis
format masterThesis
status_str publishedVersion
dc.identifier.uri.fl_str_mv https://repositorio.ufpe.br/handle/123456789/30520
url https://repositorio.ufpe.br/handle/123456789/30520
dc.language.iso.fl_str_mv eng
language eng
dc.rights.driver.fl_str_mv Attribution-NonCommercial-NoDerivs 3.0 Brazil
http://creativecommons.org/licenses/by-nc-nd/3.0/br/
info:eu-repo/semantics/openAccess
rights_invalid_str_mv Attribution-NonCommercial-NoDerivs 3.0 Brazil
http://creativecommons.org/licenses/by-nc-nd/3.0/br/
eu_rights_str_mv openAccess
dc.publisher.none.fl_str_mv Universidade Federal de Pernambuco
dc.publisher.program.fl_str_mv Programa de Pos Graduacao em Ciencia da Computacao
dc.publisher.initials.fl_str_mv UFPE
dc.publisher.country.fl_str_mv Brasil
publisher.none.fl_str_mv Universidade Federal de Pernambuco
dc.source.none.fl_str_mv reponame:Repositório Institucional da UFPE
instname:Universidade Federal de Pernambuco (UFPE)
instacron:UFPE
instname_str Universidade Federal de Pernambuco (UFPE)
instacron_str UFPE
institution UFPE
reponame_str Repositório Institucional da UFPE
collection Repositório Institucional da UFPE
bitstream.url.fl_str_mv https://repositorio.ufpe.br/bitstream/123456789/30520/5/DISSERTA%c3%87%c3%83O%20Jeanderson%20Barros%20C%c3%a2ndido.pdf.jpg
https://repositorio.ufpe.br/bitstream/123456789/30520/1/DISSERTA%c3%87%c3%83O%20Jeanderson%20Barros%20C%c3%a2ndido.pdf
https://repositorio.ufpe.br/bitstream/123456789/30520/2/license_rdf
https://repositorio.ufpe.br/bitstream/123456789/30520/3/license.txt
https://repositorio.ufpe.br/bitstream/123456789/30520/4/DISSERTA%c3%87%c3%83O%20Jeanderson%20Barros%20C%c3%a2ndido.pdf.txt
bitstream.checksum.fl_str_mv 99bdef1d2be02953ae11908a06f703b9
b6eeb5380a800bc771456b23c24ab373
e39d27027a6cc9cb039ad269a5db8e34
4b8a02c7f2818eaf00dcf2260dd5eb08
b4e405b497cc091606b73b1892413893
bitstream.checksumAlgorithm.fl_str_mv MD5
MD5
MD5
MD5
MD5
repository.name.fl_str_mv Repositório Institucional da UFPE - Universidade Federal de Pernambuco (UFPE)
repository.mail.fl_str_mv attena@ufpe.br
_version_ 1802310601923362816

Registros relacionados