Test suite parallelization in open-source projects: a study on its usage and impact
Autor(a) principal: | |
---|---|
Data de Publicação: | 2018 |
Tipo de documento: | Dissertação |
Idioma: | eng |
Título da fonte: | Repositório Institucional da UFPE |
Texto Completo: | https://repositorio.ufpe.br/handle/123456789/30520 |
Resumo: | Testing is a costly process but essential in the development process. Complex systems may contain long-running test suites. Dealing with high testing costs remains an important problem in Software Engineering despite being under active research for years. Test suite parallelization is an important approach to address this problem, given the popularity of multi-core processors and native suport from testing frameworks and build systems. This work reports our findings on the usage and impact of test suite parallelization in open-source projects. This study brings to light the benefits and burdens of that approach. It provides recommendations to practitioners and tool developers to speed up test execution. Considering a set of 468 popular Java projects we analyzed, we found that 24% of the projects contain costly test suites but parallelization features still seem underutilized in practice—only 19.1% of costly projects use parallelization. The main reported reason for adoption resistance was the concern to deal with concurrency issues. Results suggest that, on average, developers prefer high predictability than high performance in running tests. |
id |
UFPE_9c0fb78a95603b38808683df122a7b3e |
---|---|
oai_identifier_str |
oai:repositorio.ufpe.br:123456789/30520 |
network_acronym_str |
UFPE |
network_name_str |
Repositório Institucional da UFPE |
repository_id_str |
2221 |
spelling |
CÂNDIDO, Jeanderson Barroshttp://lattes.cnpq.br/2305965106700000http://lattes.cnpq.br/3762670242328435D'AMORIM, Marcelo Bezerra2019-05-07T20:27:00Z2019-05-07T20:27:00Z2018-04-10https://repositorio.ufpe.br/handle/123456789/30520Testing is a costly process but essential in the development process. Complex systems may contain long-running test suites. Dealing with high testing costs remains an important problem in Software Engineering despite being under active research for years. Test suite parallelization is an important approach to address this problem, given the popularity of multi-core processors and native suport from testing frameworks and build systems. This work reports our findings on the usage and impact of test suite parallelization in open-source projects. This study brings to light the benefits and burdens of that approach. It provides recommendations to practitioners and tool developers to speed up test execution. Considering a set of 468 popular Java projects we analyzed, we found that 24% of the projects contain costly test suites but parallelization features still seem underutilized in practice—only 19.1% of costly projects use parallelization. The main reported reason for adoption resistance was the concern to deal with concurrency issues. Results suggest that, on average, developers prefer high predictability than high performance in running tests.FACEPETeste de software é um processo custoso mas essencial no processo de desenvolvimento. Sistemas complexos podem demandar várias horas para executar toda bateria de testes. Por tanto, estratégias para mitigar os custos de teste continuam sendo um tema importante na Engenharia de Software. Paralelismo de suites de teste é uma abordagem importante para lidar com este problema, dada a popularidade de processadores multi-core e suporte nativo de bibliotecas de teste e sistemas de build. Este trabalho reporta nossas descobertas a respeito da utilização e impacto de paralelismo de suites de teste em projetos de código aberto. Este estudo destaca os benefícios e desafios desta abordagem e provê recomendações para programadores e desenvolvedores de ferramentas para acelerar a execução de testes. Considerando um conjunto de 468 projetos populares desenvolvidos em Java, nós vimos que 24% destes projetos possuem suites de testes que demandam um alto custo de tempo de execução, porém o uso de paralelismo de suites de teste ainda é subutilizado na prática — apenas 19.1% destes projetos utilizam paralelismo. O principal motivo de resistência para a adoção desta abordagem é o receio de lidar com problemas de concorrência. Os resultados sugerem que, em média, desenvolvedores preferem alta previsibilidade ao invés de alta performance na execução de testes.engUniversidade Federal de PernambucoPrograma de Pos Graduacao em Ciencia da ComputacaoUFPEBrasilAttribution-NonCommercial-NoDerivs 3.0 Brazilhttp://creativecommons.org/licenses/by-nc-nd/3.0/br/info:eu-repo/semantics/openAccessEngenharia de softwareTeste de softwareParalelismo de testesTest suite parallelization in open-source projects: a study on its usage and impactinfo:eu-repo/semantics/publishedVersioninfo:eu-repo/semantics/masterThesismestradoreponame:Repositório Institucional da UFPEinstname:Universidade Federal de Pernambuco (UFPE)instacron:UFPETHUMBNAILDISSERTAÇÃO Jeanderson Barros Cândido.pdf.jpgDISSERTAÇÃO Jeanderson Barros Cândido.pdf.jpgGenerated Thumbnailimage/jpeg1245https://repositorio.ufpe.br/bitstream/123456789/30520/5/DISSERTA%c3%87%c3%83O%20Jeanderson%20Barros%20C%c3%a2ndido.pdf.jpg99bdef1d2be02953ae11908a06f703b9MD55ORIGINALDISSERTAÇÃO Jeanderson Barros Cândido.pdfDISSERTAÇÃO Jeanderson Barros Cândido.pdfapplication/pdf553677https://repositorio.ufpe.br/bitstream/123456789/30520/1/DISSERTA%c3%87%c3%83O%20Jeanderson%20Barros%20C%c3%a2ndido.pdfb6eeb5380a800bc771456b23c24ab373MD51CC-LICENSElicense_rdflicense_rdfapplication/rdf+xml; charset=utf-8811https://repositorio.ufpe.br/bitstream/123456789/30520/2/license_rdfe39d27027a6cc9cb039ad269a5db8e34MD52LICENSElicense.txtlicense.txttext/plain; charset=utf-82311https://repositorio.ufpe.br/bitstream/123456789/30520/3/license.txt4b8a02c7f2818eaf00dcf2260dd5eb08MD53TEXTDISSERTAÇÃO Jeanderson Barros Cândido.pdf.txtDISSERTAÇÃO Jeanderson Barros Cândido.pdf.txtExtracted texttext/plain95992https://repositorio.ufpe.br/bitstream/123456789/30520/4/DISSERTA%c3%87%c3%83O%20Jeanderson%20Barros%20C%c3%a2ndido.pdf.txtb4e405b497cc091606b73b1892413893MD54123456789/305202019-10-25 10:32:10.344oai:repositorio.ufpe.br:123456789/30520TGljZW7Dp2EgZGUgRGlzdHJpYnVpw6fDo28gTsOjbyBFeGNsdXNpdmEKClRvZG8gZGVwb3NpdGFudGUgZGUgbWF0ZXJpYWwgbm8gUmVwb3NpdMOzcmlvIEluc3RpdHVjaW9uYWwgKFJJKSBkZXZlIGNvbmNlZGVyLCDDoCBVbml2ZXJzaWRhZGUgRmVkZXJhbCBkZSBQZXJuYW1idWNvIChVRlBFKSwgdW1hIExpY2Vuw6dhIGRlIERpc3RyaWJ1acOnw6NvIE7Do28gRXhjbHVzaXZhIHBhcmEgbWFudGVyIGUgdG9ybmFyIGFjZXNzw612ZWlzIG9zIHNldXMgZG9jdW1lbnRvcywgZW0gZm9ybWF0byBkaWdpdGFsLCBuZXN0ZSByZXBvc2l0w7NyaW8uCgpDb20gYSBjb25jZXNzw6NvIGRlc3RhIGxpY2Vuw6dhIG7Do28gZXhjbHVzaXZhLCBvIGRlcG9zaXRhbnRlIG1hbnTDqW0gdG9kb3Mgb3MgZGlyZWl0b3MgZGUgYXV0b3IuCl9fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fXwoKTGljZW7Dp2EgZGUgRGlzdHJpYnVpw6fDo28gTsOjbyBFeGNsdXNpdmEKCkFvIGNvbmNvcmRhciBjb20gZXN0YSBsaWNlbsOnYSBlIGFjZWl0w6EtbGEsIHZvY8OqIChhdXRvciBvdSBkZXRlbnRvciBkb3MgZGlyZWl0b3MgYXV0b3JhaXMpOgoKYSkgRGVjbGFyYSBxdWUgY29uaGVjZSBhIHBvbMOtdGljYSBkZSBjb3B5cmlnaHQgZGEgZWRpdG9yYSBkbyBzZXUgZG9jdW1lbnRvOwpiKSBEZWNsYXJhIHF1ZSBjb25oZWNlIGUgYWNlaXRhIGFzIERpcmV0cml6ZXMgcGFyYSBvIFJlcG9zaXTDs3JpbyBJbnN0aXR1Y2lvbmFsIGRhIFVGUEU7CmMpIENvbmNlZGUgw6AgVUZQRSBvIGRpcmVpdG8gbsOjbyBleGNsdXNpdm8gZGUgYXJxdWl2YXIsIHJlcHJvZHV6aXIsIGNvbnZlcnRlciAoY29tbyBkZWZpbmlkbyBhIHNlZ3VpciksIGNvbXVuaWNhciBlL291IGRpc3RyaWJ1aXIsIG5vIFJJLCBvIGRvY3VtZW50byBlbnRyZWd1ZSAoaW5jbHVpbmRvIG8gcmVzdW1vL2Fic3RyYWN0KSBlbSBmb3JtYXRvIGRpZ2l0YWwgb3UgcG9yIG91dHJvIG1laW87CmQpIERlY2xhcmEgcXVlIGF1dG9yaXphIGEgVUZQRSBhIGFycXVpdmFyIG1haXMgZGUgdW1hIGPDs3BpYSBkZXN0ZSBkb2N1bWVudG8gZSBjb252ZXJ0w6otbG8sIHNlbSBhbHRlcmFyIG8gc2V1IGNvbnRlw7pkbywgcGFyYSBxdWFscXVlciBmb3JtYXRvIGRlIGZpY2hlaXJvLCBtZWlvIG91IHN1cG9ydGUsIHBhcmEgZWZlaXRvcyBkZSBzZWd1cmFuw6dhLCBwcmVzZXJ2YcOnw6NvIChiYWNrdXApIGUgYWNlc3NvOwplKSBEZWNsYXJhIHF1ZSBvIGRvY3VtZW50byBzdWJtZXRpZG8gw6kgbyBzZXUgdHJhYmFsaG8gb3JpZ2luYWwgZSBxdWUgZGV0w6ltIG8gZGlyZWl0byBkZSBjb25jZWRlciBhIHRlcmNlaXJvcyBvcyBkaXJlaXRvcyBjb250aWRvcyBuZXN0YSBsaWNlbsOnYS4gRGVjbGFyYSB0YW1iw6ltIHF1ZSBhIGVudHJlZ2EgZG8gZG9jdW1lbnRvIG7Do28gaW5mcmluZ2Ugb3MgZGlyZWl0b3MgZGUgb3V0cmEgcGVzc29hIG91IGVudGlkYWRlOwpmKSBEZWNsYXJhIHF1ZSwgbm8gY2FzbyBkbyBkb2N1bWVudG8gc3VibWV0aWRvIGNvbnRlciBtYXRlcmlhbCBkbyBxdWFsIG7Do28gZGV0w6ltIG9zIGRpcmVpdG9zIGRlCmF1dG9yLCBvYnRldmUgYSBhdXRvcml6YcOnw6NvIGlycmVzdHJpdGEgZG8gcmVzcGVjdGl2byBkZXRlbnRvciBkZXNzZXMgZGlyZWl0b3MgcGFyYSBjZWRlciDDoApVRlBFIG9zIGRpcmVpdG9zIHJlcXVlcmlkb3MgcG9yIGVzdGEgTGljZW7Dp2EgZSBhdXRvcml6YXIgYSB1bml2ZXJzaWRhZGUgYSB1dGlsaXrDoS1sb3MgbGVnYWxtZW50ZS4gRGVjbGFyYSB0YW1iw6ltIHF1ZSBlc3NlIG1hdGVyaWFsIGN1am9zIGRpcmVpdG9zIHPDo28gZGUgdGVyY2Vpcm9zIGVzdMOhIGNsYXJhbWVudGUgaWRlbnRpZmljYWRvIGUgcmVjb25oZWNpZG8gbm8gdGV4dG8gb3UgY29udGXDumRvIGRvIGRvY3VtZW50byBlbnRyZWd1ZTsKZykgU2UgbyBkb2N1bWVudG8gZW50cmVndWUgw6kgYmFzZWFkbyBlbSB0cmFiYWxobyBmaW5hbmNpYWRvIG91IGFwb2lhZG8gcG9yIG91dHJhIGluc3RpdHVpw6fDo28gcXVlIG7Do28gYSBVRlBFLMKgZGVjbGFyYSBxdWUgY3VtcHJpdSBxdWFpc3F1ZXIgb2JyaWdhw6fDtWVzIGV4aWdpZGFzIHBlbG8gcmVzcGVjdGl2byBjb250cmF0byBvdSBhY29yZG8uCgpBIFVGUEUgaWRlbnRpZmljYXLDoSBjbGFyYW1lbnRlIG8ocykgbm9tZShzKSBkbyhzKSBhdXRvciAoZXMpIGRvcyBkaXJlaXRvcyBkbyBkb2N1bWVudG8gZW50cmVndWUgZSBuw6NvIGZhcsOhIHF1YWxxdWVyIGFsdGVyYcOnw6NvLCBwYXJhIGFsw6ltIGRvIHByZXZpc3RvIG5hIGFsw61uZWEgYykuCg==Repositório InstitucionalPUBhttps://repositorio.ufpe.br/oai/requestattena@ufpe.bropendoar:22212019-10-25T13:32:10Repositório Institucional da UFPE - Universidade Federal de Pernambuco (UFPE)false |
dc.title.pt_BR.fl_str_mv |
Test suite parallelization in open-source projects: a study on its usage and impact |
title |
Test suite parallelization in open-source projects: a study on its usage and impact |
spellingShingle |
Test suite parallelization in open-source projects: a study on its usage and impact CÂNDIDO, Jeanderson Barros Engenharia de software Teste de software Paralelismo de testes |
title_short |
Test suite parallelization in open-source projects: a study on its usage and impact |
title_full |
Test suite parallelization in open-source projects: a study on its usage and impact |
title_fullStr |
Test suite parallelization in open-source projects: a study on its usage and impact |
title_full_unstemmed |
Test suite parallelization in open-source projects: a study on its usage and impact |
title_sort |
Test suite parallelization in open-source projects: a study on its usage and impact |
author |
CÂNDIDO, Jeanderson Barros |
author_facet |
CÂNDIDO, Jeanderson Barros |
author_role |
author |
dc.contributor.authorLattes.pt_BR.fl_str_mv |
http://lattes.cnpq.br/2305965106700000 |
dc.contributor.advisorLattes.pt_BR.fl_str_mv |
http://lattes.cnpq.br/3762670242328435 |
dc.contributor.author.fl_str_mv |
CÂNDIDO, Jeanderson Barros |
dc.contributor.advisor1.fl_str_mv |
D'AMORIM, Marcelo Bezerra |
contributor_str_mv |
D'AMORIM, Marcelo Bezerra |
dc.subject.por.fl_str_mv |
Engenharia de software Teste de software Paralelismo de testes |
topic |
Engenharia de software Teste de software Paralelismo de testes |
description |
Testing is a costly process but essential in the development process. Complex systems may contain long-running test suites. Dealing with high testing costs remains an important problem in Software Engineering despite being under active research for years. Test suite parallelization is an important approach to address this problem, given the popularity of multi-core processors and native suport from testing frameworks and build systems. This work reports our findings on the usage and impact of test suite parallelization in open-source projects. This study brings to light the benefits and burdens of that approach. It provides recommendations to practitioners and tool developers to speed up test execution. Considering a set of 468 popular Java projects we analyzed, we found that 24% of the projects contain costly test suites but parallelization features still seem underutilized in practice—only 19.1% of costly projects use parallelization. The main reported reason for adoption resistance was the concern to deal with concurrency issues. Results suggest that, on average, developers prefer high predictability than high performance in running tests. |
publishDate |
2018 |
dc.date.issued.fl_str_mv |
2018-04-10 |
dc.date.accessioned.fl_str_mv |
2019-05-07T20:27:00Z |
dc.date.available.fl_str_mv |
2019-05-07T20:27:00Z |
dc.type.status.fl_str_mv |
info:eu-repo/semantics/publishedVersion |
dc.type.driver.fl_str_mv |
info:eu-repo/semantics/masterThesis |
format |
masterThesis |
status_str |
publishedVersion |
dc.identifier.uri.fl_str_mv |
https://repositorio.ufpe.br/handle/123456789/30520 |
url |
https://repositorio.ufpe.br/handle/123456789/30520 |
dc.language.iso.fl_str_mv |
eng |
language |
eng |
dc.rights.driver.fl_str_mv |
Attribution-NonCommercial-NoDerivs 3.0 Brazil http://creativecommons.org/licenses/by-nc-nd/3.0/br/ info:eu-repo/semantics/openAccess |
rights_invalid_str_mv |
Attribution-NonCommercial-NoDerivs 3.0 Brazil http://creativecommons.org/licenses/by-nc-nd/3.0/br/ |
eu_rights_str_mv |
openAccess |
dc.publisher.none.fl_str_mv |
Universidade Federal de Pernambuco |
dc.publisher.program.fl_str_mv |
Programa de Pos Graduacao em Ciencia da Computacao |
dc.publisher.initials.fl_str_mv |
UFPE |
dc.publisher.country.fl_str_mv |
Brasil |
publisher.none.fl_str_mv |
Universidade Federal de Pernambuco |
dc.source.none.fl_str_mv |
reponame:Repositório Institucional da UFPE instname:Universidade Federal de Pernambuco (UFPE) instacron:UFPE |
instname_str |
Universidade Federal de Pernambuco (UFPE) |
instacron_str |
UFPE |
institution |
UFPE |
reponame_str |
Repositório Institucional da UFPE |
collection |
Repositório Institucional da UFPE |
bitstream.url.fl_str_mv |
https://repositorio.ufpe.br/bitstream/123456789/30520/5/DISSERTA%c3%87%c3%83O%20Jeanderson%20Barros%20C%c3%a2ndido.pdf.jpg https://repositorio.ufpe.br/bitstream/123456789/30520/1/DISSERTA%c3%87%c3%83O%20Jeanderson%20Barros%20C%c3%a2ndido.pdf https://repositorio.ufpe.br/bitstream/123456789/30520/2/license_rdf https://repositorio.ufpe.br/bitstream/123456789/30520/3/license.txt https://repositorio.ufpe.br/bitstream/123456789/30520/4/DISSERTA%c3%87%c3%83O%20Jeanderson%20Barros%20C%c3%a2ndido.pdf.txt |
bitstream.checksum.fl_str_mv |
99bdef1d2be02953ae11908a06f703b9 b6eeb5380a800bc771456b23c24ab373 e39d27027a6cc9cb039ad269a5db8e34 4b8a02c7f2818eaf00dcf2260dd5eb08 b4e405b497cc091606b73b1892413893 |
bitstream.checksumAlgorithm.fl_str_mv |
MD5 MD5 MD5 MD5 MD5 |
repository.name.fl_str_mv |
Repositório Institucional da UFPE - Universidade Federal de Pernambuco (UFPE) |
repository.mail.fl_str_mv |
attena@ufpe.br |
_version_ |
1802310601923362816 |