Fault model-based variability testing

• Autor(a) principal: Machado, Ivan do Carmo
• Data de Publicação: 2014
• Tipo de documento: Tese
• Idioma: eng
• Título da fonte: Repositório Institucional da UFBA
• Texto Completo: http://repositorio.ufba.br/ri/handle/ri/22826
• topic: Sistemas Computacionais; Software product line enginering; Software testing; Variability testing; Fault model

Software Product Line (SPL) engineering has emerged as an important strategy to cope with the increasing demand of large-scale product customization. Owing to its variability management capabilities, SPL has provided companies with an efficient and effective means of delivering a set of products with higher quality at a lower cost, when compared to traditional software engineering strategies. However, such a benefit does not come for free. SPL demands cost-effective quality assurance techniques that attempt to minimize the overall effort, while improving, or at least not hurting, fault detection rates. Software testing, the most widely used approach for improving software quality in practice, has been largely explored to address this particular topic. State of the art SPL testing techniques are mainly focused on handling variability testing from a high level perspective, namely through the analysis of feature models, rather than concerning issues from a source code perspective. However, we believe that improvements in the quality of variable assets entail addressing testing issues both from high and low-level perspectives. By carrying out a series of empirical studies, gathering evidence from both the literature and the analysis of defects reported in three open source software systems, we identified and analyzed commonly reported defects from Java-based variability implementation mechanisms. Based on such evidence, we designed a method for building fault models for variability testing, from two main perspectives: test assessment, which focuses on the evaluation of the effectiveness of existing test suites; and test design, which aims to aid the construction of test sets, by focusing on fault-prone elements. The task of modeling typical or important faults provides a means to coming up with certain test inpus that can expose faults in the program unit. Hence, we hypothesize that understanding the nature of typical or important faults prior to developing the test sets would enhance their capability to find a particular set of errors. We performed a controlled experiment to assess the test effectiveness of using fault models to provide SPL testing with support to design test inputs. We observed promising results that confirm the hypothesis that combining fault models in an SPL testing process performs significantly better on improving the quality of test inputs.