Sample size estimation for power and accuracy in the experimental comparison of metaheuristics

• Autor(a) principal: Fernanda Caldeira Takahashi
• Data de Publicação: 2018
• Tipo de documento: Tese
• Idioma: eng
• Título da fonte: Repositório Institucional da UFMG
• Texto Completo: http://hdl.handle.net/1843/32571
• topic: Engenharia elétrica; Engenharia elétrica; Programação heurística; Algoritmos

Experimental algorithmics encompasses the study of guidelines and methods for computational evaluation of algorithms. In the optimization field, it is useful for testing the performance of algorithms when solving a certain type of problem. In this work we develop a methodology for generating adequate experimental designs for comparing the performance of optimization metaheuristics, with a focus on statistical power and accuracy in parameter estimation. In particular, we deal with sample size estimation for experiments involving optimization algorithms, both in terms of within-instance repeated executions and the number of instances required. A statistically sound methodology is presented for sample size calculation, allowing relevant comparisons between the performances of two algorithms for a given class of problems. The methodology’s effectiveness is validated using simulated models and exemplified with two case studies. The proposed methodology was implemented in the form of an open source R package, published in the CRAN repository.