Estimation of knots location and number in the splines regression models using an optimization approach

• Autor(a) principal: Ferreira, Alberto Rodrigues
• Data de Publicação: 2022
• Tipo de documento: Dissertação
• Idioma: eng
• Título da fonte: Biblioteca Digital de Teses e Dissertações da USP
• Texto Completo: https://www.teses.usp.br/teses/disponiveis/45/45133/tde-08082022-182210/
• topic: BFGS; BFGS; Estimação da localização dos knots; Estimação do número de knots; Knots location estimation; Knots number estimation; Métodos de regularização; Modelo de regressão splines; Regularization methods; Splines regression model

In many practical problems related to supervised statistical learning, we are interested in predicting a continuous target. Frequently, the relationship between the explanatory variable and the target variable is nonlinear, so models that introduce nonlinearity for this purpose tend to obtain better performances in general. A statistical model that addresses this problem called the regression splines model has received considerable attention in recent years. This is due to its great predictive power and good fits incorporated by its flexibility. However, the splines regression model has a significant disadvantage: one of its main components, called knots, related to the change points, are usually chosen before the estimation process. They are considered pre-specified values, which in some situations can present severe problems in practical problems. In this work, we propose a new methodology that tries to solve this considering the knots location and knots number as parameters, and we solve this problem as an optimization approach using the nonlinear optimization algorithm BFGS. Furthermore, we introduce new regularization methods to penalize variables with irrelevant knots and avoid overfitting. The proposed methodology obtained many advantages compared to the approach used in the literature, such as automatic estimation of the number and location of knots, regularization methods that avoids overfitting, and selection of irrelevant knots. Our ap- proach obtained several gains in predictive performance and knots estimation in the simulations, thus obtaining better results than the usual procedure.