Eigenvalue decay: a new method for neural network regularization

• Autor(a) principal: Ludwig, Oswaldo
• Data de Publicação: 2014
• Outros Autores: Nunes, Urbano, Araujo, Rui
• Tipo de documento: Artigo
• Idioma: eng
• Título da fonte: Repositório Científico de Acesso Aberto de Portugal (Repositórios Cientìficos)
• Texto Completo: http://hdl.handle.net/10316/27727; https://doi.org/10.1016/j.neucom.2013.08.005
• topic: Transduction; Regularization; Genetic algorithm; Classification margin; Neural network

This paper proposes two new training algorithms for multilayer perceptrons based on evolutionary computation, regularization, and transduction. Regularization is a commonly used technique for preventing the learning algorithm from overfitting the training data. In this context, this work introduces and analyzes a novel regularization scheme for neural networks (NNs) named eigenvalue decay, which aims at improving the classification margin. The introduction of eigenvalue decay led to the development of a new training method based on the same principles of SVM, and so named Support Vector NN (SVNN). Finally, by analogy with the transductive SVM (TSVM), it is proposed a transductive NN (TNN), by exploiting SVNN in order to address transductive learning. The effectiveness of the proposed algorithms is evaluated on seven benchmark datasets.