Lgclvoauto: correction of labels with gradient descent optimization for graph-based semi-supervised learning

• Autor(a) principal: Afonso, Bruno Klaus de Aquino [UNIFESP]
• Data de Publicação: 2020
• Tipo de documento: Trabalho de conclusão de curso
• Idioma: eng
• Título da fonte: Repositório Institucional da UNIFESP
• Texto Completo: https://repositorio.unifesp.br/handle/11600/60704
• topic: Aprendizado de Máquina; Ruído de rótulo; Aprendizado Semissupervisionado baseado em Grafos; Filtro; Consistência Local e Global

We consider the problem of learning with noisy labels where most of the data is unlabelled. More specifically, we focus on graph-based semi-supervised learning, a setting in which many different approaches have already been proposed, such as $\ell_1$ norm, smooth eigenbasis pursuit and bivariate formulation. We propose our own semi-supervised filter, named Automatic Leave-One-Out Filter based on Local and Global Consistency (LGCLVOAuto), that corrects and redistributes label information in order to minimize leave-one-out error, while remaining consistent with the random walk process imposed by its baseline, the Local and Global Consistency (LGC) algorithm. We explore the problem of diagonal dominance in LGC solutions and its possible relation to overfitting, and how setting it to zero leads to the leave-one-out cost. We make use of gradient descent optimization on labels to minimize this cost, transferring some of the trust from the labels themselves to the propagation model. In order to eliminate degenerate solutions, some restrictions are put in place: labels cannot change class, and the overall contribution for each class should remain the same. The optimization requires only the relations between labels: consequently, it is suited to moderately large datasets such as MNIST, in particular when labelled data is scarce. It requires a single parameter. In theory, it may be extended trivially to the more general LapRLS classifier. Results show that LGCLVOAuto is capable of outperforming its baseline significantly when there is noise, and not be too harmful in the noiseless scenario. Moreover, it is competitive with other methods that require more parameters.