Codificação de imagens em sinais táteis através de redes neurais convolucionais

• Autor(a) principal: Ribani, Ricardo
• Data de Publicação: 2020
• Tipo de documento: Tese
• Idioma: por
• Título da fonte: Biblioteca Digital de Teses e Dissertações do Mackenzie
• Texto Completo: http://dspace.mackenzie.br/handle/10899/26514
• topic: visão computacional; redes neurais; substituição sensorial; tecnologia assistiva; interface háptica; CNPQ::CIENCIAS EXATAS E DA TERRA::CIENCIA DA COMPUTACAO::SISTEMAS DE COMPUTACAO; CNPQ::CIENCIAS EXATAS E DA TERRA::CIENCIA DA COMPUTACAO::METODOLOGIA E TECNICAS DA COMPUTACAO::ENGENHARIA DE SOFTWARE

The present work proposes the creation of a system that implements the sensory subs- titution of vision through a wearable item with actuators positioned in the back of the user. A research across the sensory substitution and plasticity domain was made in order to understand the capacity of the brain to adapt and investigate ways of encode informa- tion from vision to touch. In addition to the hardware developed, a system is proposed to encode vision-to-touch using convolutional neural networks. Beyond the initial tests with classi_cation and object detection models, we evaluated two methods that have ne- ver been used for encoding vision-to-touch using convolutional neural networks, a bag of convolutional features (BoF) and a vector of locally aggregated descriptors (VLAD). In this work, a very new method is present for evaluating the semantic property of the enco- ded signal by taking the idea that objects with similar features must have similar signals in the tactile interface; we created a semantic property evaluation (SPE) metric. Using this metric, we proved the advantage of using the BoF and VLAD methods, obtaining an SPE of 70.7% and 64.5%, respectively, which is a considerable improvement over the downscaling method used by many systems such as BrainPort, with 56.2%.