Visually Impaired Aid using Convolutional Neural Networks, Transfer Learning, and Particle Competition and Cooperation

Detalhes bibliográficos
Autor(a) principal: Breve, Fabricio [UNESP]
Data de Publicação: 2020
Outros Autores: Fischer, Carlos N. [UNESP], IEEE
Tipo de documento: Artigo de conferência
Idioma: eng
Título da fonte: Repositório Institucional da UNESP
Texto Completo: http://hdl.handle.net/11449/209251
Resumo: Navigation and mobility are some of the major problems faced by visually impaired people in their daily lives. Advances in computer vision led to the proposal of some navigation systems. However, most of them require expensive and/or heavy hardware. In this paper we propose the use of convolutional neural networks (CNN), transfer learning, and semi-supervised learning (SSL) to build a framework aimed at the visually impaired aid. It has low computational costs and, therefore, may be implemented on current smartphones, without relying on any additional equipment. The smartphone camera can be used to automatically take pictures of the path ahead. Then, they will be immediately classified, providing almost instantaneous feedback to the user. We also propose a dataset to train the classifiers, including indoor and outdoor situations with different types of light, floor, and obstacles. Many different CNN architectures are evaluated as feature extractors and classifiers, by fine-tuning weights pre-trained on a much larger dataset. The graph-based SSL method, known as particle competition and cooperation, is also used for classification, allowing feedback from the user to be incorporated without retraining the underlying network. 92% and 80% classification accuracy is achieved in the proposed dataset in the best supervised and SSL scenarios, respectively.
id UNSP_1b8d72a401c7b08d8ddccea0002b916b
oai_identifier_str oai:repositorio.unesp.br:11449/209251
network_acronym_str UNSP
network_name_str Repositório Institucional da UNESP
repository_id_str 2946
spelling Visually Impaired Aid using Convolutional Neural Networks, Transfer Learning, and Particle Competition and CooperationTransfer LearningParticle Competition and CooperationConvolutional Neural NetworksSemi-Supervised LearningNavigation and mobility are some of the major problems faced by visually impaired people in their daily lives. Advances in computer vision led to the proposal of some navigation systems. However, most of them require expensive and/or heavy hardware. In this paper we propose the use of convolutional neural networks (CNN), transfer learning, and semi-supervised learning (SSL) to build a framework aimed at the visually impaired aid. It has low computational costs and, therefore, may be implemented on current smartphones, without relying on any additional equipment. The smartphone camera can be used to automatically take pictures of the path ahead. Then, they will be immediately classified, providing almost instantaneous feedback to the user. We also propose a dataset to train the classifiers, including indoor and outdoor situations with different types of light, floor, and obstacles. Many different CNN architectures are evaluated as feature extractors and classifiers, by fine-tuning weights pre-trained on a much larger dataset. The graph-based SSL method, known as particle competition and cooperation, is also used for classification, allowing feedback from the user to be incorporated without retraining the underlying network. 92% and 80% classification accuracy is achieved in the proposed dataset in the best supervised and SSL scenarios, respectively.Fundação de Amparo à Pesquisa do Estado de São Paulo (FAPESP)Sao Paulo State Univ UNESP, Inst Geosci & Exact Sci, Rio Claro, SP, BrazilSao Paulo State Univ UNESP, Inst Geosci & Exact Sci, Rio Claro, SP, BrazilFAPESP: 2016/05669-4IeeeUniversidade Estadual Paulista (Unesp)Breve, Fabricio [UNESP]Fischer, Carlos N. [UNESP]IEEE2021-06-25T11:54:12Z2021-06-25T11:54:12Z2020-01-01info:eu-repo/semantics/publishedVersioninfo:eu-repo/semantics/conferenceObject82020 International Joint Conference On Neural Networks (ijcnn). New York: Ieee, 8 p., 2020.2161-4393http://hdl.handle.net/11449/209251WOS:000626021407116Web of Sciencereponame:Repositório Institucional da UNESPinstname:Universidade Estadual Paulista (UNESP)instacron:UNESPeng2020 International Joint Conference On Neural Networks (ijcnn)info:eu-repo/semantics/openAccess2021-10-23T19:23:40Zoai:repositorio.unesp.br:11449/209251Repositório InstitucionalPUBhttp://repositorio.unesp.br/oai/requestopendoar:29462024-08-05T13:41:24.894294Repositório Institucional da UNESP - Universidade Estadual Paulista (UNESP)false
dc.title.none.fl_str_mv Visually Impaired Aid using Convolutional Neural Networks, Transfer Learning, and Particle Competition and Cooperation
title Visually Impaired Aid using Convolutional Neural Networks, Transfer Learning, and Particle Competition and Cooperation
spellingShingle Visually Impaired Aid using Convolutional Neural Networks, Transfer Learning, and Particle Competition and Cooperation
Breve, Fabricio [UNESP]
Transfer Learning
Particle Competition and Cooperation
Convolutional Neural Networks
Semi-Supervised Learning
title_short Visually Impaired Aid using Convolutional Neural Networks, Transfer Learning, and Particle Competition and Cooperation
title_full Visually Impaired Aid using Convolutional Neural Networks, Transfer Learning, and Particle Competition and Cooperation
title_fullStr Visually Impaired Aid using Convolutional Neural Networks, Transfer Learning, and Particle Competition and Cooperation
title_full_unstemmed Visually Impaired Aid using Convolutional Neural Networks, Transfer Learning, and Particle Competition and Cooperation
title_sort Visually Impaired Aid using Convolutional Neural Networks, Transfer Learning, and Particle Competition and Cooperation
author Breve, Fabricio [UNESP]
author_facet Breve, Fabricio [UNESP]
Fischer, Carlos N. [UNESP]
IEEE
author_role author
author2 Fischer, Carlos N. [UNESP]
IEEE
author2_role author
author
dc.contributor.none.fl_str_mv Universidade Estadual Paulista (Unesp)
dc.contributor.author.fl_str_mv Breve, Fabricio [UNESP]
Fischer, Carlos N. [UNESP]
IEEE
dc.subject.por.fl_str_mv Transfer Learning
Particle Competition and Cooperation
Convolutional Neural Networks
Semi-Supervised Learning
topic Transfer Learning
Particle Competition and Cooperation
Convolutional Neural Networks
Semi-Supervised Learning
description Navigation and mobility are some of the major problems faced by visually impaired people in their daily lives. Advances in computer vision led to the proposal of some navigation systems. However, most of them require expensive and/or heavy hardware. In this paper we propose the use of convolutional neural networks (CNN), transfer learning, and semi-supervised learning (SSL) to build a framework aimed at the visually impaired aid. It has low computational costs and, therefore, may be implemented on current smartphones, without relying on any additional equipment. The smartphone camera can be used to automatically take pictures of the path ahead. Then, they will be immediately classified, providing almost instantaneous feedback to the user. We also propose a dataset to train the classifiers, including indoor and outdoor situations with different types of light, floor, and obstacles. Many different CNN architectures are evaluated as feature extractors and classifiers, by fine-tuning weights pre-trained on a much larger dataset. The graph-based SSL method, known as particle competition and cooperation, is also used for classification, allowing feedback from the user to be incorporated without retraining the underlying network. 92% and 80% classification accuracy is achieved in the proposed dataset in the best supervised and SSL scenarios, respectively.
publishDate 2020
dc.date.none.fl_str_mv 2020-01-01
2021-06-25T11:54:12Z
2021-06-25T11:54:12Z
dc.type.status.fl_str_mv info:eu-repo/semantics/publishedVersion
dc.type.driver.fl_str_mv info:eu-repo/semantics/conferenceObject
format conferenceObject
status_str publishedVersion
dc.identifier.uri.fl_str_mv 2020 International Joint Conference On Neural Networks (ijcnn). New York: Ieee, 8 p., 2020.
2161-4393
http://hdl.handle.net/11449/209251
WOS:000626021407116
identifier_str_mv 2020 International Joint Conference On Neural Networks (ijcnn). New York: Ieee, 8 p., 2020.
2161-4393
WOS:000626021407116
url http://hdl.handle.net/11449/209251
dc.language.iso.fl_str_mv eng
language eng
dc.relation.none.fl_str_mv 2020 International Joint Conference On Neural Networks (ijcnn)
dc.rights.driver.fl_str_mv info:eu-repo/semantics/openAccess
eu_rights_str_mv openAccess
dc.format.none.fl_str_mv 8
dc.publisher.none.fl_str_mv Ieee
publisher.none.fl_str_mv Ieee
dc.source.none.fl_str_mv Web of Science
reponame:Repositório Institucional da UNESP
instname:Universidade Estadual Paulista (UNESP)
instacron:UNESP
instname_str Universidade Estadual Paulista (UNESP)
instacron_str UNESP
institution UNESP
reponame_str Repositório Institucional da UNESP
collection Repositório Institucional da UNESP
repository.name.fl_str_mv Repositório Institucional da UNESP - Universidade Estadual Paulista (UNESP)
repository.mail.fl_str_mv
_version_ 1808128265780461568