Visually Impaired Aid using Convolutional Neural Networks, Transfer Learning, and Particle Competition and Cooperation
Autor(a) principal: | |
---|---|
Data de Publicação: | 2020 |
Outros Autores: | , |
Tipo de documento: | Artigo de conferência |
Idioma: | eng |
Título da fonte: | Repositório Institucional da UNESP |
Texto Completo: | http://hdl.handle.net/11449/209251 |
Resumo: | Navigation and mobility are some of the major problems faced by visually impaired people in their daily lives. Advances in computer vision led to the proposal of some navigation systems. However, most of them require expensive and/or heavy hardware. In this paper we propose the use of convolutional neural networks (CNN), transfer learning, and semi-supervised learning (SSL) to build a framework aimed at the visually impaired aid. It has low computational costs and, therefore, may be implemented on current smartphones, without relying on any additional equipment. The smartphone camera can be used to automatically take pictures of the path ahead. Then, they will be immediately classified, providing almost instantaneous feedback to the user. We also propose a dataset to train the classifiers, including indoor and outdoor situations with different types of light, floor, and obstacles. Many different CNN architectures are evaluated as feature extractors and classifiers, by fine-tuning weights pre-trained on a much larger dataset. The graph-based SSL method, known as particle competition and cooperation, is also used for classification, allowing feedback from the user to be incorporated without retraining the underlying network. 92% and 80% classification accuracy is achieved in the proposed dataset in the best supervised and SSL scenarios, respectively. |
id |
UNSP_1b8d72a401c7b08d8ddccea0002b916b |
---|---|
oai_identifier_str |
oai:repositorio.unesp.br:11449/209251 |
network_acronym_str |
UNSP |
network_name_str |
Repositório Institucional da UNESP |
repository_id_str |
2946 |
spelling |
Visually Impaired Aid using Convolutional Neural Networks, Transfer Learning, and Particle Competition and CooperationTransfer LearningParticle Competition and CooperationConvolutional Neural NetworksSemi-Supervised LearningNavigation and mobility are some of the major problems faced by visually impaired people in their daily lives. Advances in computer vision led to the proposal of some navigation systems. However, most of them require expensive and/or heavy hardware. In this paper we propose the use of convolutional neural networks (CNN), transfer learning, and semi-supervised learning (SSL) to build a framework aimed at the visually impaired aid. It has low computational costs and, therefore, may be implemented on current smartphones, without relying on any additional equipment. The smartphone camera can be used to automatically take pictures of the path ahead. Then, they will be immediately classified, providing almost instantaneous feedback to the user. We also propose a dataset to train the classifiers, including indoor and outdoor situations with different types of light, floor, and obstacles. Many different CNN architectures are evaluated as feature extractors and classifiers, by fine-tuning weights pre-trained on a much larger dataset. The graph-based SSL method, known as particle competition and cooperation, is also used for classification, allowing feedback from the user to be incorporated without retraining the underlying network. 92% and 80% classification accuracy is achieved in the proposed dataset in the best supervised and SSL scenarios, respectively.Fundação de Amparo à Pesquisa do Estado de São Paulo (FAPESP)Sao Paulo State Univ UNESP, Inst Geosci & Exact Sci, Rio Claro, SP, BrazilSao Paulo State Univ UNESP, Inst Geosci & Exact Sci, Rio Claro, SP, BrazilFAPESP: 2016/05669-4IeeeUniversidade Estadual Paulista (Unesp)Breve, Fabricio [UNESP]Fischer, Carlos N. [UNESP]IEEE2021-06-25T11:54:12Z2021-06-25T11:54:12Z2020-01-01info:eu-repo/semantics/publishedVersioninfo:eu-repo/semantics/conferenceObject82020 International Joint Conference On Neural Networks (ijcnn). New York: Ieee, 8 p., 2020.2161-4393http://hdl.handle.net/11449/209251WOS:000626021407116Web of Sciencereponame:Repositório Institucional da UNESPinstname:Universidade Estadual Paulista (UNESP)instacron:UNESPeng2020 International Joint Conference On Neural Networks (ijcnn)info:eu-repo/semantics/openAccess2021-10-23T19:23:40Zoai:repositorio.unesp.br:11449/209251Repositório InstitucionalPUBhttp://repositorio.unesp.br/oai/requestopendoar:29462024-08-05T13:41:24.894294Repositório Institucional da UNESP - Universidade Estadual Paulista (UNESP)false |
dc.title.none.fl_str_mv |
Visually Impaired Aid using Convolutional Neural Networks, Transfer Learning, and Particle Competition and Cooperation |
title |
Visually Impaired Aid using Convolutional Neural Networks, Transfer Learning, and Particle Competition and Cooperation |
spellingShingle |
Visually Impaired Aid using Convolutional Neural Networks, Transfer Learning, and Particle Competition and Cooperation Breve, Fabricio [UNESP] Transfer Learning Particle Competition and Cooperation Convolutional Neural Networks Semi-Supervised Learning |
title_short |
Visually Impaired Aid using Convolutional Neural Networks, Transfer Learning, and Particle Competition and Cooperation |
title_full |
Visually Impaired Aid using Convolutional Neural Networks, Transfer Learning, and Particle Competition and Cooperation |
title_fullStr |
Visually Impaired Aid using Convolutional Neural Networks, Transfer Learning, and Particle Competition and Cooperation |
title_full_unstemmed |
Visually Impaired Aid using Convolutional Neural Networks, Transfer Learning, and Particle Competition and Cooperation |
title_sort |
Visually Impaired Aid using Convolutional Neural Networks, Transfer Learning, and Particle Competition and Cooperation |
author |
Breve, Fabricio [UNESP] |
author_facet |
Breve, Fabricio [UNESP] Fischer, Carlos N. [UNESP] IEEE |
author_role |
author |
author2 |
Fischer, Carlos N. [UNESP] IEEE |
author2_role |
author author |
dc.contributor.none.fl_str_mv |
Universidade Estadual Paulista (Unesp) |
dc.contributor.author.fl_str_mv |
Breve, Fabricio [UNESP] Fischer, Carlos N. [UNESP] IEEE |
dc.subject.por.fl_str_mv |
Transfer Learning Particle Competition and Cooperation Convolutional Neural Networks Semi-Supervised Learning |
topic |
Transfer Learning Particle Competition and Cooperation Convolutional Neural Networks Semi-Supervised Learning |
description |
Navigation and mobility are some of the major problems faced by visually impaired people in their daily lives. Advances in computer vision led to the proposal of some navigation systems. However, most of them require expensive and/or heavy hardware. In this paper we propose the use of convolutional neural networks (CNN), transfer learning, and semi-supervised learning (SSL) to build a framework aimed at the visually impaired aid. It has low computational costs and, therefore, may be implemented on current smartphones, without relying on any additional equipment. The smartphone camera can be used to automatically take pictures of the path ahead. Then, they will be immediately classified, providing almost instantaneous feedback to the user. We also propose a dataset to train the classifiers, including indoor and outdoor situations with different types of light, floor, and obstacles. Many different CNN architectures are evaluated as feature extractors and classifiers, by fine-tuning weights pre-trained on a much larger dataset. The graph-based SSL method, known as particle competition and cooperation, is also used for classification, allowing feedback from the user to be incorporated without retraining the underlying network. 92% and 80% classification accuracy is achieved in the proposed dataset in the best supervised and SSL scenarios, respectively. |
publishDate |
2020 |
dc.date.none.fl_str_mv |
2020-01-01 2021-06-25T11:54:12Z 2021-06-25T11:54:12Z |
dc.type.status.fl_str_mv |
info:eu-repo/semantics/publishedVersion |
dc.type.driver.fl_str_mv |
info:eu-repo/semantics/conferenceObject |
format |
conferenceObject |
status_str |
publishedVersion |
dc.identifier.uri.fl_str_mv |
2020 International Joint Conference On Neural Networks (ijcnn). New York: Ieee, 8 p., 2020. 2161-4393 http://hdl.handle.net/11449/209251 WOS:000626021407116 |
identifier_str_mv |
2020 International Joint Conference On Neural Networks (ijcnn). New York: Ieee, 8 p., 2020. 2161-4393 WOS:000626021407116 |
url |
http://hdl.handle.net/11449/209251 |
dc.language.iso.fl_str_mv |
eng |
language |
eng |
dc.relation.none.fl_str_mv |
2020 International Joint Conference On Neural Networks (ijcnn) |
dc.rights.driver.fl_str_mv |
info:eu-repo/semantics/openAccess |
eu_rights_str_mv |
openAccess |
dc.format.none.fl_str_mv |
8 |
dc.publisher.none.fl_str_mv |
Ieee |
publisher.none.fl_str_mv |
Ieee |
dc.source.none.fl_str_mv |
Web of Science reponame:Repositório Institucional da UNESP instname:Universidade Estadual Paulista (UNESP) instacron:UNESP |
instname_str |
Universidade Estadual Paulista (UNESP) |
instacron_str |
UNESP |
institution |
UNESP |
reponame_str |
Repositório Institucional da UNESP |
collection |
Repositório Institucional da UNESP |
repository.name.fl_str_mv |
Repositório Institucional da UNESP - Universidade Estadual Paulista (UNESP) |
repository.mail.fl_str_mv |
|
_version_ |
1808128265780461568 |