KinectFusion for Faces: Real-Time 3D Face Tracking and Modeling Using a Kinect Camera for a Markerless AR System
Autor(a) principal: | |
---|---|
Data de Publicação: | 2014 |
Outros Autores: | , |
Tipo de documento: | Artigo |
Idioma: | eng |
Título da fonte: | Journal on Interactive Systems |
Texto Completo: | https://sol.sbc.org.br/journals/index.php/jis/article/view/630 |
Resumo: | In this paper we present an extension to the KinectFusion algorithm that allows a robust real-time face tracking and modeling using the Microsoft’s Kinect sensor. This is achieved changing two steps of the original algorithm: pre-processing and tracking. In the former, we use a real-time face detection algorithm to segment the face from the rest of the image. In the latter, we use a real-time head pose estimation to give a new initial guess to the Iterative Closest Point (ICP) algorithm when it fails and an algorithm to solve occlusion.Our approach is evaluated in a markerless augmented reality (MAR) system. We show that this approach can reconstruct faces and handle more face pose changes and variations than the original KinectFusion’s tracking algorithm. In addition, we show that the realism of the system is enhanced as we solve the occlusion problem efficiently at shader level. |
id |
SBC-3_d954b7a9d1459282336dcea2902d443b |
---|---|
oai_identifier_str |
oai:ojs2.sol.sbc.org.br:article/630 |
network_acronym_str |
SBC-3 |
network_name_str |
Journal on Interactive Systems |
repository_id_str |
|
spelling |
KinectFusion for Faces: Real-Time 3D Face Tracking and Modeling Using a Kinect Camera for a Markerless AR SystemIn this paper we present an extension to the KinectFusion algorithm that allows a robust real-time face tracking and modeling using the Microsoft’s Kinect sensor. This is achieved changing two steps of the original algorithm: pre-processing and tracking. In the former, we use a real-time face detection algorithm to segment the face from the rest of the image. In the latter, we use a real-time head pose estimation to give a new initial guess to the Iterative Closest Point (ICP) algorithm when it fails and an algorithm to solve occlusion.Our approach is evaluated in a markerless augmented reality (MAR) system. We show that this approach can reconstruct faces and handle more face pose changes and variations than the original KinectFusion’s tracking algorithm. In addition, we show that the realism of the system is enhanced as we solve the occlusion problem efficiently at shader level.Nenhum resumo disponívelBrazilian Computer Society2014-01-29info:eu-repo/semantics/articleinfo:eu-repo/semantics/publishedVersionapplication/pdfhttps://sol.sbc.org.br/journals/index.php/jis/article/view/63010.5753/jis.2013.630Journal of Interactive Systems; v. 4 n. 2 (2013)Journal on Interactive Systems; Vol. 4 No. 2 (2013)2763-7719reponame:Journal on Interactive Systemsinstname:Sociedade Brasileira de Computação (SBC)instacron:SBCenghttps://sol.sbc.org.br/journals/index.php/jis/article/view/630/625Macedo, Márcio Cerqueira de FariasApolinário Jr., Antônio LopesSouza, Antonio Carlos dos Santosinfo:eu-repo/semantics/openAccess2020-09-05T16:11:10Zoai:ojs2.sol.sbc.org.br:article/630Revistahttps://sol.sbc.org.br/journals/index.php/jis/ONGhttps://sol.sbc.org.br/journals/index.php/jis/oaijis@sbc.org.br2763-77192763-7719opendoar:2020-09-05T16:11:10Journal on Interactive Systems - Sociedade Brasileira de Computação (SBC)false |
dc.title.none.fl_str_mv |
KinectFusion for Faces: Real-Time 3D Face Tracking and Modeling Using a Kinect Camera for a Markerless AR System |
title |
KinectFusion for Faces: Real-Time 3D Face Tracking and Modeling Using a Kinect Camera for a Markerless AR System |
spellingShingle |
KinectFusion for Faces: Real-Time 3D Face Tracking and Modeling Using a Kinect Camera for a Markerless AR System Macedo, Márcio Cerqueira de Farias |
title_short |
KinectFusion for Faces: Real-Time 3D Face Tracking and Modeling Using a Kinect Camera for a Markerless AR System |
title_full |
KinectFusion for Faces: Real-Time 3D Face Tracking and Modeling Using a Kinect Camera for a Markerless AR System |
title_fullStr |
KinectFusion for Faces: Real-Time 3D Face Tracking and Modeling Using a Kinect Camera for a Markerless AR System |
title_full_unstemmed |
KinectFusion for Faces: Real-Time 3D Face Tracking and Modeling Using a Kinect Camera for a Markerless AR System |
title_sort |
KinectFusion for Faces: Real-Time 3D Face Tracking and Modeling Using a Kinect Camera for a Markerless AR System |
author |
Macedo, Márcio Cerqueira de Farias |
author_facet |
Macedo, Márcio Cerqueira de Farias Apolinário Jr., Antônio Lopes Souza, Antonio Carlos dos Santos |
author_role |
author |
author2 |
Apolinário Jr., Antônio Lopes Souza, Antonio Carlos dos Santos |
author2_role |
author author |
dc.contributor.author.fl_str_mv |
Macedo, Márcio Cerqueira de Farias Apolinário Jr., Antônio Lopes Souza, Antonio Carlos dos Santos |
description |
In this paper we present an extension to the KinectFusion algorithm that allows a robust real-time face tracking and modeling using the Microsoft’s Kinect sensor. This is achieved changing two steps of the original algorithm: pre-processing and tracking. In the former, we use a real-time face detection algorithm to segment the face from the rest of the image. In the latter, we use a real-time head pose estimation to give a new initial guess to the Iterative Closest Point (ICP) algorithm when it fails and an algorithm to solve occlusion.Our approach is evaluated in a markerless augmented reality (MAR) system. We show that this approach can reconstruct faces and handle more face pose changes and variations than the original KinectFusion’s tracking algorithm. In addition, we show that the realism of the system is enhanced as we solve the occlusion problem efficiently at shader level. |
publishDate |
2014 |
dc.date.none.fl_str_mv |
2014-01-29 |
dc.type.driver.fl_str_mv |
info:eu-repo/semantics/article info:eu-repo/semantics/publishedVersion |
format |
article |
status_str |
publishedVersion |
dc.identifier.uri.fl_str_mv |
https://sol.sbc.org.br/journals/index.php/jis/article/view/630 10.5753/jis.2013.630 |
url |
https://sol.sbc.org.br/journals/index.php/jis/article/view/630 |
identifier_str_mv |
10.5753/jis.2013.630 |
dc.language.iso.fl_str_mv |
eng |
language |
eng |
dc.relation.none.fl_str_mv |
https://sol.sbc.org.br/journals/index.php/jis/article/view/630/625 |
dc.rights.driver.fl_str_mv |
info:eu-repo/semantics/openAccess |
eu_rights_str_mv |
openAccess |
dc.format.none.fl_str_mv |
application/pdf |
dc.publisher.none.fl_str_mv |
Brazilian Computer Society |
publisher.none.fl_str_mv |
Brazilian Computer Society |
dc.source.none.fl_str_mv |
Journal of Interactive Systems; v. 4 n. 2 (2013) Journal on Interactive Systems; Vol. 4 No. 2 (2013) 2763-7719 reponame:Journal on Interactive Systems instname:Sociedade Brasileira de Computação (SBC) instacron:SBC |
instname_str |
Sociedade Brasileira de Computação (SBC) |
instacron_str |
SBC |
institution |
SBC |
reponame_str |
Journal on Interactive Systems |
collection |
Journal on Interactive Systems |
repository.name.fl_str_mv |
Journal on Interactive Systems - Sociedade Brasileira de Computação (SBC) |
repository.mail.fl_str_mv |
jis@sbc.org.br |
_version_ |
1796797410676047872 |