Hadoop mapreduce tolerante a faltas bizantinas
Autor(a) principal: | |
---|---|
Data de Publicação: | 2011 |
Tipo de documento: | Dissertação |
Idioma: | por |
Título da fonte: | Repositório Científico de Acesso Aberto de Portugal (Repositórios Cientìficos) |
Texto Completo: | http://hdl.handle.net/10451/13903 |
Resumo: | MapReduce is often used to run critical jobs such as scientific data analysis. However, evidence in the literature shows that arbitrary faults do occur and can probably corrupt the results of MapReduce jobs. MapReduce runtimes like Hadoop tolerate crash faults, butnot arbitrary or Byzantine faults. In this work, it is presented a MapReduce algorithm andprototype that tolerate these faults. An experimental evaluation shows that the execution of a job with the implemented algorithm uses twice the resources of the original Hadoop,instead of the 3 or 4 times more that would be achieved with the direct application of common Byzantine fault-tolerance paradigms. It is believed that this cost is acceptable for critical applications that require that level of fault tolerance. |
id |
RCAP_06f58a2b8faa64c3624e1a1af3634b1c |
---|---|
oai_identifier_str |
oai:repositorio.ul.pt:10451/13903 |
network_acronym_str |
RCAP |
network_name_str |
Repositório Científico de Acesso Aberto de Portugal (Repositórios Cientìficos) |
repository_id_str |
7160 |
spelling |
Hadoop mapreduce tolerante a faltas bizantinasHadoop MapReducearbitrary faultsreplicationByzantine Fault-ToleranceMapReduce is often used to run critical jobs such as scientific data analysis. However, evidence in the literature shows that arbitrary faults do occur and can probably corrupt the results of MapReduce jobs. MapReduce runtimes like Hadoop tolerate crash faults, butnot arbitrary or Byzantine faults. In this work, it is presented a MapReduce algorithm andprototype that tolerate these faults. An experimental evaluation shows that the execution of a job with the implemented algorithm uses twice the resources of the original Hadoop,instead of the 3 or 4 times more that would be achieved with the direct application of common Byzantine fault-tolerance paradigms. It is believed that this cost is acceptable for critical applications that require that level of fault tolerance.Pasin, MarceloRepositório da Universidade de Lisboada Costa, Pedro Alexandre Reis Sá2012-02-13T15:05:24Z20112011-01-01T00:00:00Zinfo:eu-repo/semantics/publishedVersioninfo:eu-repo/semantics/masterThesisapplication/pdfhttp://hdl.handle.net/10451/13903porinfo:eu-repo/semantics/openAccessreponame:Repositório Científico de Acesso Aberto de Portugal (Repositórios Cientìficos)instname:Agência para a Sociedade do Conhecimento (UMIC) - FCT - Sociedade da Informaçãoinstacron:RCAAP2023-11-08T15:59:22Zoai:repositorio.ul.pt:10451/13903Portal AgregadorONGhttps://www.rcaap.pt/oai/openaireopendoar:71602024-03-19T21:35:50.255277Repositório Científico de Acesso Aberto de Portugal (Repositórios Cientìficos) - Agência para a Sociedade do Conhecimento (UMIC) - FCT - Sociedade da Informaçãofalse |
dc.title.none.fl_str_mv |
Hadoop mapreduce tolerante a faltas bizantinas |
title |
Hadoop mapreduce tolerante a faltas bizantinas |
spellingShingle |
Hadoop mapreduce tolerante a faltas bizantinas da Costa, Pedro Alexandre Reis Sá Hadoop MapReduce arbitrary faults replication Byzantine Fault-Tolerance |
title_short |
Hadoop mapreduce tolerante a faltas bizantinas |
title_full |
Hadoop mapreduce tolerante a faltas bizantinas |
title_fullStr |
Hadoop mapreduce tolerante a faltas bizantinas |
title_full_unstemmed |
Hadoop mapreduce tolerante a faltas bizantinas |
title_sort |
Hadoop mapreduce tolerante a faltas bizantinas |
author |
da Costa, Pedro Alexandre Reis Sá |
author_facet |
da Costa, Pedro Alexandre Reis Sá |
author_role |
author |
dc.contributor.none.fl_str_mv |
Pasin, Marcelo Repositório da Universidade de Lisboa |
dc.contributor.author.fl_str_mv |
da Costa, Pedro Alexandre Reis Sá |
dc.subject.por.fl_str_mv |
Hadoop MapReduce arbitrary faults replication Byzantine Fault-Tolerance |
topic |
Hadoop MapReduce arbitrary faults replication Byzantine Fault-Tolerance |
description |
MapReduce is often used to run critical jobs such as scientific data analysis. However, evidence in the literature shows that arbitrary faults do occur and can probably corrupt the results of MapReduce jobs. MapReduce runtimes like Hadoop tolerate crash faults, butnot arbitrary or Byzantine faults. In this work, it is presented a MapReduce algorithm andprototype that tolerate these faults. An experimental evaluation shows that the execution of a job with the implemented algorithm uses twice the resources of the original Hadoop,instead of the 3 or 4 times more that would be achieved with the direct application of common Byzantine fault-tolerance paradigms. It is believed that this cost is acceptable for critical applications that require that level of fault tolerance. |
publishDate |
2011 |
dc.date.none.fl_str_mv |
2011 2011-01-01T00:00:00Z 2012-02-13T15:05:24Z |
dc.type.status.fl_str_mv |
info:eu-repo/semantics/publishedVersion |
dc.type.driver.fl_str_mv |
info:eu-repo/semantics/masterThesis |
format |
masterThesis |
status_str |
publishedVersion |
dc.identifier.uri.fl_str_mv |
http://hdl.handle.net/10451/13903 |
url |
http://hdl.handle.net/10451/13903 |
dc.language.iso.fl_str_mv |
por |
language |
por |
dc.rights.driver.fl_str_mv |
info:eu-repo/semantics/openAccess |
eu_rights_str_mv |
openAccess |
dc.format.none.fl_str_mv |
application/pdf |
dc.source.none.fl_str_mv |
reponame:Repositório Científico de Acesso Aberto de Portugal (Repositórios Cientìficos) instname:Agência para a Sociedade do Conhecimento (UMIC) - FCT - Sociedade da Informação instacron:RCAAP |
instname_str |
Agência para a Sociedade do Conhecimento (UMIC) - FCT - Sociedade da Informação |
instacron_str |
RCAAP |
institution |
RCAAP |
reponame_str |
Repositório Científico de Acesso Aberto de Portugal (Repositórios Cientìficos) |
collection |
Repositório Científico de Acesso Aberto de Portugal (Repositórios Cientìficos) |
repository.name.fl_str_mv |
Repositório Científico de Acesso Aberto de Portugal (Repositórios Cientìficos) - Agência para a Sociedade do Conhecimento (UMIC) - FCT - Sociedade da Informação |
repository.mail.fl_str_mv |
|
_version_ |
1799134257537351680 |