Hadoop mapreduce tolerante a faltas bizantinas

Detalhes bibliográficos
Autor(a) principal: da Costa, Pedro Alexandre Reis Sá
Data de Publicação: 2011
Tipo de documento: Dissertação
Idioma: por
Título da fonte: Repositório Científico de Acesso Aberto de Portugal (Repositórios Cientìficos)
Texto Completo: http://hdl.handle.net/10451/13903
Resumo: MapReduce is often used to run critical jobs such as scientific data analysis. However, evidence in the literature shows that arbitrary faults do occur and can probably corrupt the results of MapReduce jobs. MapReduce runtimes like Hadoop tolerate crash faults, butnot arbitrary or Byzantine faults. In this work, it is presented a MapReduce algorithm andprototype that tolerate these faults. An experimental evaluation shows that the execution of a job with the implemented algorithm uses twice the resources of the original Hadoop,instead of the 3 or 4 times more that would be achieved with the direct application of common Byzantine fault-tolerance paradigms. It is believed that this cost is acceptable for critical applications that require that level of fault tolerance.
id RCAP_06f58a2b8faa64c3624e1a1af3634b1c
oai_identifier_str oai:repositorio.ul.pt:10451/13903
network_acronym_str RCAP
network_name_str Repositório Científico de Acesso Aberto de Portugal (Repositórios Cientìficos)
repository_id_str 7160
spelling Hadoop mapreduce tolerante a faltas bizantinasHadoop MapReducearbitrary faultsreplicationByzantine Fault-ToleranceMapReduce is often used to run critical jobs such as scientific data analysis. However, evidence in the literature shows that arbitrary faults do occur and can probably corrupt the results of MapReduce jobs. MapReduce runtimes like Hadoop tolerate crash faults, butnot arbitrary or Byzantine faults. In this work, it is presented a MapReduce algorithm andprototype that tolerate these faults. An experimental evaluation shows that the execution of a job with the implemented algorithm uses twice the resources of the original Hadoop,instead of the 3 or 4 times more that would be achieved with the direct application of common Byzantine fault-tolerance paradigms. It is believed that this cost is acceptable for critical applications that require that level of fault tolerance.Pasin, MarceloRepositório da Universidade de Lisboada Costa, Pedro Alexandre Reis Sá2012-02-13T15:05:24Z20112011-01-01T00:00:00Zinfo:eu-repo/semantics/publishedVersioninfo:eu-repo/semantics/masterThesisapplication/pdfhttp://hdl.handle.net/10451/13903porinfo:eu-repo/semantics/openAccessreponame:Repositório Científico de Acesso Aberto de Portugal (Repositórios Cientìficos)instname:Agência para a Sociedade do Conhecimento (UMIC) - FCT - Sociedade da Informaçãoinstacron:RCAAP2023-11-08T15:59:22Zoai:repositorio.ul.pt:10451/13903Portal AgregadorONGhttps://www.rcaap.pt/oai/openaireopendoar:71602024-03-19T21:35:50.255277Repositório Científico de Acesso Aberto de Portugal (Repositórios Cientìficos) - Agência para a Sociedade do Conhecimento (UMIC) - FCT - Sociedade da Informaçãofalse
dc.title.none.fl_str_mv Hadoop mapreduce tolerante a faltas bizantinas
title Hadoop mapreduce tolerante a faltas bizantinas
spellingShingle Hadoop mapreduce tolerante a faltas bizantinas
da Costa, Pedro Alexandre Reis Sá
Hadoop MapReduce
arbitrary faults
replication
Byzantine Fault-Tolerance
title_short Hadoop mapreduce tolerante a faltas bizantinas
title_full Hadoop mapreduce tolerante a faltas bizantinas
title_fullStr Hadoop mapreduce tolerante a faltas bizantinas
title_full_unstemmed Hadoop mapreduce tolerante a faltas bizantinas
title_sort Hadoop mapreduce tolerante a faltas bizantinas
author da Costa, Pedro Alexandre Reis Sá
author_facet da Costa, Pedro Alexandre Reis Sá
author_role author
dc.contributor.none.fl_str_mv Pasin, Marcelo
Repositório da Universidade de Lisboa
dc.contributor.author.fl_str_mv da Costa, Pedro Alexandre Reis Sá
dc.subject.por.fl_str_mv Hadoop MapReduce
arbitrary faults
replication
Byzantine Fault-Tolerance
topic Hadoop MapReduce
arbitrary faults
replication
Byzantine Fault-Tolerance
description MapReduce is often used to run critical jobs such as scientific data analysis. However, evidence in the literature shows that arbitrary faults do occur and can probably corrupt the results of MapReduce jobs. MapReduce runtimes like Hadoop tolerate crash faults, butnot arbitrary or Byzantine faults. In this work, it is presented a MapReduce algorithm andprototype that tolerate these faults. An experimental evaluation shows that the execution of a job with the implemented algorithm uses twice the resources of the original Hadoop,instead of the 3 or 4 times more that would be achieved with the direct application of common Byzantine fault-tolerance paradigms. It is believed that this cost is acceptable for critical applications that require that level of fault tolerance.
publishDate 2011
dc.date.none.fl_str_mv 2011
2011-01-01T00:00:00Z
2012-02-13T15:05:24Z
dc.type.status.fl_str_mv info:eu-repo/semantics/publishedVersion
dc.type.driver.fl_str_mv info:eu-repo/semantics/masterThesis
format masterThesis
status_str publishedVersion
dc.identifier.uri.fl_str_mv http://hdl.handle.net/10451/13903
url http://hdl.handle.net/10451/13903
dc.language.iso.fl_str_mv por
language por
dc.rights.driver.fl_str_mv info:eu-repo/semantics/openAccess
eu_rights_str_mv openAccess
dc.format.none.fl_str_mv application/pdf
dc.source.none.fl_str_mv reponame:Repositório Científico de Acesso Aberto de Portugal (Repositórios Cientìficos)
instname:Agência para a Sociedade do Conhecimento (UMIC) - FCT - Sociedade da Informação
instacron:RCAAP
instname_str Agência para a Sociedade do Conhecimento (UMIC) - FCT - Sociedade da Informação
instacron_str RCAAP
institution RCAAP
reponame_str Repositório Científico de Acesso Aberto de Portugal (Repositórios Cientìficos)
collection Repositório Científico de Acesso Aberto de Portugal (Repositórios Cientìficos)
repository.name.fl_str_mv Repositório Científico de Acesso Aberto de Portugal (Repositórios Cientìficos) - Agência para a Sociedade do Conhecimento (UMIC) - FCT - Sociedade da Informação
repository.mail.fl_str_mv
_version_ 1799134257537351680