Mechanisms to improve fuzz testing for message brokers

Detalhes bibliográficos
Autor(a) principal: Rodriguez, Luis Gustavo Araujo
Data de Publicação: 2023
Tipo de documento: Tese
Idioma: eng
Título da fonte: Biblioteca Digital de Teses e Dissertações da USP
Texto Completo: https://www.teses.usp.br/teses/disponiveis/45/45134/tde-20122023-223756/
Resumo: Publish/subscribe (or pub/sub) protocols have played a key role in the success of the Internet of Things. As the Internet of Things expands to new users and environments, the need to better test pub/sub protocols becomes even more pressing. However, pub/sub protocol testing is an under-explored field, with few research studies examining effective strategies to increase the reliability and robustness of message brokers against malformed packets. Considering that several bugs discovered in message brokers are related to malformed packets, fuzz testing (or fuzzing) has emerged as one of the most promising, necessary, and ideal techniques to mitigate this issue. However, fuzzing faces many challenges when applied to pub/sub protocols, which distinguish themselves from other network-based systems by their message-publishing features and event-driven architecture. This poses the question as to whether developers and existing fuzz testing tools (or fuzzers) consider the unique attributes of the pub/sub messaging pattern. The objective of this PhD thesis is to study and develop effective fuzzing strategies for pub/sub protocols, aiming at contributing to the development of more robust applications in IoT and Smart Cities. According to the research findings, there is a lack of systematic approaches in the literature to fuzz-test pub/sub protocols. Furthermore, MQTT stands out as the most popular pub/sub protocol for which developers have proposed fuzzing techniques in the literature. However, as MQTT is the most widely-used pub/sub protocol, it provides an opportunity to understand the requirements and strategies to effectively fuzz a pub/sub protocol. This PhD research was divided into three phases. In the first phase, it was analyzed whether a systematic testing approach such as grammar-based fuzzing can be applied to a pub/sub protocol such as MQTT, thereby understanding the challenges and necessary requirements. A grammar-based methodology and architecture was therefore conceived and proposed for a pub/sub protocol. The end result is a fuzzer called MQTTGRAM, which was then compared with two other fuzzing approaches and outperformed both of them, despite exchanging up to 9x fewer packets. In the second phase, a taxonomy was developed that classifies all existing fuzzing techniques for MQTT, six of which were evaluated under equivalent conditions in order to determine whether developers are considering the unique attributes of the pub/sub design pattern. Furthermore, the fuzzers were evaluated in terms of their resource usage or stress-testing capabilities. In the third phase, MQTTGRAM was improved by incorporating three essential elements for pub/sub protocol fuzzing, which are lacking across all fuzzing techniques proposed for MQTT: two-way communication; topic awareness; and version support. Overall, this PhD research provides three main contributions: (1) the development and refinement of a grammar-based fuzzing approach for a pub/sub protocol; (2) taxonomy and performance evaluation of MQTT fuzzers under equivalent conditions; and (3) identification of shortcomings for future work.
id USP_ffe5b6b0ee46e6af2a404c5ec557dd26
oai_identifier_str oai:teses.usp.br:tde-20122023-223756
network_acronym_str USP
network_name_str Biblioteca Digital de Teses e Dissertações da USP
repository_id_str 2721
spelling Mechanisms to improve fuzz testing for message brokersMecanismos para melhorar testes fuzzing em brokers de mensagensBroker de mensagensFuzzingFuzzingIoTIoTMessage brokerMQTTMQTTNetwork protocolsProtocolos de redePublicação-assinaturaPublish-subscribeTestesTestingPublish/subscribe (or pub/sub) protocols have played a key role in the success of the Internet of Things. As the Internet of Things expands to new users and environments, the need to better test pub/sub protocols becomes even more pressing. However, pub/sub protocol testing is an under-explored field, with few research studies examining effective strategies to increase the reliability and robustness of message brokers against malformed packets. Considering that several bugs discovered in message brokers are related to malformed packets, fuzz testing (or fuzzing) has emerged as one of the most promising, necessary, and ideal techniques to mitigate this issue. However, fuzzing faces many challenges when applied to pub/sub protocols, which distinguish themselves from other network-based systems by their message-publishing features and event-driven architecture. This poses the question as to whether developers and existing fuzz testing tools (or fuzzers) consider the unique attributes of the pub/sub messaging pattern. The objective of this PhD thesis is to study and develop effective fuzzing strategies for pub/sub protocols, aiming at contributing to the development of more robust applications in IoT and Smart Cities. According to the research findings, there is a lack of systematic approaches in the literature to fuzz-test pub/sub protocols. Furthermore, MQTT stands out as the most popular pub/sub protocol for which developers have proposed fuzzing techniques in the literature. However, as MQTT is the most widely-used pub/sub protocol, it provides an opportunity to understand the requirements and strategies to effectively fuzz a pub/sub protocol. This PhD research was divided into three phases. In the first phase, it was analyzed whether a systematic testing approach such as grammar-based fuzzing can be applied to a pub/sub protocol such as MQTT, thereby understanding the challenges and necessary requirements. A grammar-based methodology and architecture was therefore conceived and proposed for a pub/sub protocol. The end result is a fuzzer called MQTTGRAM, which was then compared with two other fuzzing approaches and outperformed both of them, despite exchanging up to 9x fewer packets. In the second phase, a taxonomy was developed that classifies all existing fuzzing techniques for MQTT, six of which were evaluated under equivalent conditions in order to determine whether developers are considering the unique attributes of the pub/sub design pattern. Furthermore, the fuzzers were evaluated in terms of their resource usage or stress-testing capabilities. In the third phase, MQTTGRAM was improved by incorporating three essential elements for pub/sub protocol fuzzing, which are lacking across all fuzzing techniques proposed for MQTT: two-way communication; topic awareness; and version support. Overall, this PhD research provides three main contributions: (1) the development and refinement of a grammar-based fuzzing approach for a pub/sub protocol; (2) taxonomy and performance evaluation of MQTT fuzzers under equivalent conditions; and (3) identification of shortcomings for future work.Os protocolos de publicação-assinatura têm desempenhado um papel fundamental no sucesso da Internet das Coisas. À medida que a Internet das Coisas se expande para novos usuários e ambientes, a necessidade de testar melhor os protocolos de publicação-assinatura torna-se ainda maior. No entanto, a área de testes para protocolos publicação-assinatura é pouco explorada, com poucos estudos que examinam estratégias eficazes para aumentar a confiabilidade e a robustez de brokers de mensagens contra pacotes malformados. Considerando que diversas falhas descobertas nos brokers são por causa de pacotes malformados, testes baseados em fuzzing surgiram como uma das técnicas mais importantes para mitigar esse problema. No entanto, o fuzzing enfrenta muitos desafios quando aplicado aos protocolos de publicação-assinatura, que se distinguem de outros protocolos por sua funcionalidade de publicação de mensagens e arquitetura orientada a eventos. Isso levanta a questão sobre se os desenvolvedores e as ferramentas baseadas em fuzzing (ou fuzzers) estão considerando os atributos exclusivos do padrão publicação-assinatura na hora de realizar os testes. O objetivo desta tese de doutorado é apresentar estratégias eficazes de fuzzing para os protocolos de publicação-assinatura, com o objetivo de contribuir para o desenvolvimento de aplicações mais robustas na Internet das Coisas e Cidades Inteligentes. De acordo com as pesquisas preliminares, há uma falta de abordagens sistemáticas baseadas em fuzzing na literatura para testar os protocolos de publicação-assinatura. Além disso, MQTT se destaca como o protocolo de publicação-assinatura mais popular para o qual os desenvolvedores propuseram técnicas de fuzzing na literatura. Por tanto, MQTT oferece uma oportunidade de entender os requisitos e estratégias para testar efetivamente um protocolo de publicação-assinatura. Esta pesquisa de doutorado foi dividida em três fases. Na primeira fase, foi analisado se uma abordagem de fuzzing baseada em gramática pode ser aplicada a um protocolo de publicação-assinatura, entendendo assim os desafios e requisitos necessários. Assim, foi proposta uma metodologia e arquitetura para desenvolver um fuzzer baseado em gramática para testar um protocolo de publicação-assinatura. O resultado final é um fuzzer chamado MQTTGRAM, que foi então comparado com duas outras abordagens de fuzzing, e superou ambas, apesar de realizar menos testes. Na segunda fase, foi desenvolvida uma taxonomia que classifica todas as técnicas de fuzzing existentes para MQTT, das quais seis foram avaliadas em condições equivalentes para determinar se os desenvolvedores estão considerando os atributos exclusivos do padrão publicação-assinatura na hora de realizar os testes. Além disso, os fuzzers para MQTT foram avaliados em termos de testes de estresse. Na terceira fase, MQTTGRAM foi aprimorado, incorporando três elementos essenciais para testar os protocolos de publicação-assinatura: comunicação bidirecional; conhecimento de tópicos; e suporte a múltiplas versões. Esta pesquisa de doutorado fornece três contribuições principais: (1) o desenvolvimento e aprimoramento de uma abordagem de fuzzing baseada em gramática para um protocolo de publicação-assinatura; (2) taxonomia e avaliação de desempenho de fuzzers para MQTT em condições equivalentes; e (3) identificação de deficiências dos fuzzers para trabalhos futuros.Biblioteca Digitais de Teses e Dissertações da USPBatista, Daniel MacedoRodriguez, Luis Gustavo Araujo2023-09-01info:eu-repo/semantics/publishedVersioninfo:eu-repo/semantics/doctoralThesisapplication/pdfhttps://www.teses.usp.br/teses/disponiveis/45/45134/tde-20122023-223756/reponame:Biblioteca Digital de Teses e Dissertações da USPinstname:Universidade de São Paulo (USP)instacron:USPLiberar o conteúdo para acesso público.info:eu-repo/semantics/openAccesseng2024-03-22T19:42:02Zoai:teses.usp.br:tde-20122023-223756Biblioteca Digital de Teses e Dissertaçõeshttp://www.teses.usp.br/PUBhttp://www.teses.usp.br/cgi-bin/mtd2br.plvirginia@if.usp.br|| atendimento@aguia.usp.br||virginia@if.usp.bropendoar:27212024-03-22T19:42:02Biblioteca Digital de Teses e Dissertações da USP - Universidade de São Paulo (USP)false
dc.title.none.fl_str_mv Mechanisms to improve fuzz testing for message brokers
Mecanismos para melhorar testes fuzzing em brokers de mensagens
title Mechanisms to improve fuzz testing for message brokers
spellingShingle Mechanisms to improve fuzz testing for message brokers
Rodriguez, Luis Gustavo Araujo
Broker de mensagens
Fuzzing
Fuzzing
IoT
IoT
Message broker
MQTT
MQTT
Network protocols
Protocolos de rede
Publicação-assinatura
Publish-subscribe
Testes
Testing
title_short Mechanisms to improve fuzz testing for message brokers
title_full Mechanisms to improve fuzz testing for message brokers
title_fullStr Mechanisms to improve fuzz testing for message brokers
title_full_unstemmed Mechanisms to improve fuzz testing for message brokers
title_sort Mechanisms to improve fuzz testing for message brokers
author Rodriguez, Luis Gustavo Araujo
author_facet Rodriguez, Luis Gustavo Araujo
author_role author
dc.contributor.none.fl_str_mv Batista, Daniel Macedo
dc.contributor.author.fl_str_mv Rodriguez, Luis Gustavo Araujo
dc.subject.por.fl_str_mv Broker de mensagens
Fuzzing
Fuzzing
IoT
IoT
Message broker
MQTT
MQTT
Network protocols
Protocolos de rede
Publicação-assinatura
Publish-subscribe
Testes
Testing
topic Broker de mensagens
Fuzzing
Fuzzing
IoT
IoT
Message broker
MQTT
MQTT
Network protocols
Protocolos de rede
Publicação-assinatura
Publish-subscribe
Testes
Testing
description Publish/subscribe (or pub/sub) protocols have played a key role in the success of the Internet of Things. As the Internet of Things expands to new users and environments, the need to better test pub/sub protocols becomes even more pressing. However, pub/sub protocol testing is an under-explored field, with few research studies examining effective strategies to increase the reliability and robustness of message brokers against malformed packets. Considering that several bugs discovered in message brokers are related to malformed packets, fuzz testing (or fuzzing) has emerged as one of the most promising, necessary, and ideal techniques to mitigate this issue. However, fuzzing faces many challenges when applied to pub/sub protocols, which distinguish themselves from other network-based systems by their message-publishing features and event-driven architecture. This poses the question as to whether developers and existing fuzz testing tools (or fuzzers) consider the unique attributes of the pub/sub messaging pattern. The objective of this PhD thesis is to study and develop effective fuzzing strategies for pub/sub protocols, aiming at contributing to the development of more robust applications in IoT and Smart Cities. According to the research findings, there is a lack of systematic approaches in the literature to fuzz-test pub/sub protocols. Furthermore, MQTT stands out as the most popular pub/sub protocol for which developers have proposed fuzzing techniques in the literature. However, as MQTT is the most widely-used pub/sub protocol, it provides an opportunity to understand the requirements and strategies to effectively fuzz a pub/sub protocol. This PhD research was divided into three phases. In the first phase, it was analyzed whether a systematic testing approach such as grammar-based fuzzing can be applied to a pub/sub protocol such as MQTT, thereby understanding the challenges and necessary requirements. A grammar-based methodology and architecture was therefore conceived and proposed for a pub/sub protocol. The end result is a fuzzer called MQTTGRAM, which was then compared with two other fuzzing approaches and outperformed both of them, despite exchanging up to 9x fewer packets. In the second phase, a taxonomy was developed that classifies all existing fuzzing techniques for MQTT, six of which were evaluated under equivalent conditions in order to determine whether developers are considering the unique attributes of the pub/sub design pattern. Furthermore, the fuzzers were evaluated in terms of their resource usage or stress-testing capabilities. In the third phase, MQTTGRAM was improved by incorporating three essential elements for pub/sub protocol fuzzing, which are lacking across all fuzzing techniques proposed for MQTT: two-way communication; topic awareness; and version support. Overall, this PhD research provides three main contributions: (1) the development and refinement of a grammar-based fuzzing approach for a pub/sub protocol; (2) taxonomy and performance evaluation of MQTT fuzzers under equivalent conditions; and (3) identification of shortcomings for future work.
publishDate 2023
dc.date.none.fl_str_mv 2023-09-01
dc.type.status.fl_str_mv info:eu-repo/semantics/publishedVersion
dc.type.driver.fl_str_mv info:eu-repo/semantics/doctoralThesis
format doctoralThesis
status_str publishedVersion
dc.identifier.uri.fl_str_mv https://www.teses.usp.br/teses/disponiveis/45/45134/tde-20122023-223756/
url https://www.teses.usp.br/teses/disponiveis/45/45134/tde-20122023-223756/
dc.language.iso.fl_str_mv eng
language eng
dc.relation.none.fl_str_mv
dc.rights.driver.fl_str_mv Liberar o conteúdo para acesso público.
info:eu-repo/semantics/openAccess
rights_invalid_str_mv Liberar o conteúdo para acesso público.
eu_rights_str_mv openAccess
dc.format.none.fl_str_mv application/pdf
dc.coverage.none.fl_str_mv
dc.publisher.none.fl_str_mv Biblioteca Digitais de Teses e Dissertações da USP
publisher.none.fl_str_mv Biblioteca Digitais de Teses e Dissertações da USP
dc.source.none.fl_str_mv
reponame:Biblioteca Digital de Teses e Dissertações da USP
instname:Universidade de São Paulo (USP)
instacron:USP
instname_str Universidade de São Paulo (USP)
instacron_str USP
institution USP
reponame_str Biblioteca Digital de Teses e Dissertações da USP
collection Biblioteca Digital de Teses e Dissertações da USP
repository.name.fl_str_mv Biblioteca Digital de Teses e Dissertações da USP - Universidade de São Paulo (USP)
repository.mail.fl_str_mv virginia@if.usp.br|| atendimento@aguia.usp.br||virginia@if.usp.br
_version_ 1815257188736696320