Fault-Tolerant, Scalable and Interoperable IoT Platform

Detalhes bibliográficos
Autor(a) principal: Soares, José Marques Barga
Data de Publicação: 2020
Tipo de documento: Dissertação
Idioma: eng
Título da fonte: Repositório Científico de Acesso Aberto de Portugal (Repositórios Cientìficos)
Texto Completo: http://hdl.handle.net/10451/48348
Resumo: Tese de mestrado, Engenharia Informática (Engenharia de Software) Universidade de Lisboa, Faculdade de Ciências, 2020
id RCAP_7d17f42ec1e718bbbc5e39d08ae998d1
oai_identifier_str oai:repositorio.ul.pt:10451/48348
network_acronym_str RCAP
network_name_str Repositório Científico de Acesso Aberto de Portugal (Repositórios Cientìficos)
repository_id_str 7160
spelling Fault-Tolerant, Scalable and Interoperable IoT PlatformIoTLoRaSigfoxNB-IoTBluetoothZigBeeWi-FiLTEGSMMQTTCoAPLoRaWANAMQPRESTOPC-UAKubernetesKafkaComputação em NuvemEscalabilidadeInteroperabilidadeTolerância a faltasResiliênciaDisponibilidadefacilidade de gestãoPublish-SubscribeRequest-ResponseMicro-serviçosContentores DockerTeses de mestrado - 2020Departamento de InformáticaTese de mestrado, Engenharia Informática (Engenharia de Software) Universidade de Lisboa, Faculdade de Ciências, 2020Nowadays the growth of Internet usage is quite visible. Everyday the number of devices connected to the Internet increases, everything may be a smart device capable of interacting with the Internet, from smartphones, smartwatches, refrigerators and much more. All of these devices are called things in the Internet of Things. Many of them are usually constrained devices due to it’s size, usually very small with low capacities such as memory and/or processing power. These kind of devices need to be very efficient in all of their actives. For example, the battery lifetime should be maximized as possible so that the necessity to change each device’s battery could be minimized. There are many technologies that allow communication between devices. Besides the technologies, protocols may be involved in the communication between each device in an IoT system. Communication Protocols define the behaviour that is followed by things when communicating with each other. For example, in some protocols acknowledgments must be used to ensure data arrival, while in others this feature is not enforced. There are many communication Protocols available in the literature. The use of communication protocols and communication models bring many benefits to IoT systems, but they may also benefit from using the cloud. One of the biggest struggles in IoT is the fact that things are very constrained devices in terms of resources (CPU and RAM). With the cloud this would no longer be an issue. Plus, the cloud is able of providing device management, scalability, storage and real time transmission. The characteristics of the communication protocols were studied and an innovative system architecture based on micro-services, Kubernetes and Kafka is proposed in this thesis. This proposal tries to address issues such as scalability, interoperability, fault tolerance, resiliency, availability and simple management of large IoT systems. Supported by Kubernetes, which is an open-source technology that allows micro-services to be extensible, configurable and automatically managed with fault tolerance and Kafka, which is a distributed event log that uses the publish-subscribe pattern, the proposed architecture is able to deal with high number of devices producing and consuming data at the same time. The proposed Fault-Tolerant and Interoperable IoT Architecture is a cluster composed of many components (micro-services) that were implemented using docker containers. The current implementation of the system supports the MQTT, CoAP and REST protocols for data incoming and the same plus websockets for data output. Since the system is based on micro-services, more protocols may be added in a simple way (just a new micro-service must be added). The system is able to convert any protocol into another protocol, e.g., if a message arrives at the system through MQTT protocol, it can be consumed using the CoAP or REST protocol. When messages are sent to the system the payload is stored in Kafka independently of the protocol, and when clients request it, it is consumed from Kafka and encapsulated by the client protocol to be sent to the client. In order to evaluate and demonstrate the capabilities of our proposal a set of experiments were made, which allows to collect information about the performance of the Communication Protocols, the system as a whole, Kubernetes and Kafka. From the experiments we were able to conclude that the message size is not so much important, since the system is able to deal with messages from 39 bytes to 2000 bytes. Since we are designing the system for IoT applications, we considered that messages with 2000 Bytes are big messages. Also, it was recognized that the system is able to recover from crashed nodes and to respond well in terms of average delay and packet loss when low and high throughput are compared. In this situation, there is a significant impact of the RAM usage, but the system still works without problems. In terms of scalability, the evaluation of the system through its cluster under-layer platform (Kubernetes) allowed us to understand that there is no direct relation between the time spent toconstant. However, the same conclusion is not true for the number of instances that are needed at high layer (application layer). Here, time spent to increase the number of instances of a specific application is directly proportional to the number of instances that are already running. In respect to data redundancy and persistence, the experiments showed that the average delay and packet loss of a message sent from a Producer to a Receiver is approximately the same regardless of the number of Kafka instances being used. Additionally, using a high number of partitions has a negative impact on the system’s behaviour.Cecílio, José Manuel da SilvaRepositório da Universidade de LisboaSoares, José Marques Barga2021-06-05T10:39:53Z202020202020-01-01T00:00:00Zinfo:eu-repo/semantics/publishedVersioninfo:eu-repo/semantics/masterThesisapplication/pdfhttp://hdl.handle.net/10451/48348TID:202604802enginfo:eu-repo/semantics/openAccessreponame:Repositório Científico de Acesso Aberto de Portugal (Repositórios Cientìficos)instname:Agência para a Sociedade do Conhecimento (UMIC) - FCT - Sociedade da Informaçãoinstacron:RCAAP2023-11-08T16:51:42Zoai:repositorio.ul.pt:10451/48348Portal AgregadorONGhttps://www.rcaap.pt/oai/openaireopendoar:71602024-03-19T22:00:15.296275Repositório Científico de Acesso Aberto de Portugal (Repositórios Cientìficos) - Agência para a Sociedade do Conhecimento (UMIC) - FCT - Sociedade da Informaçãofalse
dc.title.none.fl_str_mv Fault-Tolerant, Scalable and Interoperable IoT Platform
title Fault-Tolerant, Scalable and Interoperable IoT Platform
spellingShingle Fault-Tolerant, Scalable and Interoperable IoT Platform
Soares, José Marques Barga
IoT
LoRa
Sigfox
NB-IoT
Bluetooth
ZigBee
Wi-Fi
LTE
GSM
MQTT
CoAP
LoRaWAN
AMQP
REST
OPC-UA
Kubernetes
Kafka
Computação em Nuvem
Escalabilidade
Interoperabilidade
Tolerância a faltas
Resiliência
Disponibilidade
facilidade de gestão
Publish-Subscribe
Request-Response
Micro-serviços
Contentores Docker
Teses de mestrado - 2020
Departamento de Informática
title_short Fault-Tolerant, Scalable and Interoperable IoT Platform
title_full Fault-Tolerant, Scalable and Interoperable IoT Platform
title_fullStr Fault-Tolerant, Scalable and Interoperable IoT Platform
title_full_unstemmed Fault-Tolerant, Scalable and Interoperable IoT Platform
title_sort Fault-Tolerant, Scalable and Interoperable IoT Platform
author Soares, José Marques Barga
author_facet Soares, José Marques Barga
author_role author
dc.contributor.none.fl_str_mv Cecílio, José Manuel da Silva
Repositório da Universidade de Lisboa
dc.contributor.author.fl_str_mv Soares, José Marques Barga
dc.subject.por.fl_str_mv IoT
LoRa
Sigfox
NB-IoT
Bluetooth
ZigBee
Wi-Fi
LTE
GSM
MQTT
CoAP
LoRaWAN
AMQP
REST
OPC-UA
Kubernetes
Kafka
Computação em Nuvem
Escalabilidade
Interoperabilidade
Tolerância a faltas
Resiliência
Disponibilidade
facilidade de gestão
Publish-Subscribe
Request-Response
Micro-serviços
Contentores Docker
Teses de mestrado - 2020
Departamento de Informática
topic IoT
LoRa
Sigfox
NB-IoT
Bluetooth
ZigBee
Wi-Fi
LTE
GSM
MQTT
CoAP
LoRaWAN
AMQP
REST
OPC-UA
Kubernetes
Kafka
Computação em Nuvem
Escalabilidade
Interoperabilidade
Tolerância a faltas
Resiliência
Disponibilidade
facilidade de gestão
Publish-Subscribe
Request-Response
Micro-serviços
Contentores Docker
Teses de mestrado - 2020
Departamento de Informática
description Tese de mestrado, Engenharia Informática (Engenharia de Software) Universidade de Lisboa, Faculdade de Ciências, 2020
publishDate 2020
dc.date.none.fl_str_mv 2020
2020
2020-01-01T00:00:00Z
2021-06-05T10:39:53Z
dc.type.status.fl_str_mv info:eu-repo/semantics/publishedVersion
dc.type.driver.fl_str_mv info:eu-repo/semantics/masterThesis
format masterThesis
status_str publishedVersion
dc.identifier.uri.fl_str_mv http://hdl.handle.net/10451/48348
TID:202604802
url http://hdl.handle.net/10451/48348
identifier_str_mv TID:202604802
dc.language.iso.fl_str_mv eng
language eng
dc.rights.driver.fl_str_mv info:eu-repo/semantics/openAccess
eu_rights_str_mv openAccess
dc.format.none.fl_str_mv application/pdf
dc.source.none.fl_str_mv reponame:Repositório Científico de Acesso Aberto de Portugal (Repositórios Cientìficos)
instname:Agência para a Sociedade do Conhecimento (UMIC) - FCT - Sociedade da Informação
instacron:RCAAP
instname_str Agência para a Sociedade do Conhecimento (UMIC) - FCT - Sociedade da Informação
instacron_str RCAAP
institution RCAAP
reponame_str Repositório Científico de Acesso Aberto de Portugal (Repositórios Cientìficos)
collection Repositório Científico de Acesso Aberto de Portugal (Repositórios Cientìficos)
repository.name.fl_str_mv Repositório Científico de Acesso Aberto de Portugal (Repositórios Cientìficos) - Agência para a Sociedade do Conhecimento (UMIC) - FCT - Sociedade da Informação
repository.mail.fl_str_mv
_version_ 1799134549924380672

Registros relacionados