Systematic Literature Review and Content Analysis in the Area of Communication and Information: the reliability problem and how to fix it

Detalhes bibliográficos
Autor(a) principal: Lycarião, Diógenes
Data de Publicação: 2023
Outros Autores: Roque, Robson, Costa, Débora
Tipo de documento: Artigo
Idioma: por
Título da fonte: Transinformação (Online)
Texto Completo: https://periodicos.puc-campinas.edu.br/transinfo/article/view/10064
Resumo: This work presents an analysis of the methodological quality of systematic reviews of Portuguese-speaking literature, in the area of Communication and Information by using principles and techniques of this methodology common to content analysis, notably those relating to replicability and reliability. For this, we used 4 indexing bases, namely Web of Science, Scopus, DOAJ and SciELO, under a specific temporal delimitation (2010-2021). By developing a codebook and carrying out reliability tests, we selected and analyzed 49 works regarding the following characteristics: (a) databases; (b) search strings; (c) temporal delimitation (number of years); (d) number of analyzed works; (e) inclusion and/or exclusion criteria; (f) replicability; (g) reliability. The main results indicate a reduced concern of the analyzed works in incorporating or reporting procedures responsive to reliability (n=3). On the other hand, most studies (n=40) proved to be, in general terms, replicable. The main innovation and contribution of this research consisted in proposing and testing a procedure to increase the reliability of systematic reviews by consulting the authors of the papers. We highlight, in addition to the innovative character of such a procedure, its epistemic improvements. Finally, we discuss directions for future research by indicating possible gains that the methodological strategies conceived and tested in this work offer to guide the methodological rigor of systematic literature reviews.
id PUC_CAMP-4_3b77712fcd884919086675a2540426e1
oai_identifier_str oai:ojs.periodicos.puc-campinas.edu.br:article/10064
network_acronym_str PUC_CAMP-4
network_name_str Transinformação (Online)
repository_id_str
spelling Systematic Literature Review and Content Analysis in the Area of Communication and Information: the reliability problem and how to fix itRevisão Sistemática de Literatura e Análise de Conteúdo na Área da Comunicação e Informação: o problema da confiabilidade e como resolvê-loSystematic Literature ReviewContent analysisReliabilityReplicabilityMethodologyRevisão Sistemática de LiteraturaAnálise de ConteúdoConfiabilidadeReplicabilidadeMetodologiaThis work presents an analysis of the methodological quality of systematic reviews of Portuguese-speaking literature, in the area of Communication and Information by using principles and techniques of this methodology common to content analysis, notably those relating to replicability and reliability. For this, we used 4 indexing bases, namely Web of Science, Scopus, DOAJ and SciELO, under a specific temporal delimitation (2010-2021). By developing a codebook and carrying out reliability tests, we selected and analyzed 49 works regarding the following characteristics: (a) databases; (b) search strings; (c) temporal delimitation (number of years); (d) number of analyzed works; (e) inclusion and/or exclusion criteria; (f) replicability; (g) reliability. The main results indicate a reduced concern of the analyzed works in incorporating or reporting procedures responsive to reliability (n=3). On the other hand, most studies (n=40) proved to be, in general terms, replicable. The main innovation and contribution of this research consisted in proposing and testing a procedure to increase the reliability of systematic reviews by consulting the authors of the papers. We highlight, in addition to the innovative character of such a procedure, its epistemic improvements. Finally, we discuss directions for future research by indicating possible gains that the methodological strategies conceived and tested in this work offer to guide the methodological rigor of systematic literature reviews.Este trabalho apresenta uma análise da qualidade metodológica das revisões sistemáticas de literatura lusófonas, na área da Comunicação e Informação, utilizando princípios e técnicas desta metodologia comuns à análise de conteúdo, notadamente as relativas à replicabilidade e à confiabilidade. Para isso, 4 bases de indexação foram utilizadas: Web of Science, Scopus, DOAJ e SciELO, sob delimitação temporal específica (2010-2021). Mediante o desenvolvimento de um livro de códigos e realização de testes de confiabilidade, 49 trabalhos foram selecionados e analisados em relação a informações sobre (a) bases utilizadas; (b) termos de busca; (c) delimitação temporal (quantidade de anos); (d) número de trabalhos analisados; (e) critérios de inclusão e/ou exclusão; (f) replicabilidade; (g) confiabilidade. Os principais resultados indicam uma reduzida preocupação dos trabalhos analisados em incorporar ou reportar procedimentos responsivos à confiabilidade (n=3). Por outro lado, a maior parte dos estudos (n=40) mostrou-se, em linhas gerais, replicável. A principal inovação e contribuição desta pesquisa consistiu em propor e testar um procedimento de incremento da confiabilidade das revisões sistemáticas mediante consulta aos autores dos trabalhos revisados. Destaca-se, para além do caráter inovador de tal procedimento, os ganhos epistêmicos por ele trazidos. Discute-se, ao final, as implicações dos resultados para pesquisas futuras, indicando os possíveis ganhos que as estratégias metodológicas concebidas e testadas neste trabalho oferecem para balizar o rigor metodológico das revisões sistemáticas de literatura. Núcleo de Editoração - PUC-Campinas2023-10-19info:eu-repo/semantics/articleinfo:eu-repo/semantics/publishedVersionPeer-reviewed ArticleArtículo revisado por paresAvaliado pelos Paresapplication/pdfhttps://periodicos.puc-campinas.edu.br/transinfo/article/view/10064Transinformação; Vol. 35 (2023)Transinformação; Vol. 35 (2023)Transinformação; v. 35 (2023)2318-08890103-3786reponame:Transinformação (Online)instname:Pontifícia Universidade Católica de Campinas (PUC-CAMPINAS)instacron:PUC_CAMPporhttps://periodicos.puc-campinas.edu.br/transinfo/article/view/10064/7423Copyright (c) 2023 Transinformaçãohttps://creativecommons.org/licenses/by/4.0info:eu-repo/semantics/openAccessLycarião, DiógenesRoque, Robson Costa, Débora2024-04-02T12:31:59Zoai:ojs.periodicos.puc-campinas.edu.br:article/10064Revistahttp://periodicos.puc-campinas.edu.br/seer/index.php/transinfo/indexPRIhttps://old.scielo.br/oai/scielo-oai.phpsbi.nucleodeeditoracao@puc-campinas.edu.br2318-08890103-3786opendoar:2024-04-02T12:31:59Transinformação (Online) - Pontifícia Universidade Católica de Campinas (PUC-CAMPINAS)false
dc.title.none.fl_str_mv Systematic Literature Review and Content Analysis in the Area of Communication and Information: the reliability problem and how to fix it
Revisão Sistemática de Literatura e Análise de Conteúdo na Área da Comunicação e Informação: o problema da confiabilidade e como resolvê-lo
title Systematic Literature Review and Content Analysis in the Area of Communication and Information: the reliability problem and how to fix it
spellingShingle Systematic Literature Review and Content Analysis in the Area of Communication and Information: the reliability problem and how to fix it
Lycarião, Diógenes
Systematic Literature Review
Content analysis
Reliability
Replicability
Methodology
Revisão Sistemática de Literatura
Análise de Conteúdo
Confiabilidade
Replicabilidade
Metodologia
title_short Systematic Literature Review and Content Analysis in the Area of Communication and Information: the reliability problem and how to fix it
title_full Systematic Literature Review and Content Analysis in the Area of Communication and Information: the reliability problem and how to fix it
title_fullStr Systematic Literature Review and Content Analysis in the Area of Communication and Information: the reliability problem and how to fix it
title_full_unstemmed Systematic Literature Review and Content Analysis in the Area of Communication and Information: the reliability problem and how to fix it
title_sort Systematic Literature Review and Content Analysis in the Area of Communication and Information: the reliability problem and how to fix it
author Lycarião, Diógenes
author_facet Lycarião, Diógenes
Roque, Robson
Costa, Débora
author_role author
author2 Roque, Robson
Costa, Débora
author2_role author
author
dc.contributor.author.fl_str_mv Lycarião, Diógenes
Roque, Robson
Costa, Débora
dc.subject.por.fl_str_mv Systematic Literature Review
Content analysis
Reliability
Replicability
Methodology
Revisão Sistemática de Literatura
Análise de Conteúdo
Confiabilidade
Replicabilidade
Metodologia
topic Systematic Literature Review
Content analysis
Reliability
Replicability
Methodology
Revisão Sistemática de Literatura
Análise de Conteúdo
Confiabilidade
Replicabilidade
Metodologia
description This work presents an analysis of the methodological quality of systematic reviews of Portuguese-speaking literature, in the area of Communication and Information by using principles and techniques of this methodology common to content analysis, notably those relating to replicability and reliability. For this, we used 4 indexing bases, namely Web of Science, Scopus, DOAJ and SciELO, under a specific temporal delimitation (2010-2021). By developing a codebook and carrying out reliability tests, we selected and analyzed 49 works regarding the following characteristics: (a) databases; (b) search strings; (c) temporal delimitation (number of years); (d) number of analyzed works; (e) inclusion and/or exclusion criteria; (f) replicability; (g) reliability. The main results indicate a reduced concern of the analyzed works in incorporating or reporting procedures responsive to reliability (n=3). On the other hand, most studies (n=40) proved to be, in general terms, replicable. The main innovation and contribution of this research consisted in proposing and testing a procedure to increase the reliability of systematic reviews by consulting the authors of the papers. We highlight, in addition to the innovative character of such a procedure, its epistemic improvements. Finally, we discuss directions for future research by indicating possible gains that the methodological strategies conceived and tested in this work offer to guide the methodological rigor of systematic literature reviews.
publishDate 2023
dc.date.none.fl_str_mv 2023-10-19
dc.type.driver.fl_str_mv info:eu-repo/semantics/article
info:eu-repo/semantics/publishedVersion
Peer-reviewed Article
Artículo revisado por pares
Avaliado pelos Pares
format article
status_str publishedVersion
dc.identifier.uri.fl_str_mv https://periodicos.puc-campinas.edu.br/transinfo/article/view/10064
url https://periodicos.puc-campinas.edu.br/transinfo/article/view/10064
dc.language.iso.fl_str_mv por
language por
dc.relation.none.fl_str_mv https://periodicos.puc-campinas.edu.br/transinfo/article/view/10064/7423
dc.rights.driver.fl_str_mv Copyright (c) 2023 Transinformação
https://creativecommons.org/licenses/by/4.0
info:eu-repo/semantics/openAccess
rights_invalid_str_mv Copyright (c) 2023 Transinformação
https://creativecommons.org/licenses/by/4.0
eu_rights_str_mv openAccess
dc.format.none.fl_str_mv application/pdf
dc.publisher.none.fl_str_mv Núcleo de Editoração - PUC-Campinas
publisher.none.fl_str_mv Núcleo de Editoração - PUC-Campinas
dc.source.none.fl_str_mv Transinformação; Vol. 35 (2023)
Transinformação; Vol. 35 (2023)
Transinformação; v. 35 (2023)
2318-0889
0103-3786
reponame:Transinformação (Online)
instname:Pontifícia Universidade Católica de Campinas (PUC-CAMPINAS)
instacron:PUC_CAMP
instname_str Pontifícia Universidade Católica de Campinas (PUC-CAMPINAS)
instacron_str PUC_CAMP
institution PUC_CAMP
reponame_str Transinformação (Online)
collection Transinformação (Online)
repository.name.fl_str_mv Transinformação (Online) - Pontifícia Universidade Católica de Campinas (PUC-CAMPINAS)
repository.mail.fl_str_mv sbi.nucleodeeditoracao@puc-campinas.edu.br
_version_ 1799125987222355968

Registros relacionados