Assessing the coding reliability of work accidents statistical data: how coders make a difference
Autor(a) principal: | |
---|---|
Data de Publicação: | 2016 |
Outros Autores: | , , |
Tipo de documento: | Artigo |
Idioma: | eng |
Título da fonte: | Repositório Científico de Acesso Aberto de Portugal (Repositórios Cientìficos) |
Texto Completo: | http://hdl.handle.net/10071/12839 |
Resumo: | Introduction: This study assesses the reliability of the coding procedure for a set of variables belonging to the European Statistics of Accidents at Work (ESAW). The work focused on the Portuguese data and experience with the system. In Portugal, this task has been systematically carried out by GEP (the governmental Cabinet for Strategy and Planning), here defined as the “reference group” or “expert group”. However, it is anticipated that this coding task will be performed by non-expert people, since paper-forms will be replaced by e-forms, similarly to what happened in a few EU countries. Objective: This study aims to: 1) assess the current situation, i.e., to quantify reliability of data coded by GEP (reference group), and 2) assess the impact on the reliability level when the coding is carried out by non-experts (two different groups of coders). Methods: The study comprises the estimation of both intercoder and intracoder reliability for a set of 8 nominal variables. The assessment applies 3 reliability coefficients calculated by 3 software packages. Results: The results reveal that the expert group (GEP) holds good to excellent reliability (inter- and intracoder agreements), between 68-98%, while there is a considerable “loss of reliability” (-5% to -39%) when the coding process is transferred to other people, without special training or knowledge in this task. Conclusions: this work gives quantified evidence that reliability of coding accident data is substantially affected by the coders’ profile. Moreover, certain variables, regardless of the coder, systematically hold a higher level of coding reliability than others, suggesting that certain codes may need improvement. Future studies should assess coding quality across the EU countries using the ESAW protocol. Practical Applications: directions for improving the quality of accident data and related statistics; data that is used by researchers and governmental decision-makers to derive prevention strategies. |
id |
RCAP_e0da690e28ee8d1c7048155fcfcd8857 |
---|---|
oai_identifier_str |
oai:repositorio.iscte-iul.pt:10071/12839 |
network_acronym_str |
RCAP |
network_name_str |
Repositório Científico de Acesso Aberto de Portugal (Repositórios Cientìficos) |
repository_id_str |
7160 |
spelling |
Assessing the coding reliability of work accidents statistical data: how coders make a differenceAccident dataContent analysisReliability coefficientsIntercoder reliabilityESAW variablesIntroduction: This study assesses the reliability of the coding procedure for a set of variables belonging to the European Statistics of Accidents at Work (ESAW). The work focused on the Portuguese data and experience with the system. In Portugal, this task has been systematically carried out by GEP (the governmental Cabinet for Strategy and Planning), here defined as the “reference group” or “expert group”. However, it is anticipated that this coding task will be performed by non-expert people, since paper-forms will be replaced by e-forms, similarly to what happened in a few EU countries. Objective: This study aims to: 1) assess the current situation, i.e., to quantify reliability of data coded by GEP (reference group), and 2) assess the impact on the reliability level when the coding is carried out by non-experts (two different groups of coders). Methods: The study comprises the estimation of both intercoder and intracoder reliability for a set of 8 nominal variables. The assessment applies 3 reliability coefficients calculated by 3 software packages. Results: The results reveal that the expert group (GEP) holds good to excellent reliability (inter- and intracoder agreements), between 68-98%, while there is a considerable “loss of reliability” (-5% to -39%) when the coding process is transferred to other people, without special training or knowledge in this task. Conclusions: this work gives quantified evidence that reliability of coding accident data is substantially affected by the coders’ profile. Moreover, certain variables, regardless of the coder, systematically hold a higher level of coding reliability than others, suggesting that certain codes may need improvement. Future studies should assess coding quality across the EU countries using the ESAW protocol. Practical Applications: directions for improving the quality of accident data and related statistics; data that is used by researchers and governmental decision-makers to derive prevention strategies.Pergamon/Elsevier2017-04-07T13:52:10Z2016-01-01T00:00:00Z20162019-04-12T15:31:24Zinfo:eu-repo/semantics/publishedVersioninfo:eu-repo/semantics/articleapplication/pdfhttp://hdl.handle.net/10071/12839eng0022-437510.1016/j.jsr.2016.09.005Jacinto, C.Santos, F. P.Soares, C. G.Silva, S. A.info:eu-repo/semantics/embargoedAccessreponame:Repositório Científico de Acesso Aberto de Portugal (Repositórios Cientìficos)instname:Agência para a Sociedade do Conhecimento (UMIC) - FCT - Sociedade da Informaçãoinstacron:RCAAP2023-11-09T17:54:13Zoai:repositorio.iscte-iul.pt:10071/12839Portal AgregadorONGhttps://www.rcaap.pt/oai/openaireopendoar:71602024-03-19T22:27:16.830236Repositório Científico de Acesso Aberto de Portugal (Repositórios Cientìficos) - Agência para a Sociedade do Conhecimento (UMIC) - FCT - Sociedade da Informaçãofalse |
dc.title.none.fl_str_mv |
Assessing the coding reliability of work accidents statistical data: how coders make a difference |
title |
Assessing the coding reliability of work accidents statistical data: how coders make a difference |
spellingShingle |
Assessing the coding reliability of work accidents statistical data: how coders make a difference Jacinto, C. Accident data Content analysis Reliability coefficients Intercoder reliability ESAW variables |
title_short |
Assessing the coding reliability of work accidents statistical data: how coders make a difference |
title_full |
Assessing the coding reliability of work accidents statistical data: how coders make a difference |
title_fullStr |
Assessing the coding reliability of work accidents statistical data: how coders make a difference |
title_full_unstemmed |
Assessing the coding reliability of work accidents statistical data: how coders make a difference |
title_sort |
Assessing the coding reliability of work accidents statistical data: how coders make a difference |
author |
Jacinto, C. |
author_facet |
Jacinto, C. Santos, F. P. Soares, C. G. Silva, S. A. |
author_role |
author |
author2 |
Santos, F. P. Soares, C. G. Silva, S. A. |
author2_role |
author author author |
dc.contributor.author.fl_str_mv |
Jacinto, C. Santos, F. P. Soares, C. G. Silva, S. A. |
dc.subject.por.fl_str_mv |
Accident data Content analysis Reliability coefficients Intercoder reliability ESAW variables |
topic |
Accident data Content analysis Reliability coefficients Intercoder reliability ESAW variables |
description |
Introduction: This study assesses the reliability of the coding procedure for a set of variables belonging to the European Statistics of Accidents at Work (ESAW). The work focused on the Portuguese data and experience with the system. In Portugal, this task has been systematically carried out by GEP (the governmental Cabinet for Strategy and Planning), here defined as the “reference group” or “expert group”. However, it is anticipated that this coding task will be performed by non-expert people, since paper-forms will be replaced by e-forms, similarly to what happened in a few EU countries. Objective: This study aims to: 1) assess the current situation, i.e., to quantify reliability of data coded by GEP (reference group), and 2) assess the impact on the reliability level when the coding is carried out by non-experts (two different groups of coders). Methods: The study comprises the estimation of both intercoder and intracoder reliability for a set of 8 nominal variables. The assessment applies 3 reliability coefficients calculated by 3 software packages. Results: The results reveal that the expert group (GEP) holds good to excellent reliability (inter- and intracoder agreements), between 68-98%, while there is a considerable “loss of reliability” (-5% to -39%) when the coding process is transferred to other people, without special training or knowledge in this task. Conclusions: this work gives quantified evidence that reliability of coding accident data is substantially affected by the coders’ profile. Moreover, certain variables, regardless of the coder, systematically hold a higher level of coding reliability than others, suggesting that certain codes may need improvement. Future studies should assess coding quality across the EU countries using the ESAW protocol. Practical Applications: directions for improving the quality of accident data and related statistics; data that is used by researchers and governmental decision-makers to derive prevention strategies. |
publishDate |
2016 |
dc.date.none.fl_str_mv |
2016-01-01T00:00:00Z 2016 2017-04-07T13:52:10Z 2019-04-12T15:31:24Z |
dc.type.status.fl_str_mv |
info:eu-repo/semantics/publishedVersion |
dc.type.driver.fl_str_mv |
info:eu-repo/semantics/article |
format |
article |
status_str |
publishedVersion |
dc.identifier.uri.fl_str_mv |
http://hdl.handle.net/10071/12839 |
url |
http://hdl.handle.net/10071/12839 |
dc.language.iso.fl_str_mv |
eng |
language |
eng |
dc.relation.none.fl_str_mv |
0022-4375 10.1016/j.jsr.2016.09.005 |
dc.rights.driver.fl_str_mv |
info:eu-repo/semantics/embargoedAccess |
eu_rights_str_mv |
embargoedAccess |
dc.format.none.fl_str_mv |
application/pdf |
dc.publisher.none.fl_str_mv |
Pergamon/Elsevier |
publisher.none.fl_str_mv |
Pergamon/Elsevier |
dc.source.none.fl_str_mv |
reponame:Repositório Científico de Acesso Aberto de Portugal (Repositórios Cientìficos) instname:Agência para a Sociedade do Conhecimento (UMIC) - FCT - Sociedade da Informação instacron:RCAAP |
instname_str |
Agência para a Sociedade do Conhecimento (UMIC) - FCT - Sociedade da Informação |
instacron_str |
RCAAP |
institution |
RCAAP |
reponame_str |
Repositório Científico de Acesso Aberto de Portugal (Repositórios Cientìficos) |
collection |
Repositório Científico de Acesso Aberto de Portugal (Repositórios Cientìficos) |
repository.name.fl_str_mv |
Repositório Científico de Acesso Aberto de Portugal (Repositórios Cientìficos) - Agência para a Sociedade do Conhecimento (UMIC) - FCT - Sociedade da Informação |
repository.mail.fl_str_mv |
|
_version_ |
1799134836044070912 |