Do item-writing flaws reduce examinations psychometric quality?
Autor(a) principal: | |
---|---|
Data de Publicação: | 2016 |
Outros Autores: | , , , , , , , |
Tipo de documento: | Artigo |
Idioma: | eng |
Título da fonte: | Repositório Científico de Acesso Aberto de Portugal (Repositórios Cientìficos) |
Texto Completo: | http://hdl.handle.net/10400.16/2091 |
Resumo: | Background The psychometric characteristics of multiple-choice questions (MCQ) changed when taking into account their anatomical sites and the presence of item-writing flaws (IWF). The aim is to understand the impact of the anatomical sites and the presence of IWF in the psychometric qualities of the MCQ. Results 800 Clinical Anatomy MCQ from eight examinations were classified as standard or flawed items and according to one of the eight anatomical sites. An item was classified as flawed if it violated at least one of the principles of item writing. The difficulty and discrimination indices of each item were obtained. 55.8 % of the MCQ were flawed items. The anatomical site of the items explained 6.2 and 3.2 % of the difficulty and discrimination parameters and the IWF explained 2.8 and 0.8 %, respectively. Conclusions The impact of the IWF was heterogeneous, the Writing the Stem and Writing the Choices categories had a negative impact (higher difficulty and lower discrimination) while the other categories did not have any impact. The anatomical site effect was higher than IWF effect in the psychometric characteristics of the examination. When constructing MCQ, the focus should be in the topic/area of the items and only after in the presence of IWF. |
id |
RCAP_f12324e926b0ae3aa1de0b9dd5686eb2 |
---|---|
oai_identifier_str |
oai:repositorio.chporto.pt:10400.16/2091 |
network_acronym_str |
RCAP |
network_name_str |
Repositório Científico de Acesso Aberto de Portugal (Repositórios Cientìficos) |
repository_id_str |
7160 |
spelling |
Do item-writing flaws reduce examinations psychometric quality?AssessmentExaminationItem-writing flawsMultiple-choice questionsClinical anatomyPsychometric characteristicsBackground The psychometric characteristics of multiple-choice questions (MCQ) changed when taking into account their anatomical sites and the presence of item-writing flaws (IWF). The aim is to understand the impact of the anatomical sites and the presence of IWF in the psychometric qualities of the MCQ. Results 800 Clinical Anatomy MCQ from eight examinations were classified as standard or flawed items and according to one of the eight anatomical sites. An item was classified as flawed if it violated at least one of the principles of item writing. The difficulty and discrimination indices of each item were obtained. 55.8 % of the MCQ were flawed items. The anatomical site of the items explained 6.2 and 3.2 % of the difficulty and discrimination parameters and the IWF explained 2.8 and 0.8 %, respectively. Conclusions The impact of the IWF was heterogeneous, the Writing the Stem and Writing the Choices categories had a negative impact (higher difficulty and lower discrimination) while the other categories did not have any impact. The anatomical site effect was higher than IWF effect in the psychometric characteristics of the examination. When constructing MCQ, the focus should be in the topic/area of the items and only after in the presence of IWF.This work was supported by the IJUP Project under Grant Number PP_IJUP2011 67BioMed CentralRepositório Científico do Centro Hospitalar Universitário de Santo AntónioPais, J.Silva, A.Guimarães, B.Povo, A.Coelho, E.Silva-Pereira, F.Lourinho, I.Ferreira, M.Severo, M.2017-05-22T09:04:52Z2016-08-112016-08-11T00:00:00Zinfo:eu-repo/semantics/publishedVersioninfo:eu-repo/semantics/articleapplication/pdfhttp://hdl.handle.net/10400.16/2091engBMC Res Notes. 2016 Aug 11;9(1):399.1756-050010.1186/s13104-016-2202-4info:eu-repo/semantics/openAccessreponame:Repositório Científico de Acesso Aberto de Portugal (Repositórios Cientìficos)instname:Agência para a Sociedade do Conhecimento (UMIC) - FCT - Sociedade da Informaçãoinstacron:RCAAP2023-10-20T10:59:03Zoai:repositorio.chporto.pt:10400.16/2091Portal AgregadorONGhttps://www.rcaap.pt/oai/openaireopendoar:71602024-03-19T20:38:21.807208Repositório Científico de Acesso Aberto de Portugal (Repositórios Cientìficos) - Agência para a Sociedade do Conhecimento (UMIC) - FCT - Sociedade da Informaçãofalse |
dc.title.none.fl_str_mv |
Do item-writing flaws reduce examinations psychometric quality? |
title |
Do item-writing flaws reduce examinations psychometric quality? |
spellingShingle |
Do item-writing flaws reduce examinations psychometric quality? Pais, J. Assessment Examination Item-writing flaws Multiple-choice questions Clinical anatomy Psychometric characteristics |
title_short |
Do item-writing flaws reduce examinations psychometric quality? |
title_full |
Do item-writing flaws reduce examinations psychometric quality? |
title_fullStr |
Do item-writing flaws reduce examinations psychometric quality? |
title_full_unstemmed |
Do item-writing flaws reduce examinations psychometric quality? |
title_sort |
Do item-writing flaws reduce examinations psychometric quality? |
author |
Pais, J. |
author_facet |
Pais, J. Silva, A. Guimarães, B. Povo, A. Coelho, E. Silva-Pereira, F. Lourinho, I. Ferreira, M. Severo, M. |
author_role |
author |
author2 |
Silva, A. Guimarães, B. Povo, A. Coelho, E. Silva-Pereira, F. Lourinho, I. Ferreira, M. Severo, M. |
author2_role |
author author author author author author author author |
dc.contributor.none.fl_str_mv |
Repositório Científico do Centro Hospitalar Universitário de Santo António |
dc.contributor.author.fl_str_mv |
Pais, J. Silva, A. Guimarães, B. Povo, A. Coelho, E. Silva-Pereira, F. Lourinho, I. Ferreira, M. Severo, M. |
dc.subject.por.fl_str_mv |
Assessment Examination Item-writing flaws Multiple-choice questions Clinical anatomy Psychometric characteristics |
topic |
Assessment Examination Item-writing flaws Multiple-choice questions Clinical anatomy Psychometric characteristics |
description |
Background The psychometric characteristics of multiple-choice questions (MCQ) changed when taking into account their anatomical sites and the presence of item-writing flaws (IWF). The aim is to understand the impact of the anatomical sites and the presence of IWF in the psychometric qualities of the MCQ. Results 800 Clinical Anatomy MCQ from eight examinations were classified as standard or flawed items and according to one of the eight anatomical sites. An item was classified as flawed if it violated at least one of the principles of item writing. The difficulty and discrimination indices of each item were obtained. 55.8 % of the MCQ were flawed items. The anatomical site of the items explained 6.2 and 3.2 % of the difficulty and discrimination parameters and the IWF explained 2.8 and 0.8 %, respectively. Conclusions The impact of the IWF was heterogeneous, the Writing the Stem and Writing the Choices categories had a negative impact (higher difficulty and lower discrimination) while the other categories did not have any impact. The anatomical site effect was higher than IWF effect in the psychometric characteristics of the examination. When constructing MCQ, the focus should be in the topic/area of the items and only after in the presence of IWF. |
publishDate |
2016 |
dc.date.none.fl_str_mv |
2016-08-11 2016-08-11T00:00:00Z 2017-05-22T09:04:52Z |
dc.type.status.fl_str_mv |
info:eu-repo/semantics/publishedVersion |
dc.type.driver.fl_str_mv |
info:eu-repo/semantics/article |
format |
article |
status_str |
publishedVersion |
dc.identifier.uri.fl_str_mv |
http://hdl.handle.net/10400.16/2091 |
url |
http://hdl.handle.net/10400.16/2091 |
dc.language.iso.fl_str_mv |
eng |
language |
eng |
dc.relation.none.fl_str_mv |
BMC Res Notes. 2016 Aug 11;9(1):399. 1756-0500 10.1186/s13104-016-2202-4 |
dc.rights.driver.fl_str_mv |
info:eu-repo/semantics/openAccess |
eu_rights_str_mv |
openAccess |
dc.format.none.fl_str_mv |
application/pdf |
dc.publisher.none.fl_str_mv |
BioMed Central |
publisher.none.fl_str_mv |
BioMed Central |
dc.source.none.fl_str_mv |
reponame:Repositório Científico de Acesso Aberto de Portugal (Repositórios Cientìficos) instname:Agência para a Sociedade do Conhecimento (UMIC) - FCT - Sociedade da Informação instacron:RCAAP |
instname_str |
Agência para a Sociedade do Conhecimento (UMIC) - FCT - Sociedade da Informação |
instacron_str |
RCAAP |
institution |
RCAAP |
reponame_str |
Repositório Científico de Acesso Aberto de Portugal (Repositórios Cientìficos) |
collection |
Repositório Científico de Acesso Aberto de Portugal (Repositórios Cientìficos) |
repository.name.fl_str_mv |
Repositório Científico de Acesso Aberto de Portugal (Repositórios Cientìficos) - Agência para a Sociedade do Conhecimento (UMIC) - FCT - Sociedade da Informação |
repository.mail.fl_str_mv |
|
_version_ |
1799133644964495360 |