Item pre-knowledge true prevalence in clinical anatomy: application of gated item response theory model
Autor(a) principal: | |
---|---|
Data de Publicação: | 2019 |
Outros Autores: | , , , |
Tipo de documento: | Artigo |
Idioma: | eng |
Título da fonte: | Repositório Científico de Acesso Aberto de Portugal (Repositórios Cientìficos) |
Texto Completo: | http://hdl.handle.net/10773/26317 |
Resumo: | Background: Computer and paper examinations in our days are constructed from an item pool which is regularly updated. Given the way that exams are created, one of the major concerns is the security of the items that are being used in order to ensure a good estimation of abilities. The aim of this study is to measure the prevalence of item pre-knowledge in our medical school. Methods: The Deterministic, Gated Item Response Theory Model (DGM) was applied to estimate the prevalence of students who have had item pre-knowledge from six multiple choice examinations of the Clinical Anatomy course at the Faculty of Medicine of University of Porto. Each examination consisted of 100 items with an average of 200 students and 20% repeated items per examination. The estimation of the sensitivity and specificity was based on a simulation study. The sensitivity and specificity estimates, and apparent prevalence were used to estimate true prevalence of cheating students in the examinations under study. Results: The specificity in the DGM for different simulation scenarios was between 68 and 98%, while the sensitivity ranged from 60 to 91%. The apparent prevalence was between 0.0 and 3.4%, while the true prevalence ranged from 1.2 to 3.7%. Conclusions: The true prevalence was much lower compared to the students self-reported copying of responses from other students; however, it is important to keep monitoring the pre-knowledge prevalence in order to enforce measures in case an increase occurs. |
id |
RCAP_4cb2e9460c6002238e3e4f7e8c42505e |
---|---|
oai_identifier_str |
oai:ria.ua.pt:10773/26317 |
network_acronym_str |
RCAP |
network_name_str |
Repositório Científico de Acesso Aberto de Portugal (Repositórios Cientìficos) |
repository_id_str |
7160 |
spelling |
Item pre-knowledge true prevalence in clinical anatomy: application of gated item response theory modelDeterministic Gated item response theory modelItem pre-knowledgeItem sharingItem exposureRepeat itemsBackground: Computer and paper examinations in our days are constructed from an item pool which is regularly updated. Given the way that exams are created, one of the major concerns is the security of the items that are being used in order to ensure a good estimation of abilities. The aim of this study is to measure the prevalence of item pre-knowledge in our medical school. Methods: The Deterministic, Gated Item Response Theory Model (DGM) was applied to estimate the prevalence of students who have had item pre-knowledge from six multiple choice examinations of the Clinical Anatomy course at the Faculty of Medicine of University of Porto. Each examination consisted of 100 items with an average of 200 students and 20% repeated items per examination. The estimation of the sensitivity and specificity was based on a simulation study. The sensitivity and specificity estimates, and apparent prevalence were used to estimate true prevalence of cheating students in the examinations under study. Results: The specificity in the DGM for different simulation scenarios was between 68 and 98%, while the sensitivity ranged from 60 to 91%. The apparent prevalence was between 0.0 and 3.4%, while the true prevalence ranged from 1.2 to 3.7%. Conclusions: The true prevalence was much lower compared to the students self-reported copying of responses from other students; however, it is important to keep monitoring the pre-knowledge prevalence in order to enforce measures in case an increase occurs.Springer Nature2019-07-29T10:00:51Z2019-07-25T00:00:00Z2019-07-25info:eu-repo/semantics/publishedVersioninfo:eu-repo/semantics/articleapplication/pdfhttp://hdl.handle.net/10773/26317eng1472-692010.1186/s12909-019-1710-zSevero, MiltonSilva-Pereira, FernandaFerreira, Maria AméliaMonteiro, MagdaPereira, Isabelinfo:eu-repo/semantics/openAccessreponame:Repositório Científico de Acesso Aberto de Portugal (Repositórios Cientìficos)instname:Agência para a Sociedade do Conhecimento (UMIC) - FCT - Sociedade da Informaçãoinstacron:RCAAP2024-02-22T11:50:59Zoai:ria.ua.pt:10773/26317Portal AgregadorONGhttps://www.rcaap.pt/oai/openaireopendoar:71602024-03-20T02:59:20.919144Repositório Científico de Acesso Aberto de Portugal (Repositórios Cientìficos) - Agência para a Sociedade do Conhecimento (UMIC) - FCT - Sociedade da Informaçãofalse |
dc.title.none.fl_str_mv |
Item pre-knowledge true prevalence in clinical anatomy: application of gated item response theory model |
title |
Item pre-knowledge true prevalence in clinical anatomy: application of gated item response theory model |
spellingShingle |
Item pre-knowledge true prevalence in clinical anatomy: application of gated item response theory model Severo, Milton Deterministic Gated item response theory model Item pre-knowledge Item sharing Item exposure Repeat items |
title_short |
Item pre-knowledge true prevalence in clinical anatomy: application of gated item response theory model |
title_full |
Item pre-knowledge true prevalence in clinical anatomy: application of gated item response theory model |
title_fullStr |
Item pre-knowledge true prevalence in clinical anatomy: application of gated item response theory model |
title_full_unstemmed |
Item pre-knowledge true prevalence in clinical anatomy: application of gated item response theory model |
title_sort |
Item pre-knowledge true prevalence in clinical anatomy: application of gated item response theory model |
author |
Severo, Milton |
author_facet |
Severo, Milton Silva-Pereira, Fernanda Ferreira, Maria Amélia Monteiro, Magda Pereira, Isabel |
author_role |
author |
author2 |
Silva-Pereira, Fernanda Ferreira, Maria Amélia Monteiro, Magda Pereira, Isabel |
author2_role |
author author author author |
dc.contributor.author.fl_str_mv |
Severo, Milton Silva-Pereira, Fernanda Ferreira, Maria Amélia Monteiro, Magda Pereira, Isabel |
dc.subject.por.fl_str_mv |
Deterministic Gated item response theory model Item pre-knowledge Item sharing Item exposure Repeat items |
topic |
Deterministic Gated item response theory model Item pre-knowledge Item sharing Item exposure Repeat items |
description |
Background: Computer and paper examinations in our days are constructed from an item pool which is regularly updated. Given the way that exams are created, one of the major concerns is the security of the items that are being used in order to ensure a good estimation of abilities. The aim of this study is to measure the prevalence of item pre-knowledge in our medical school. Methods: The Deterministic, Gated Item Response Theory Model (DGM) was applied to estimate the prevalence of students who have had item pre-knowledge from six multiple choice examinations of the Clinical Anatomy course at the Faculty of Medicine of University of Porto. Each examination consisted of 100 items with an average of 200 students and 20% repeated items per examination. The estimation of the sensitivity and specificity was based on a simulation study. The sensitivity and specificity estimates, and apparent prevalence were used to estimate true prevalence of cheating students in the examinations under study. Results: The specificity in the DGM for different simulation scenarios was between 68 and 98%, while the sensitivity ranged from 60 to 91%. The apparent prevalence was between 0.0 and 3.4%, while the true prevalence ranged from 1.2 to 3.7%. Conclusions: The true prevalence was much lower compared to the students self-reported copying of responses from other students; however, it is important to keep monitoring the pre-knowledge prevalence in order to enforce measures in case an increase occurs. |
publishDate |
2019 |
dc.date.none.fl_str_mv |
2019-07-29T10:00:51Z 2019-07-25T00:00:00Z 2019-07-25 |
dc.type.status.fl_str_mv |
info:eu-repo/semantics/publishedVersion |
dc.type.driver.fl_str_mv |
info:eu-repo/semantics/article |
format |
article |
status_str |
publishedVersion |
dc.identifier.uri.fl_str_mv |
http://hdl.handle.net/10773/26317 |
url |
http://hdl.handle.net/10773/26317 |
dc.language.iso.fl_str_mv |
eng |
language |
eng |
dc.relation.none.fl_str_mv |
1472-6920 10.1186/s12909-019-1710-z |
dc.rights.driver.fl_str_mv |
info:eu-repo/semantics/openAccess |
eu_rights_str_mv |
openAccess |
dc.format.none.fl_str_mv |
application/pdf |
dc.publisher.none.fl_str_mv |
Springer Nature |
publisher.none.fl_str_mv |
Springer Nature |
dc.source.none.fl_str_mv |
reponame:Repositório Científico de Acesso Aberto de Portugal (Repositórios Cientìficos) instname:Agência para a Sociedade do Conhecimento (UMIC) - FCT - Sociedade da Informação instacron:RCAAP |
instname_str |
Agência para a Sociedade do Conhecimento (UMIC) - FCT - Sociedade da Informação |
instacron_str |
RCAAP |
institution |
RCAAP |
reponame_str |
Repositório Científico de Acesso Aberto de Portugal (Repositórios Cientìficos) |
collection |
Repositório Científico de Acesso Aberto de Portugal (Repositórios Cientìficos) |
repository.name.fl_str_mv |
Repositório Científico de Acesso Aberto de Portugal (Repositórios Cientìficos) - Agência para a Sociedade do Conhecimento (UMIC) - FCT - Sociedade da Informação |
repository.mail.fl_str_mv |
|
_version_ |
1799137648308125696 |