Exploring pooled analysis of pretested items to monitor the performance of medical students exposed to different curriculum designs
Autor(a) principal: | |
---|---|
Data de Publicação: | 2021 |
Outros Autores: | , , , , |
Tipo de documento: | Artigo |
Idioma: | eng |
Título da fonte: | Repositório Institucional da UNESP |
Texto Completo: | http://dx.doi.org/10.1371/journal.pone.0257293 http://hdl.handle.net/11449/229511 |
Resumo: | Several methods have been proposed for analyzing differences between test scores, such as using mean scores, cumulative deviation, and mixed-effect models. Here, we explore the pooled analysis of retested Progress Test items to monitor the performance of first-year medical students who were exposed to a new curriculum design. This was a cross-sectional study of students in their first year of a medical program who participated in the annual interinstitutional Progress Tests from 2013 to 2019. We analyzed the performance of first-year students in the 2019 test and compared it with that of first-year students taking the test from 2013 to 2018 and encountering the same items. For each item, we calculated odds ratios with 95% confidence intervals; we also performed meta-analyses with fixed effects for each content area in the pooled analysis and presented the odds ratio (OR) with a 95% confidence interval (CI). In all, we used 63 items, which were divided into basic sciences, internal medicine, pediatrics, surgery, obstetrics and gynecology, and public health. Significant differences were found between groups in basic sciences (OR = 1.172 [CI95% 1.005 CI 1.366], p = 0.043) and public health (OR = 1.54 [CI95% CI 1.25–1.897], p < 0.001), which may reflect the characteristics of the new curriculum. Thus, pooled analysis of pretested items may provide indicators of different performance. This method may complement analysis of score differences on benchmark assessments. |
id |
UNSP_860e4f8fb914416e5626cc77fc02e7e7 |
---|---|
oai_identifier_str |
oai:repositorio.unesp.br:11449/229511 |
network_acronym_str |
UNSP |
network_name_str |
Repositório Institucional da UNESP |
repository_id_str |
2946 |
spelling |
Exploring pooled analysis of pretested items to monitor the performance of medical students exposed to different curriculum designsSeveral methods have been proposed for analyzing differences between test scores, such as using mean scores, cumulative deviation, and mixed-effect models. Here, we explore the pooled analysis of retested Progress Test items to monitor the performance of first-year medical students who were exposed to a new curriculum design. This was a cross-sectional study of students in their first year of a medical program who participated in the annual interinstitutional Progress Tests from 2013 to 2019. We analyzed the performance of first-year students in the 2019 test and compared it with that of first-year students taking the test from 2013 to 2018 and encountering the same items. For each item, we calculated odds ratios with 95% confidence intervals; we also performed meta-analyses with fixed effects for each content area in the pooled analysis and presented the odds ratio (OR) with a 95% confidence interval (CI). In all, we used 63 items, which were divided into basic sciences, internal medicine, pediatrics, surgery, obstetrics and gynecology, and public health. Significant differences were found between groups in basic sciences (OR = 1.172 [CI95% 1.005 CI 1.366], p = 0.043) and public health (OR = 1.54 [CI95% CI 1.25–1.897], p < 0.001), which may reflect the characteristics of the new curriculum. Thus, pooled analysis of pretested items may provide indicators of different performance. This method may complement analysis of score differences on benchmark assessments.Department of Neurology Psychology and Psychiatry Botucatu Medical School UNESP–Universidade Estadual Paulista, São PauloDepartment of Surgery and Orthopedics Botucatu Medical School UNESP–Universidade Estadual Paulista, São PauloDepartment of Gynecology and Obstetrics Botucatu Medical School UNESP–Universidade Estadual Paulista, São PauloDepartment of Medical Psychology and Psychiatry School of Medical Sciences UNICAMP–Universidade Estadual de Campinas, São PauloDepartment of Internal Medicine Botucatu Medical School UNESP–Universidade Estadual Paulista, São PauloDepartment of Pediatrics School of Medical Sciences UNICAMP–Universidade Estadual de Campinas, São PauloDepartment of Neurology Psychology and Psychiatry Botucatu Medical School UNESP–Universidade Estadual Paulista, São PauloDepartment of Surgery and Orthopedics Botucatu Medical School UNESP–Universidade Estadual Paulista, São PauloDepartment of Gynecology and Obstetrics Botucatu Medical School UNESP–Universidade Estadual Paulista, São PauloDepartment of Internal Medicine Botucatu Medical School UNESP–Universidade Estadual Paulista, São PauloUniversidade Estadual Paulista (UNESP)Universidade Estadual de Campinas (UNICAMP)Filho, Pedro Tadao Hamamoto [UNESP]de Arruda Lourenção, Pedro Luiz Toledo [UNESP]Abbade, Joélcio Francisco [UNESP]Cecílio-Fernandes, DarioCaramori, Jacqueline Teixeira [UNESP]Bicudo, Angélica Maria2022-04-29T08:33:02Z2022-04-29T08:33:02Z2021-09-01info:eu-repo/semantics/publishedVersioninfo:eu-repo/semantics/articlehttp://dx.doi.org/10.1371/journal.pone.0257293PLoS ONE, v. 16, n. 9 September, 2021.1932-6203http://hdl.handle.net/11449/22951110.1371/journal.pone.02572932-s2.0-85114799057Scopusreponame:Repositório Institucional da UNESPinstname:Universidade Estadual Paulista (UNESP)instacron:UNESPengPLoS ONEinfo:eu-repo/semantics/openAccess2022-04-29T08:33:02Zoai:repositorio.unesp.br:11449/229511Repositório InstitucionalPUBhttp://repositorio.unesp.br/oai/requestopendoar:29462024-05-23T11:04:11.125164Repositório Institucional da UNESP - Universidade Estadual Paulista (UNESP)false |
dc.title.none.fl_str_mv |
Exploring pooled analysis of pretested items to monitor the performance of medical students exposed to different curriculum designs |
title |
Exploring pooled analysis of pretested items to monitor the performance of medical students exposed to different curriculum designs |
spellingShingle |
Exploring pooled analysis of pretested items to monitor the performance of medical students exposed to different curriculum designs Filho, Pedro Tadao Hamamoto [UNESP] |
title_short |
Exploring pooled analysis of pretested items to monitor the performance of medical students exposed to different curriculum designs |
title_full |
Exploring pooled analysis of pretested items to monitor the performance of medical students exposed to different curriculum designs |
title_fullStr |
Exploring pooled analysis of pretested items to monitor the performance of medical students exposed to different curriculum designs |
title_full_unstemmed |
Exploring pooled analysis of pretested items to monitor the performance of medical students exposed to different curriculum designs |
title_sort |
Exploring pooled analysis of pretested items to monitor the performance of medical students exposed to different curriculum designs |
author |
Filho, Pedro Tadao Hamamoto [UNESP] |
author_facet |
Filho, Pedro Tadao Hamamoto [UNESP] de Arruda Lourenção, Pedro Luiz Toledo [UNESP] Abbade, Joélcio Francisco [UNESP] Cecílio-Fernandes, Dario Caramori, Jacqueline Teixeira [UNESP] Bicudo, Angélica Maria |
author_role |
author |
author2 |
de Arruda Lourenção, Pedro Luiz Toledo [UNESP] Abbade, Joélcio Francisco [UNESP] Cecílio-Fernandes, Dario Caramori, Jacqueline Teixeira [UNESP] Bicudo, Angélica Maria |
author2_role |
author author author author author |
dc.contributor.none.fl_str_mv |
Universidade Estadual Paulista (UNESP) Universidade Estadual de Campinas (UNICAMP) |
dc.contributor.author.fl_str_mv |
Filho, Pedro Tadao Hamamoto [UNESP] de Arruda Lourenção, Pedro Luiz Toledo [UNESP] Abbade, Joélcio Francisco [UNESP] Cecílio-Fernandes, Dario Caramori, Jacqueline Teixeira [UNESP] Bicudo, Angélica Maria |
description |
Several methods have been proposed for analyzing differences between test scores, such as using mean scores, cumulative deviation, and mixed-effect models. Here, we explore the pooled analysis of retested Progress Test items to monitor the performance of first-year medical students who were exposed to a new curriculum design. This was a cross-sectional study of students in their first year of a medical program who participated in the annual interinstitutional Progress Tests from 2013 to 2019. We analyzed the performance of first-year students in the 2019 test and compared it with that of first-year students taking the test from 2013 to 2018 and encountering the same items. For each item, we calculated odds ratios with 95% confidence intervals; we also performed meta-analyses with fixed effects for each content area in the pooled analysis and presented the odds ratio (OR) with a 95% confidence interval (CI). In all, we used 63 items, which were divided into basic sciences, internal medicine, pediatrics, surgery, obstetrics and gynecology, and public health. Significant differences were found between groups in basic sciences (OR = 1.172 [CI95% 1.005 CI 1.366], p = 0.043) and public health (OR = 1.54 [CI95% CI 1.25–1.897], p < 0.001), which may reflect the characteristics of the new curriculum. Thus, pooled analysis of pretested items may provide indicators of different performance. This method may complement analysis of score differences on benchmark assessments. |
publishDate |
2021 |
dc.date.none.fl_str_mv |
2021-09-01 2022-04-29T08:33:02Z 2022-04-29T08:33:02Z |
dc.type.status.fl_str_mv |
info:eu-repo/semantics/publishedVersion |
dc.type.driver.fl_str_mv |
info:eu-repo/semantics/article |
format |
article |
status_str |
publishedVersion |
dc.identifier.uri.fl_str_mv |
http://dx.doi.org/10.1371/journal.pone.0257293 PLoS ONE, v. 16, n. 9 September, 2021. 1932-6203 http://hdl.handle.net/11449/229511 10.1371/journal.pone.0257293 2-s2.0-85114799057 |
url |
http://dx.doi.org/10.1371/journal.pone.0257293 http://hdl.handle.net/11449/229511 |
identifier_str_mv |
PLoS ONE, v. 16, n. 9 September, 2021. 1932-6203 10.1371/journal.pone.0257293 2-s2.0-85114799057 |
dc.language.iso.fl_str_mv |
eng |
language |
eng |
dc.relation.none.fl_str_mv |
PLoS ONE |
dc.rights.driver.fl_str_mv |
info:eu-repo/semantics/openAccess |
eu_rights_str_mv |
openAccess |
dc.source.none.fl_str_mv |
Scopus reponame:Repositório Institucional da UNESP instname:Universidade Estadual Paulista (UNESP) instacron:UNESP |
instname_str |
Universidade Estadual Paulista (UNESP) |
instacron_str |
UNESP |
institution |
UNESP |
reponame_str |
Repositório Institucional da UNESP |
collection |
Repositório Institucional da UNESP |
repository.name.fl_str_mv |
Repositório Institucional da UNESP - Universidade Estadual Paulista (UNESP) |
repository.mail.fl_str_mv |
|
_version_ |
1803045764663869440 |