Test preparation breeds success: Two quasi-experimental interventions in the context of the Swiss aptitude test for medical-school admissions.
EMS
Fairness
Medical-school admissions
Practice materials
Scholastic aptitude test
Student selection procedures
Test preparation information
Journal
BMC medical education
ISSN: 1472-6920
Titre abrégé: BMC Med Educ
Pays: England
ID NLM: 101088679
Informations de publication
Date de publication:
13 Sep 2024
13 Sep 2024
Historique:
received:
21
03
2024
accepted:
29
08
2024
medline:
14
9
2024
pubmed:
14
9
2024
entrez:
13
9
2024
Statut:
epublish
Résumé
In Switzerland, the scholastic aptitude test for medical-school selection takes place in three languages. This study examined the effects of two quasi-experimental interventions that aimed to reduce existing differences in test results between the French- and German-speaking language candidates. Between 2018 and 2023, the population of applicants to Swiss medical schools consisted of N = 18'824 German- and French-speaking individuals. Based on a quasi-experimental design, we examined the effects of two interventions regarding preparatory material, in these cohorts. The first intervention (2022) consisting of practice trials in baccalaureate schools in the canton of Fribourg enabled French-speaking candidates primarily from the canton of Fribourg to prepare more intensively with official tasks. Practice trials enable future candidates to complete a published test version under original conditions and thus prepare how to approach the real test. The second intervention (2023) released new preparatory material in all languages for one group of tasks for which differences between the language groups were more pronounced than in the other tasks. The test provider offered this material for free download together with existing preparation materials and thereby enabled more intensive preparation. After the first intervention, the initially small to medium-sized mean differences in z-transformed test scores between French-speaking candidates from Fribourg and German-speaking candidates were nearly eliminated (from 0.39 to 0.05). Also for French-speaking candidates from outside of the canton of Fribourg, the mean differences were smaller than before the intervention (0.48 before, 0.39 after first intervention). After the second intervention, particularly the mean differences in test scores between German-speaking and French-speaking candidates from outside of Fribourg were further reduced (to 0.24). The two interventions regarding material for preparing to participate in the aptitude test affected candidates' test scores. They reduced the gap between German- and French-speaking candidates showing that the additional benefits of commercial offers for test preparation are limited. Hence, offering comparable official preparation material to all language groups enhances test fairness.
Sections du résumé
BACKGROUND
BACKGROUND
In Switzerland, the scholastic aptitude test for medical-school selection takes place in three languages. This study examined the effects of two quasi-experimental interventions that aimed to reduce existing differences in test results between the French- and German-speaking language candidates.
METHODS
METHODS
Between 2018 and 2023, the population of applicants to Swiss medical schools consisted of N = 18'824 German- and French-speaking individuals. Based on a quasi-experimental design, we examined the effects of two interventions regarding preparatory material, in these cohorts. The first intervention (2022) consisting of practice trials in baccalaureate schools in the canton of Fribourg enabled French-speaking candidates primarily from the canton of Fribourg to prepare more intensively with official tasks. Practice trials enable future candidates to complete a published test version under original conditions and thus prepare how to approach the real test. The second intervention (2023) released new preparatory material in all languages for one group of tasks for which differences between the language groups were more pronounced than in the other tasks. The test provider offered this material for free download together with existing preparation materials and thereby enabled more intensive preparation.
RESULTS
RESULTS
After the first intervention, the initially small to medium-sized mean differences in z-transformed test scores between French-speaking candidates from Fribourg and German-speaking candidates were nearly eliminated (from 0.39 to 0.05). Also for French-speaking candidates from outside of the canton of Fribourg, the mean differences were smaller than before the intervention (0.48 before, 0.39 after first intervention). After the second intervention, particularly the mean differences in test scores between German-speaking and French-speaking candidates from outside of Fribourg were further reduced (to 0.24).
CONCLUSIONS
CONCLUSIONS
The two interventions regarding material for preparing to participate in the aptitude test affected candidates' test scores. They reduced the gap between German- and French-speaking candidates showing that the additional benefits of commercial offers for test preparation are limited. Hence, offering comparable official preparation material to all language groups enhances test fairness.
Identifiants
pubmed: 39272134
doi: 10.1186/s12909-024-05971-5
pii: 10.1186/s12909-024-05971-5
doi:
Types de publication
Journal Article
Langues
eng
Sous-ensembles de citation
IM
Pagination
998Informations de copyright
© 2024. The Author(s).
Références
Hell B, Trappmann. Sabrina, Schuler, Heinz. Eine Metaanalyse Der Validität Von Fachspezifischen Studierfähigkeitstests Im Deutschsprachigen Raum. [A meta-analysis of the validity of subject-specific scholastic aptitude tests in german-speaking regions]. Empirische Pädagogik. 2007;21(3):251–70.
Kuncel NR, Credé M, Thomas LL. The validity of self-reported Grade Point averages, Class Ranks, and test scores: a Meta-analysis and review of the literature. Rev Educ Res. 2005;75(1):63–82.
doi: 10.3102/00346543075001063
Chetty R, Deming DJ, Friedman JN. Diversifying society’s leaders? The determinants and causal effects of admission to highly selective private colleges. NBER working paper 31492. http://www.nber.org/papers/w31492
Gilliland SW, Steiner DD. Applicant Reactions to Testing and Selection. In: Schmitt N, editor. The Oxford Handbook of Personnel Assessment and Selection [Internet]. 1st ed. Oxford University Press; 2012 [cited 2024 Mar 8]. pp. 629–66. https://academic.oup.com/edited-volume/28202/chapter/213200238
Tippins NT, Sackett PR. Oswald, Fred. Principles for the Validation and Use of Personnel Selection procedures | Fifth Edition. 5th ed. Bowling Green, OH: Society Industrial Organizational Psychology.
Denker M, Schütte C, Kersting M, Weppert D, Stegt SJ. How can applicants’ reactions to scholastic aptitude tests be improved? A closer look at specific and general tests. Front Educ. 2023;7:931841.
doi: 10.3389/feduc.2022.931841
Lievens F, De Corte W, Brysse K. Applicant perceptions of selection procedures: the role of selection information, belief in tests, and comparative anxiety. Int J Sel Assess. 2003;11(1):67–77.
doi: 10.1111/1468-2389.00227
Clause CS, Delbridge K, Schmitt N, Chan D, Jennings D. Test Preparation activities and Employment Test performance. Hum Perform. 2001;14(2):149–67.
doi: 10.1207/S15327043HUP1402_02
Jackson D, Ward D, Agwu JC, Spruce A. Preparing for selection success: Socio-demographic differences in opportunities and obstacles. Med Educ. 2022;56(9):922–35.
doi: 10.1111/medu.14811
Laurence CO, Zajac IT, Lorimer M, Turnbull DA, Sumner KE. The impact of preparatory activities on medical school selection outcomes: a cross-sectional survey of applicants to the university of Adelaide medical school in 2007. BMC Med Educ. 2013;13(1):159–68.
doi: 10.1186/1472-6920-13-159
Burns GN, Siers BP, Christiansen ND. Effects of providing Pre-test Information and Preparation materials on applicant reactions to selection procedures. Int J Sel Assess. 2008;16(1):73–7.
doi: 10.1111/j.1468-2389.2008.00411.x
Ryan AM, Ployhart RE, Greguras GJ, Schmit MJ. Test preparation programs in selection contexts: self-selection and program effectiveness. Pers Psychol. 1998;51(3):599–621.
doi: 10.1111/j.1744-6570.1998.tb00253.x
Weppert D, Amelung D, Escher M, Troll L, Kadmon M, Listunova L et al. The impact of preparatory activities on the largest clinical aptitude test for prospective medical students in Germany. Front Educ [Internet]. 2023 [cited 2024 Feb 15];8. https://www.frontiersin.org/articles/ https://doi.org/10.3389/feduc.2023.1104464
Kulkarni S, Parry J, Sitch A. An assessment of the impact of formal preparation activities on performance in the University Clinical Aptitude Test (UCAT): a national study. BMC Med Educ. 2022;22(1):747–60.
doi: 10.1186/s12909-022-03811-y
Harris AD, McGregor JC, Perencevich EN, Furuno JP, Zhu J, Peterson DE, et al. The Use and Interpretation of Quasi-experimental studies in Medical Informatics. J Am Med Inf Assoc. 2006;13(1):16–23.
doi: 10.1197/jamia.M1749
Grant AM, Wall TD. The Neglected Science and Art of Quasi-experimentation: Why-to, When-to, and how-to advice for Organizational Researchers. Organ Res Methods. 2009;12(4):653–86.
doi: 10.1177/1094428108320737
Bundesamt für Statistik. Längschnittanalysen im Bildungsbereich [Longitudinal analyses in the education sector]; https://www.bfs.admin.ch/asset/de/28905348 . Accessed 31 July 2024.