Psychometric Properties of Visual Indicators of Teaching and Learning Success "VITALS" Instrument for Evaluation of Clinical Teachers.
clinical teaching
generalizability theory
reliability
tutors’ evaluation
validity
Journal
Advances in medical education and practice
ISSN: 1179-7258
Titre abrégé: Adv Med Educ Pract
Pays: New Zealand
ID NLM: 101562700
Informations de publication
Date de publication:
2021
2021
Historique:
received:
05
05
2021
accepted:
28
07
2021
entrez:
25
8
2021
pubmed:
26
8
2021
medline:
26
8
2021
Statut:
epublish
Résumé
We have previously developed an instrument for students' evaluation of clinical teachers that we called Visual Indicators of Clinical Teaching and Learning Success (VITALS). This study measures the reliability of VITALS as an instrument for student evaluation of clinical tutors. Additionally, the study explores the minimum number of student raters necessary for an acceptable reliability, and provides evidence of construct validity of the evaluation scores. This retrospective study included 1825 evaluation forms completed by medical students evaluating clinical tutors using the VITALS instrument. Reliability was measured by applying generalizability theory (G-theory) analysis using a two-facet design (raters and items). A D-study was used to determine the minimum number of raters required to achieve a reliability ≥0.80. Face validity was tested by measuring tutors' degree of agreement with the items of the study instrument. The overall G-coefficient was 0.89. The subject of measurement (clinical tutors' scores) represented 15.8% of the variance across all tutors and items. The variance due to the interaction between raters (students) and tutors contributed to 43.5%, while the variance due to items was negligible. The remaining 40% of the variance was due to unexplained sources of error. The D-study demonstrated that a minimum of 12 raters (students) are required to achieve a reliability of 0.80. Finally, most of the clinical tutors agreed that all items in the instrument were appropriate. We demonstrate that VITALS exhibits good psychometric properties. There should be at least 12 students rating each clinical tutor to have an acceptable level of reliability for the study instrument. Face validity of the study instrument was evidenced by its high level of approval among clinical tutors.
Identifiants
pubmed: 34429680
doi: 10.2147/AMEP.S318798
pii: 318798
pmc: PMC8380291
doi:
Types de publication
Journal Article
Langues
eng
Pagination
905-911Informations de copyright
© 2021 Al-Yousuf et al.
Déclaration de conflit d'intérêts
The authors report no conflicts of interest and no financial interest in this work.
Références
Acad Med. 2000 Feb;75(2):161-6
pubmed: 10693849
Am J Med. 2006 Feb;119(2):166.e7-16
pubmed: 16443422
Perspect Med Educ. 2015 Oct;4(5):264-7
pubmed: 26399537
J Educ Eval Health Prof. 2017 Jun 29;14:14
pubmed: 28669146
Acad Med. 2010 Nov;85(11):1732-8
pubmed: 20881822
Med Educ. 2005 Dec;39(12):1221-9
pubmed: 16313581
Adv Health Sci Educ Theory Pract. 2011 May;16(2):197-210
pubmed: 20978840
Acad Med. 1999 Oct;74(10 Suppl):S19-21
pubmed: 10536582
Adv Med Educ Pract. 2018 Nov 30;9:881-886
pubmed: 30555266
BMC Med Educ. 2017 Mar 16;17(1):56
pubmed: 28302151
J Med Ethics Hist Med. 2019 Sep 4;12:11
pubmed: 32328224
Clin Teach. 2015 Jun;12(3):197-202
pubmed: 26009956
Med Teach. 2017 Aug;39(8):820-827
pubmed: 28532203
Med Educ. 2016 Jul;50(7):730-7
pubmed: 27295477
PLoS One. 2011 May 03;6(5):e19142
pubmed: 21559275
Acad Med. 2014 Apr;89(4):671-7
pubmed: 24556764
Acad Med. 1998 Dec;73(12):1294-8
pubmed: 9883207
Adv Health Sci Educ Theory Pract. 2007 Feb;12(1):55-69
pubmed: 17160501
Eval Health Prof. 2016 Mar;39(1):21-32
pubmed: 25280728
BMC Med Educ. 2019 May 21;19(1):155
pubmed: 31113457
Educ Health (Abingdon). 2001;14(2):267-76
pubmed: 14742025
J Gen Intern Med. 2010 Dec;25(12):1337-45
pubmed: 20703952