Effect of moderation on rubric criteria for inter-rater reliability in an objective structured clinical examination with real patients.
Inter-rater agreement
Moderation
OSCE
Real patients
Video assessment
Journal
Fujita medical journal
ISSN: 2189-7255
Titre abrégé: Fujita Med J
Pays: Japan
ID NLM: 101745077
Informations de publication
Date de publication:
Aug 2022
Aug 2022
Historique:
received:
19
05
2021
accepted:
07
08
2021
entrez:
11
8
2022
pubmed:
12
8
2022
medline:
12
8
2022
Statut:
ppublish
Résumé
Objectives: Objective structured clinical examinations (OSCEs) are used to assess clinical competence in medical education. Evaluations using video-recorded OSCEs are effective in reducing costs in terms of time and human resources. To improve inter-rater reliability, these evaluations undergo moderation in the form of a discussion between the raters to obtain consistency in grading according to the rubric criteria. We examined the effect of moderation related to the rubric criteria on the inter-rater reliability of a video-recorded OSCE with real patients. Forty OSCE videos in which students performed range-of-motion tests at shoulder abduction on real patients were assessed by two raters. The two raters scored videos 1 to 10 without moderation and videos 11 to 40 with moderation each time. The inter-rater reliability of the OSCE was calculated using the weighted kappa coefficient. The mean scores of the weighted kappa coefficients were 0.49 for videos 1 to 10, 0.57 for videos 11 to 20, 0.66 for videos 21 to 30, and 0.82 for videos 31 to 40. An assessment of video-recorded OSCEs was conducted with real patients in a real clinical setting. Repeated moderation improved the inter-rater reliability. This study suggests the effectiveness of moderation in OSCEs with real patients.
Identifiants
pubmed: 35949517
doi: 10.20407/fmj.2021-010
pmc: PMC9358671
doi:
Types de publication
Journal Article
Langues
eng
Pagination
83-87Déclaration de conflit d'intérêts
The authors declare no conflicts of interest.
Références
Oman Med J. 2011 Jul;26(4):219-22
pubmed: 22043423
Am J Pharm Educ. 2010 Apr 12;74(3):44
pubmed: 20498737
J Thorac Cardiovasc Surg. 2014 Dec;148(6):2491-6.e1-2
pubmed: 25308119
Ann Intern Med. 2004 Jun 1;140(11):874-81
pubmed: 15172901
Br Med J. 1975 Feb 22;1(5955):447-51
pubmed: 1115966
World J Surg. 2006 Apr;30(4):527-34
pubmed: 16547622
Ann Acad Med Singap. 2010 Aug;39(8):613-8
pubmed: 20838702
Arthritis Rheum. 2007 Jun 15;57(5):869-76
pubmed: 17530689
Nurse Educ Today. 2007 Jul;27(5):481-90
pubmed: 17070622
Indian J Radiol Imaging. 2010 May;20(2):83-8
pubmed: 20607015
Ann Otol Rhinol Laryngol. 2020 Jun;129(6):565-571
pubmed: 31958985
Patient Educ Couns. 2011 Jun;83(3):472-7
pubmed: 21555198
Adv Health Sci Educ Theory Pract. 1996 Jan;1(3):227-33
pubmed: 24179023
Biometrics. 1977 Mar;33(1):159-74
pubmed: 843571
Fam Med. 2009 Feb;41(2):89-91
pubmed: 19184682
Adv Physiol Educ. 2005 Dec;29(4):197-203
pubmed: 16298956
Nurse Educ Pract. 2019 Feb;35:130-134
pubmed: 30802783
J Educ Eval Health Prof. 2018;15:18
pubmed: 30078286
J Surg Educ. 2019 Jul - Aug;76(4):1088-1093
pubmed: 30709756
Pharm Pract (Granada). 2015 Oct-Dec;13(4):627
pubmed: 26759616
J Surg Educ. 2007 Mar-Apr;64(2):79-87
pubmed: 17462207
BMC Med Educ. 2015 Oct 24;15:183
pubmed: 26498479