Development and initial validation of an instrument for video-based assessment of technical skill in ERCP.
Journal
Gastrointestinal endoscopy
ISSN: 1097-6779
Titre abrégé: Gastrointest Endosc
Pays: United States
ID NLM: 0010505
Informations de publication
Date de publication:
04 2021
04 2021
Historique:
received:
21
05
2020
accepted:
24
07
2020
pubmed:
3
8
2020
medline:
1
6
2021
entrez:
3
8
2020
Statut:
ppublish
Résumé
The accurate measurement of technical skill in ERCP is essential for endoscopic training, quality assurance, and coaching of this procedure. Hypothesizing that technical skill can be measured by analysis of ERCP videos, we aimed to develop and validate a video-based ERCP skill assessment tool. Based on review of procedural videos, the task of ERCP was deconstructed into its basic components by an expert panel that developed an initial version of the Bethesda ERCP Skill Assessment Tool (BESAT). Subsequently, 2 modified Delphi panels and 3 validation exercises were conducted with the goal of iteratively refining the tool. Fully crossed generalizability studies investigated the contributions of assessors, ERCP performance, and technical elements to reliability. Twenty-nine technical elements were initially generated from task deconstruction. Ultimately, after iterative refinement, the tool comprised 6 technical elements and 11 subelements. The developmental process achieved consistent improvements in the performance characteristics of the tool with every iteration. For the most recent version of the tool, BESAT-v4, the generalizability coefficient (a reliability index) was .67. Most variance in BESAT scores (43.55%) was attributed to differences in endoscopists' skill, indicating that the tool can reliably differentiate between endoscopists based on video analysis. Video-based assessment of ERCP skill appears to be feasible with a novel instrument that demonstrates favorable validity evidence. Future steps include determining whether the tool can discriminate between endoscopists of varying experience levels and predict important outcomes in clinical practice.
Sections du résumé
BACKGROUND AND AIMS
The accurate measurement of technical skill in ERCP is essential for endoscopic training, quality assurance, and coaching of this procedure. Hypothesizing that technical skill can be measured by analysis of ERCP videos, we aimed to develop and validate a video-based ERCP skill assessment tool.
METHODS
Based on review of procedural videos, the task of ERCP was deconstructed into its basic components by an expert panel that developed an initial version of the Bethesda ERCP Skill Assessment Tool (BESAT). Subsequently, 2 modified Delphi panels and 3 validation exercises were conducted with the goal of iteratively refining the tool. Fully crossed generalizability studies investigated the contributions of assessors, ERCP performance, and technical elements to reliability.
RESULTS
Twenty-nine technical elements were initially generated from task deconstruction. Ultimately, after iterative refinement, the tool comprised 6 technical elements and 11 subelements. The developmental process achieved consistent improvements in the performance characteristics of the tool with every iteration. For the most recent version of the tool, BESAT-v4, the generalizability coefficient (a reliability index) was .67. Most variance in BESAT scores (43.55%) was attributed to differences in endoscopists' skill, indicating that the tool can reliably differentiate between endoscopists based on video analysis.
CONCLUSIONS
Video-based assessment of ERCP skill appears to be feasible with a novel instrument that demonstrates favorable validity evidence. Future steps include determining whether the tool can discriminate between endoscopists of varying experience levels and predict important outcomes in clinical practice.
Identifiants
pubmed: 32739484
pii: S0016-5107(20)34650-2
doi: 10.1016/j.gie.2020.07.055
pmc: PMC8961206
mid: NIHMS1643805
pii:
doi:
Types de publication
Journal Article
Langues
eng
Sous-ensembles de citation
IM
Pagination
914-923Subventions
Organisme : NIDDK NIH HHS
ID : P30 DK123704
Pays : United States
Organisme : NIDDK NIH HHS
ID : U01 DK104833
Pays : United States
Commentaires et corrections
Type : CommentIn
Informations de copyright
Copyright © 2021 American Society for Gastrointestinal Endoscopy. All rights reserved.
Références
Gastrointest Endosc. 2019 Jun;89(6):1222-1230.e2
pubmed: 30844372
Gastroenterology. 2018 Nov;155(5):1483-1494.e7
pubmed: 30056094
Surg Endosc. 2020 Feb;34(2):521-535
pubmed: 31748927
Med Educ. 2004 Sep;38(9):1006-12
pubmed: 15327684
Gastrointest Endosc. 2011 Jun;73(6):1232-9.e2
pubmed: 21628015
Surg Endosc. 2012 Sep;26(9):2489-503
pubmed: 22476826
J Minim Invasive Gynecol. 2017 Mar - Apr;24(3):426-431
pubmed: 28063907
Adv Health Sci Educ Theory Pract. 2013 May;18(2):291-303
pubmed: 22484964
Am J Surg. 2020 Feb;219(2):221-226
pubmed: 31918843
Gastrointest Endosc. 2002 Nov;56(5):652-6
pubmed: 12397271
Am J Med Qual. 2013 May-Jun;28(3):256-60
pubmed: 22930708
Med Teach. 2012;34(11):960-92
pubmed: 23140303
Clin Gastroenterol Hepatol. 2017 Dec;15(12):1866-1875.e3
pubmed: 28606848
Am J Gastroenterol. 2015 Jan;110(1):91-101
pubmed: 25448869
Med Care. 2013 Dec;51(12):1040-7
pubmed: 24226304
PLoS One. 2018 Jan 2;13(1):e0190580
pubmed: 29293635
Am J Obstet Gynecol. 2003 Sep;189(3):688-91
pubmed: 14526294
Med Teach. 2005 Nov;27(7):639-43
pubmed: 16332558
Gastrointest Endosc. 2018 Mar;87(3):766-775
pubmed: 28859953
N Engl J Med. 2013 Oct 10;369(15):1434-42
pubmed: 24106936
Trials. 2016 Mar 03;17(1):120
pubmed: 26941086
Gastrointest Endosc. 1998 Jul;48(1):1-10
pubmed: 9684657
Clin Gastroenterol Hepatol. 2013 Apr;11(4):430-436.e1
pubmed: 23313840
Gastrointest Endosc. 2001 Oct;54(4):425-34
pubmed: 11577302
JAMA Surg. 2017 Apr 1;152(4):318-325
pubmed: 27973648
Ann Surg. 2016 Sep;264(3):482-91
pubmed: 27433897
Gastrointest Endosc. 2019 Jun;89(6):1231-1233
pubmed: 31104751
Gastrointest Endosc. 2019 Jun;89(6):1212-1221
pubmed: 30825535