Discrimination, Reliability, Sensitivity, and Specificity of Robotic Surgical Proficiency Assessment With Global Evaluative Assessment of Robotic Skills and Binary Scoring Metrics: Results From a Randomized Controlled Trial.
Basic skill training
GEARS
proficiency-based progression
robotic surgery
training
Journal
Annals of surgery open : perspectives of surgical history, education, and clinical approaches
ISSN: 2691-3593
Titre abrégé: Ann Surg Open
Pays: United States
ID NLM: 101769928
Informations de publication
Date de publication:
Sep 2023
Sep 2023
Historique:
received:
01
05
2023
accepted:
03
06
2023
medline:
25
9
2023
pubmed:
25
9
2023
entrez:
25
9
2023
Statut:
epublish
Résumé
To compare binary metrics and Global Evaluative Assessment of Robotic Skills (GEARS) evaluations of training outcome assessments for reliability, sensitivity, and specificity. GEARS-Likert-scale skills assessment are a widely accepted tool for robotic surgical training outcome evaluations. Proficiency-based progression (PBP) training is another methodology but uses binary performance metrics for evaluations. In a prospective, randomized, and blinded study, we compared conventional with PBP training for a robotic suturing, knot-tying anastomosis task. Thirty-six surgical residents from 16 Belgium residency programs were randomized. In the skills laboratory, the PBP group trained until they demonstrated a quantitatively defined proficiency benchmark. The conventional group were yoked to the same training time but without the proficiency requirement. The final trial was video recorded and assessed with binary metrics and GEARS by robotic surgeons blinded to individual, group, and residency program. Sensitivity and specificity of the two assessment methods were evaluated with area under the curve (AUC) and receiver operating characteristics (ROC) curves. The PBP group made 42% fewer objectively assessed performance errors than the conventional group ( Binary metrics for scoring a robotic VUA task demonstrated better psychometric properties than the GEARS assessment.
Sections du résumé
Objective
UNASSIGNED
To compare binary metrics and Global Evaluative Assessment of Robotic Skills (GEARS) evaluations of training outcome assessments for reliability, sensitivity, and specificity.
Background
UNASSIGNED
GEARS-Likert-scale skills assessment are a widely accepted tool for robotic surgical training outcome evaluations. Proficiency-based progression (PBP) training is another methodology but uses binary performance metrics for evaluations.
Methods
UNASSIGNED
In a prospective, randomized, and blinded study, we compared conventional with PBP training for a robotic suturing, knot-tying anastomosis task. Thirty-six surgical residents from 16 Belgium residency programs were randomized. In the skills laboratory, the PBP group trained until they demonstrated a quantitatively defined proficiency benchmark. The conventional group were yoked to the same training time but without the proficiency requirement. The final trial was video recorded and assessed with binary metrics and GEARS by robotic surgeons blinded to individual, group, and residency program. Sensitivity and specificity of the two assessment methods were evaluated with area under the curve (AUC) and receiver operating characteristics (ROC) curves.
Results
UNASSIGNED
The PBP group made 42% fewer objectively assessed performance errors than the conventional group (
Conclusions
UNASSIGNED
Binary metrics for scoring a robotic VUA task demonstrated better psychometric properties than the GEARS assessment.
Identifiants
pubmed: 37746611
doi: 10.1097/AS9.0000000000000307
pmc: PMC10513364
doi:
Types de publication
Journal Article
Langues
eng
Pagination
e307Informations de copyright
Copyright © 2023 The Author(s). Published by Wolters Kluwer Health, Inc.
Références
Arthroscopy. 2015 Oct;31(10):1854-71
pubmed: 26341047
Med Educ. 2015 Jun;49(6):560-75
pubmed: 25989405
J Am Acad Child Adolesc Psychiatry. 1997 Oct;36(10):1349-56
pubmed: 9334547
J Am Coll Surg. 2008 Oct;207(4):560-8
pubmed: 18926460
Arch Esp Urol. 2018 Jan;71(1):97-107
pubmed: 29336338
N Engl J Med. 2013 Oct 10;369(15):1434-42
pubmed: 24106936
Curr Opin Urol. 2020 Jan;30(1):65-72
pubmed: 31725006
Br J Surg. 1997 Feb;84(2):273-8
pubmed: 9052454
Ann Surg. 2013 Jun;257(6):1025-31
pubmed: 23426342
Arthroscopy. 2015 Sep;31(9):1639-54
pubmed: 26129726
Eur Urol. 2022 Apr;81(4):394-395
pubmed: 35074249
CA Cancer J Clin. 2014 Jan-Feb;64(1):9-29
pubmed: 24399786
Ann Surg. 2021 Aug 1;274(2):281-289
pubmed: 33630473
BJU Int. 2014 May;113(5):836-42
pubmed: 24224500
BJU Int. 2021 Jul;128(1):103-111
pubmed: 33251703
Ann Surg. 2019 Nov;270(5):735-737
pubmed: 31634177
Arthroscopy. 2015 Oct;31(10):1872-9
pubmed: 26298642
BMJ Simul Technol Enhanc Learn. 2016 Feb 8;2(1):1-5
pubmed: 35516451
Curr Urol Rep. 2017 Aug;18(8):58
pubmed: 28647793
J Robot Surg. 2016 Sep;10(3):227-31
pubmed: 27039189
Surg Endosc. 2003 Oct;17(10):1525-9
pubmed: 14502403
BMJ Open. 2019 Jul 9;9(7):e025992
pubmed: 31289064
Eur Urol Focus. 2018 Sep;4(5):674-676
pubmed: 30217631
Surg Endosc. 2021 Aug;35(8):4285-4294
pubmed: 32857241
Ann Surg. 2020 Aug;272(2):384-392
pubmed: 32675553
Arthroscopy. 2015 Aug;31(8):1430-40
pubmed: 26239785
Int J Med Robot. 2008 Mar;4(1):1-2
pubmed: 18314918
Ulster Med J. 2012 Sep;81(3):107-13
pubmed: 23620606
BMJ Open. 2018 Oct 15;8(10):e020099
pubmed: 30327396
ANZ J Surg. 2014 Jul-Aug;84(7-8):568-73
pubmed: 22943748
Ann Surg. 2005 Feb;241(2):364-72
pubmed: 15650649
BJU Int. 2022 Oct;130(4):528-535
pubmed: 37382230
Am J Surg. 2007 Jun;193(6):797-804
pubmed: 17512301
J Urol. 2019 Mar;201(3):461-469
pubmed: 30053510
Arthroscopy. 2021 Apr;37(4):1099-1106.e5
pubmed: 33359814
Ann Surg. 2002 Oct;236(4):458-63; discussion 463-4
pubmed: 12368674
Med Educ. 2003 Sep;37(9):830-7
pubmed: 14506816