Intraoperative robotic-assisted low anterior rectal resection performance assessment using procedure-specific binary metrics and a global rating scale.
Journal
BJS open
ISSN: 2474-9842
Titre abrégé: BJS Open
Pays: England
ID NLM: 101722685
Informations de publication
Date de publication:
02 05 2022
02 05 2022
Historique:
received:
11
08
2021
revised:
23
02
2022
accepted:
04
03
2022
entrez:
11
5
2022
pubmed:
12
5
2022
medline:
14
5
2022
Statut:
ppublish
Résumé
This study aimed to evaluate the use of binary metric-based (proficiency-based progression; PBP) performance assessments and global evaluative assessment of robotic skills (GEARS) of a robotic-assisted low anterior rectal resection (RA-LAR) procedure. A prospective study of video analysis of RA-LAR procedures was carried out using the PBP metrics with binary parameters previously developed, and GEARS. Recordings were collected from five novice surgeons (≤30 RA-LAR previously performed) and seven experienced surgeons (>30 RA-LAR previously performed). Two consultant colorectal surgeons were trained to be assessors in the use of PBP binary parameters to evaluate the procedure phases, surgical steps, errors, and critical errors in male and female patients and GEARS scores. Novice and experienced surgeons were categorized and assessed using PBP metrics and GEARS; mean scores obtained were compared for statistical purpose. Also, the inter-rater reliability (IRR) of these assessment tools was evaluated. Twenty unedited recordings of RA-LAR procedures were blindly assessed. Overall, using PBP metric-based assessment, a subgroup of experienced surgeons made more errors (20 versus 16, P = 0.158) and critical errors (9.2 versus 7.8, P = 0.417) than the novice group, although not significantly. However, during the critical phase of RA-LAR, experienced surgeons made significantly fewer errors than the novice group (95% CI of the difference, Lower = 0.104 - Upper = 5.155, df = 11.9, t = 2.23, p = 0.042), and a similar pattern was observed for critical errors. The PBP metric and GEARS assessment tools distinguished between the objectively assessed performance of experienced and novice colorectal surgeons performing RA-LAR (total error scores with PBP metrics, P = 0.019-0.008; GEARS scores, P = 0.029-0.025). GEARS demonstrated poor IRR (mean IRR 0.49) and weaker discrimination between groups (15-41 per cent difference). PBP binary metrics demonstrated good IRR (mean 0.94) and robust discrimination particularly for total error scores (58-64 per cent). PBP binary metrics seem to be useful for metric-based training for surgeons learning RA-LAR procedures.
Sections du résumé
BACKGROUND
This study aimed to evaluate the use of binary metric-based (proficiency-based progression; PBP) performance assessments and global evaluative assessment of robotic skills (GEARS) of a robotic-assisted low anterior rectal resection (RA-LAR) procedure.
METHOD
A prospective study of video analysis of RA-LAR procedures was carried out using the PBP metrics with binary parameters previously developed, and GEARS. Recordings were collected from five novice surgeons (≤30 RA-LAR previously performed) and seven experienced surgeons (>30 RA-LAR previously performed). Two consultant colorectal surgeons were trained to be assessors in the use of PBP binary parameters to evaluate the procedure phases, surgical steps, errors, and critical errors in male and female patients and GEARS scores. Novice and experienced surgeons were categorized and assessed using PBP metrics and GEARS; mean scores obtained were compared for statistical purpose. Also, the inter-rater reliability (IRR) of these assessment tools was evaluated.
RESULTS
Twenty unedited recordings of RA-LAR procedures were blindly assessed. Overall, using PBP metric-based assessment, a subgroup of experienced surgeons made more errors (20 versus 16, P = 0.158) and critical errors (9.2 versus 7.8, P = 0.417) than the novice group, although not significantly. However, during the critical phase of RA-LAR, experienced surgeons made significantly fewer errors than the novice group (95% CI of the difference, Lower = 0.104 - Upper = 5.155, df = 11.9, t = 2.23, p = 0.042), and a similar pattern was observed for critical errors. The PBP metric and GEARS assessment tools distinguished between the objectively assessed performance of experienced and novice colorectal surgeons performing RA-LAR (total error scores with PBP metrics, P = 0.019-0.008; GEARS scores, P = 0.029-0.025). GEARS demonstrated poor IRR (mean IRR 0.49) and weaker discrimination between groups (15-41 per cent difference). PBP binary metrics demonstrated good IRR (mean 0.94) and robust discrimination particularly for total error scores (58-64 per cent).
CONCLUSIONS
PBP binary metrics seem to be useful for metric-based training for surgeons learning RA-LAR procedures.
Identifiants
pubmed: 35543264
pii: 6583541
doi: 10.1093/bjsopen/zrac041
pmc: PMC9092445
pii:
doi:
Types de publication
Journal Article
Research Support, Non-U.S. Gov't
Langues
eng
Sous-ensembles de citation
IM
Informations de copyright
© The Author(s) 2022. Published by Oxford University Press on behalf of BJS Society Ltd.
Références
Arthroscopy. 2015 Oct;31(10):1854-71
pubmed: 26341047
Surg Endosc. 2015 Jul;29(7):1679-85
pubmed: 25277477
Ann Surg. 1997 May;225(5):445-58
pubmed: 9193173
J Am Coll Surg. 2008 Oct;207(4):560-8
pubmed: 18926460
J Robot Surg. 2016 Sep;10(3):227-31
pubmed: 27039189
Int J Colorectal Dis. 2013 Jun;28(6):815-21
pubmed: 23242270
N Engl J Med. 2014 Aug 28;371(9):794-5
pubmed: 25075620
Arthroscopy. 2015 Aug;31(8):1428-9
pubmed: 26239784
JAMA Surg. 2020 Jul 1;155(7):590-598
pubmed: 32374371
Colorectal Dis. 2020 Dec;22(12):1826-1829
pubmed: 32790893
Br J Surg. 2020 Mar;107(4):467
pubmed: 32129479
Colorectal Dis. 2019 Apr;21(4):489-490
pubmed: 30788899
BMJ Open. 2018 Oct 15;8(10):e020099
pubmed: 30327396
Arthroscopy. 2015 Sep;31(9):1639-54
pubmed: 26129726
Ann Surg. 2005 Feb;241(2):364-72
pubmed: 15650649
J Urol. 2012 Jan;187(1):247-52
pubmed: 22099993
Ann Surg. 2021 Aug 1;274(2):281-289
pubmed: 33630473
Bull Am Coll Surg. 2006 Mar;91(3):8-12
pubmed: 18557053
Medicine (Baltimore). 2014 Nov;93(25):e109
pubmed: 25437022
BJU Int. 2014 May;113(5):836-42
pubmed: 24224500
BJU Int. 2021 Jul;128(1):103-111
pubmed: 33251703
J Appl Behav Anal. 1977 Spring;10(1):141-50
pubmed: 16795543
Surg Endosc. 2015 Nov;29(11):3261-6
pubmed: 25609318
Arthroscopy. 2015 Oct;31(10):1872-9
pubmed: 26298642
BMJ Simul Technol Enhanc Learn. 2016 Feb 8;2(1):1-5
pubmed: 35516451
Surg Endosc. 2013 Sep;27(9):3297-307
pubmed: 23508818
BMJ Open. 2019 Jul 9;9(7):e025992
pubmed: 31289064
Arthroscopy. 2015 Sep;31(9):1655-70
pubmed: 26238730
N Engl J Med. 2013 Oct 10;369(15):1434-42
pubmed: 24106936
Anaesthesia. 2017 Sep;72(9):1117-1124
pubmed: 28741649
Ann Surg. 2020 Aug;272(2):384-392
pubmed: 32675553
Bull Am Coll Surg. 2002 Nov;87(11):10-1
pubmed: 17387916
Arthroscopy. 2018 Jul;34(7):2191-2198
pubmed: 29730215
Ulster Med J. 2012 Sep;81(3):107-13
pubmed: 23620606
Surg Endosc. 2017 Jul;31(7):2820-2828
pubmed: 27815742
Arthroscopy. 2015 Aug;31(8):1430-40
pubmed: 26239785
Ann Surg. 1994 Nov;220(5):626-34
pubmed: 7979610
Colorectal Dis. 2020 Dec;22(12):2232-2242
pubmed: 32663361
Surg Endosc. 2011 Mar;25(3):855-60
pubmed: 20734081
Ann Surg. 2013 Jun;257(6):1025-31
pubmed: 23426342
Am J Surg. 2007 Jun;193(6):797-804
pubmed: 17512301
Arthroscopy. 2021 Apr;37(4):1099-1106.e5
pubmed: 33359814
Int J Cardiol. 2020 May 15;307:48-54
pubmed: 32081470
Ann Surg. 2002 Oct;236(4):458-63; discussion 463-4
pubmed: 12368674