RAS-NOTECHS: validity and reliability of a tool for measuring non-technical skills in robotic-assisted surgery settings.
Assessment
Da Vinci system
Human factors
Non-technical skills
Robotic-assisted surgery
Surgical teamwork
Journal
Surgical endoscopy
ISSN: 1432-2218
Titre abrégé: Surg Endosc
Pays: Germany
ID NLM: 8806653
Informations de publication
Date de publication:
03 2022
03 2022
Historique:
received:
12
10
2020
accepted:
26
03
2021
pubmed:
13
4
2021
medline:
4
3
2022
entrez:
12
4
2021
Statut:
ppublish
Résumé
Non-technical skills (NTS) are essential for safe surgical practice as they impact workflow and patient outcomes. Observational tools to measure operating room (OR) teams' NTS have been introduced. However, there are none that account for the specific teamwork challenges introduced by robotic-assisted surgery (RAS). We set out to develop and content-validate a tool to assess multidisciplinary NTS in RAS. Stepwise, multi-method procedure. Observations in different surgical departments and a scoping literature review were first used to compile a set of RAS-specific teamwork behaviours. This list was refined and expert validated using a Delphi consensus approach consisting of qualitative interviews and a quantitative survey. Then, RAS-specific behaviours were merged with a well-established assessment tool on OR teamwork (NOTECHS II). Finally, the new tool-RAS-NOTECHS-was applied in standardized observations of real-world procedures to test its reliability (inter-rater agreement via intra-class correlations). Our scoping review revealed 5242 articles, of which 21 were included based on pre-established inclusion criteria. We elicited 16 RAS-specific behaviours from the literature base. These were synthesized with further 18 behavioural markers (obtained from 12 OR-observations) into a list of 26 behavioural markers. This list was reviewed by seven RAS experts and condensed to 15 expert-validated RAS-specific behavioural markers which were then merged into NOTECHS II. For five observations of urologic RAS procedures (duration: 13 h and 41 min), inter-rater agreement for identification of behavioural markers was strong. Agreement of RAS-NOTECHS scores indicated moderate to strong agreement. RAS-NOTECHS is the first observational tool for multidisciplinary NTS in RAS. In preliminary application, it has been shown to be reliable. Since RAS is rapidly increasing and challenges for effective and safe teamwork remain at the forefront of quality and safety of surgical care, RAS-NOTECHS may contribute to training and improvement efforts in technology-facilitated surgeries.
Sections du résumé
BACKGROUND
Non-technical skills (NTS) are essential for safe surgical practice as they impact workflow and patient outcomes. Observational tools to measure operating room (OR) teams' NTS have been introduced. However, there are none that account for the specific teamwork challenges introduced by robotic-assisted surgery (RAS). We set out to develop and content-validate a tool to assess multidisciplinary NTS in RAS.
METHODOLOGY
Stepwise, multi-method procedure. Observations in different surgical departments and a scoping literature review were first used to compile a set of RAS-specific teamwork behaviours. This list was refined and expert validated using a Delphi consensus approach consisting of qualitative interviews and a quantitative survey. Then, RAS-specific behaviours were merged with a well-established assessment tool on OR teamwork (NOTECHS II). Finally, the new tool-RAS-NOTECHS-was applied in standardized observations of real-world procedures to test its reliability (inter-rater agreement via intra-class correlations).
RESULTS
Our scoping review revealed 5242 articles, of which 21 were included based on pre-established inclusion criteria. We elicited 16 RAS-specific behaviours from the literature base. These were synthesized with further 18 behavioural markers (obtained from 12 OR-observations) into a list of 26 behavioural markers. This list was reviewed by seven RAS experts and condensed to 15 expert-validated RAS-specific behavioural markers which were then merged into NOTECHS II. For five observations of urologic RAS procedures (duration: 13 h and 41 min), inter-rater agreement for identification of behavioural markers was strong. Agreement of RAS-NOTECHS scores indicated moderate to strong agreement.
CONCLUSIONS
RAS-NOTECHS is the first observational tool for multidisciplinary NTS in RAS. In preliminary application, it has been shown to be reliable. Since RAS is rapidly increasing and challenges for effective and safe teamwork remain at the forefront of quality and safety of surgical care, RAS-NOTECHS may contribute to training and improvement efforts in technology-facilitated surgeries.
Identifiants
pubmed: 33844085
doi: 10.1007/s00464-021-08474-2
pii: 10.1007/s00464-021-08474-2
pmc: PMC8505574
mid: NIHMS1697928
doi:
Types de publication
Journal Article
Research Support, Non-U.S. Gov't
Research Support, U.S. Gov't, P.H.S.
Review
Langues
eng
Sous-ensembles de citation
IM
Pagination
1916-1926Subventions
Organisme : AHRQ HHS
ID : R01 HS026491
Pays : United States
Informations de copyright
© 2021. The Author(s).
Références
Pfandler M, Stefan P, Mehren C, Lazarovici M, Weigl M (2019) Technical and nontechnical skills in surgery: a simulated operating room environment study. Spine (Phila Pa 1976) 44(23):E1396–E400. https://doi.org/10.1097/brs.0000000000003154
doi: 10.1097/brs.0000000000003154
Hull L, Arora S, Aggarwal R, Darzi A, Vincent C, Sevdalis N (2012) The impact of nontechnical skills on technical performance in surgery: a systematic review. J Am Coll Surg 214(2):214–230. https://doi.org/10.1016/j.jamcollsurg.2011.10.016
doi: 10.1016/j.jamcollsurg.2011.10.016
pubmed: 22200377
Schmutz JB, Meier LL, Manser T (2019) How effective is teamwork really? The relationship between teamwork and performance in healthcare teams: a systematic review and meta-analysis. BMJ Open 9(9):e028280. https://doi.org/10.1136/bmjopen-2018-028280
doi: 10.1136/bmjopen-2018-028280
pubmed: 31515415
pmcid: 6747874
Doumouras AG, Hamidi M, Lung K, Tarola CL, Tsao MW, Scott JW et al (2017) Non-technical skills of surgeons and anaesthetists in simulated operating theatre crises. Br J Surg 104(8):1028–1036. https://doi.org/10.1002/bjs.10526
doi: 10.1002/bjs.10526
pubmed: 28376246
Fecso AB, Kuzulugil SS, Babaoglu C, Bener AB, Grantcharov TP (2018) Relationship between intraoperative non-technical performance and technical events in bariatric surgery. Br J Surg 105(8):1044–1050. https://doi.org/10.1002/bjs.10811
doi: 10.1002/bjs.10811
pubmed: 29601079
Gjeraa K, Spanager L, Konge L, Petersen RH, Østergaard D (2016) Non-technical skills in minimally invasive surgery teams: a systematic review. Surg Endosc 30(12):5185–5199. https://doi.org/10.1007/s00464-016-4890-1
doi: 10.1007/s00464-016-4890-1
pubmed: 27066972
Flin R, O’Connor P, Crichton M (2008) Safety at the sharp end: a guide to non-technical Skills. Ashgate, Aldershot
Gillespie BM, Harbeck E, Kang E, Steel C, Fairweather N, Chaboyer W (2017) Correlates of non-technical skills in surgery: a prospective study. BMJ Open 7(1):e014480. https://doi.org/10.1136/bmjopen-2016-014480
doi: 10.1136/bmjopen-2016-014480
pubmed: 28137931
pmcid: 5293872
Al-Hakim L (2011) The impact of preventable disruption on the operative time for minimally invasive surgery. Surg Endosc 25(10):3385–3392. https://doi.org/10.1007/s00464-011-1735-9
doi: 10.1007/s00464-011-1735-9
pubmed: 21590502
Mushtaq F, O’Driscoll C, Smith F, Wilkins D, Kapur N, Lawton R (2018) Contributory factors in surgical incidents as delineated by a confidential reporting system. Ann R Coll Surg Engl 100(5):401–405. https://doi.org/10.1308/rcsann.2018.0025
doi: 10.1308/rcsann.2018.0025
pubmed: 29543056
pmcid: 5956595
Siu J, Maran N, Paterson-Brown S (2016) Observation of behavioural markers of non-technical skills in the operating room and their relationship to intra-operative incidents. Surgeon 14(3):119–128. https://doi.org/10.1016/j.surge.2014.06.005
doi: 10.1016/j.surge.2014.06.005
pubmed: 25022767
Li N, Marshall D, Sykes M, McCulloch P, Shalhoub J, Maruthappu M (2018) Systematic review of methods for quantifying teamwork in the operating theatre. BJS Open 2(2):42–51. https://doi.org/10.1002/bjs5.40
doi: 10.1002/bjs5.40
pubmed: 29951628
pmcid: 5952378
McMullan RD, Urwin R, Sunderland N, Westbrook J (2020) Observational tools that quantify nontechnical skills in the operating room: a systematic review. J Surg Res 247:306–322. https://doi.org/10.1016/j.jss.2019.10.012
doi: 10.1016/j.jss.2019.10.012
pubmed: 31706538
Mishra A, Catchpole K, McCulloch P (2009) The Oxford NOTECHS System: reliability and validity of a tool for measuring teamwork behaviour in the operating theatre. Qual Saf Health Care 18(2):104–108. https://doi.org/10.1136/qshc.2007.024760
doi: 10.1136/qshc.2007.024760
pubmed: 19342523
Catchpole K, Bisantz A, Hallbeck MS, Weigl M, Randell R, Kossack M et al (2019) Human factors in robotic assisted surgery: lessons from studies “in the Wild.” Appl Ergon 78:270–276. https://doi.org/10.1016/j.apergo.2018.02.011
doi: 10.1016/j.apergo.2018.02.011
pubmed: 29478667
Gillespie BM, Gillespie J, Boorman RJ, Granqvist K, Stranne J, Erichsen-Andersson A (2020) The impact of robotic-assisted surgery on team performance: a systematic mixed studies review. Hum Factors. https://doi.org/10.1177/0018720820928624
doi: 10.1177/0018720820928624
pubmed: 32613863
Tiferes J, Hussein AA, Bisantz A, Kozlowski JD, Sharif MA, Winder NM et al (2016) the loud surgeon behind the console: understanding team activities during robot-assisted surgery. J Surg Educ 73(3):504–512. https://doi.org/10.1016/j.jsurg.2015.12.009
doi: 10.1016/j.jsurg.2015.12.009
pubmed: 27068189
Raison N, Wood T, Brunckhorst O, Abe T, Ross T, Challacombe B et al (2017) Development and validation of a tool for non-technical skills evaluation in robotic surgery-the ICARS system. Surg Endosc 31(12):5403–5410. https://doi.org/10.1007/s00464-017-5622-x
doi: 10.1007/s00464-017-5622-x
pubmed: 28634630
Robertson ER, Hadi M, Morgan LJ, Pickering SP, Collins G, New S et al (2014) Oxford NOTECHS II: a modified theatre team non-technical skills scoring system. PLoS ONE 9(3):e90320. https://doi.org/10.1371/journal.pone.0090320
doi: 10.1371/journal.pone.0090320
pubmed: 24594911
pmcid: 3942429
Peters MDJ GC, McInerney P, Munn Z, Tricco AC, Khalil, H. Chapter 11: Scoping reviews (2020 version). In: Aromataris E, Munn Z (eds) JBI manual for evidence synthesis. JBI. 2020. http://doi.org/ https://doi.org/10.46658/JBIMES-20-12 . https://reviewersmanual.joannabriggs.org/
Flin R, Martin L, Goeters KM, Hoermann H, Amalberti R, Valot C et al (2003) Development of the NOTECHS (non-technical skills) system for assessing pilots’ CRM skills. Hum Factors Aerosp Saf 3:97–119
Myklebust MV, Storheim H, Hartvik M, Dysvik E (2020) Anesthesia professionals’ perspectives of teamwork during robotic-assisted surgery. Aorn J 111(1):87–96. https://doi.org/10.1002/aorn.12897
doi: 10.1002/aorn.12897
pubmed: 31886556
Iqbal S, Pipon-Young L (2009) The Delphi method. A step-by-step guide. Psychologist 22:598–601
Hull L, Arora S, Kassab E, Kneebone R, Sevdalis N (2011) Observational teamwork assessment for surgery: content validation and tool refinement. J Am Coll Surg 212(2):234-43.e1–5. https://doi.org/10.1016/j.jamcollsurg.2010.11.001
doi: 10.1016/j.jamcollsurg.2010.11.001
pubmed: 21276535
Koo TK, Li MY (2016) A guideline of selecting and reporting intraclass correlation coefficients for reliability research. J Chiropr Med 15(2):155–163. https://doi.org/10.1016/j.jcm.2016.02.012
doi: 10.1016/j.jcm.2016.02.012
pubmed: 27330520
pmcid: 4913118
Gwet K (2002) Kappa statistic is not satisfactory for assessing the extent of agreement between raters. Stat Methods Inter-rater Reliab Assess 1:1–6
Pelikan HR (2018) “What’s going on there?” Negotiating common ground in robotic vs. open surgery: a comparison of surgeon-initiated requests for action in open and robotic surgery. University of Twente, Enschede
Hsu RL, Kaye AD, Urman RD (2013) Anesthetic challenges in robotic-assisted urologic surgery. Rev Urol 15(4):178–184. https://doi.org/10.3909/riu0589
doi: 10.3909/riu0589
pubmed: 24659914
pmcid: 3922322
Cao CG, Taylor H (2004) Effects of new technology on the operating room team. Department of Mechanical Engineering, Tufts University, Medford, pp 309–12
Higuchi T, Gettman M (2011) Robotic instrumentation, personnel and operating room setup. In: Su LM (ed) Atlas of robotic urologic surgery. Humana Press, New York, pp 15–30. https://doi.org/10.1007/978-1-60761-026-7_2
doi: 10.1007/978-1-60761-026-7_2
Lai F, Entin E (2005) Robotic surgery and the operating room team. Proc Hum Factors Ergon Soc Annu Meet 49(11):1070–1073. https://doi.org/10.1177/154193120504901115
doi: 10.1177/154193120504901115
Wastler KE (2015) Robotic surgical and anesthesia communication tool. J Robot Surg 9(1):97–98. https://doi.org/10.1007/s11701-014-0494-8
doi: 10.1007/s11701-014-0494-8
pubmed: 26530978
Payne TN, Pitter MC (2011) Robotic-assisted surgery for the community gynecologist: can it be adopted? Clin Obstet Gynecol 54(3):391–411. https://doi.org/10.1097/GRF.0b013e31822b4998
doi: 10.1097/GRF.0b013e31822b4998
pubmed: 21857170
Satchidanand A, Higginbotham J, Bisantz A, Aldhaam NA, Elsayed AS, Kannappan R et al (2019) Vague swirling ends in no communication: referencing strategies during robot assisted surgery. J Urol 201(4):E288–E289
Raheem S, Ahmed YE, Hussein AA, Johnson A, Cavuoto L, May P et al (2018) Variability and interpretation of communication taxonomy during robot-assisted surgery: do we all speak the same language? BJU Int 122(1):99–105. https://doi.org/10.1111/bju.14150
doi: 10.1111/bju.14150
pubmed: 29388382
Randell R, Greenhalgh J, Hindmarsh J, Honey S, Pearman A, Alvarado N et al (2019) How do team experience and relationships shape new divisions of labour in robot-assisted surgery? A realist investigation. Health. https://doi.org/10.1177/1363459319874115
doi: 10.1177/1363459319874115
pubmed: 31522572
Randell R, Honey S, Hindmarsh J, Alvarado N, Greenhalgh J, Pearman A et al (2017) A realist process evaluation of robot-assisted surgery: integration into routine practice and impacts on communication, collaboration and decision-making. Health Serv Deliv Res. https://doi.org/10.3310/hsdr05200
doi: 10.3310/hsdr05200
Alvarado N, Honey S, Greenhalgh J, Pearman A, Dowding D, Cope A et al (2017) Eliciting context-mechanism-outcome configurations: experiences from a realist evaluation investigating the impact of robotic surgery on teamwork in the operating theatre. Evaluation 23(4):444–462. https://doi.org/10.1177/1356389017732172
doi: 10.1177/1356389017732172
Gill A, Randell R (2016) Robotic surgery and its impact on teamwork in the operating theatre. J Perioper Pract 26(3):42–45. https://doi.org/10.1177/175045891602600303
doi: 10.1177/175045891602600303
pubmed: 27149832
El-Hamamsy D, Walton TJ, Griffiths TRL, Anderson ES, Tincello DG (2020) Surgeon-team separation in robotic theaters: a qualitative observational and interview study. Female Pelvic Med Reconstr Surg 26(2):86–91. https://doi.org/10.1097/spv.0000000000000829
doi: 10.1097/spv.0000000000000829
pubmed: 31990793
Nyssen A-S, Blavier A (2009) Verbal communication as a sign of adaptation in socio-technical systems: the case of robotic surgery. In: 9th Bi-annual international conference on naturalistic decision making (NDM9), vol 9. pp 267–72. http://doi.org/ https://doi.org/10.14236/ewic/NDM2009.39
Almeras C (2019) Operating room communication in robotic surgery: place, modalities and evolution of a safe system of interaction. J Visc Surg 156(5):397–403. https://doi.org/10.1016/j.jviscsurg.2019.02.004
doi: 10.1016/j.jviscsurg.2019.02.004
pubmed: 30826303
Sergeeva A, Huysman M, Faraj S (2015) Transforming work practices of operating room teams: the case of the Da Vinci robot. In: Transforming surgery practice with the Da Vinci robot thirty sixth international conference on information systems. Fort Worth.
Viswanathan M, Berkman N (2011) Development of the RTI Item Bank on risk of bias and precision of observational studies [Internet] Appendix A, AC1 Statistic. Agency for Healthcare Research and Quality (US), Rockville. https://www.ncbi.nlm.nih.gov/books/NBK82266/ .
Fleiss J (1971) Measuring nominal scale agreement among many raters. Psychol Bull 76:378–382. https://doi.org/10.1037/h0031619
doi: 10.1037/h0031619
Peters BS, Armijo PR, Krause C, Choudhury SA, Oleynikov D (2018) Review of emerging surgical robotic technology. Surg Endosc 32(4):1636–1655. https://doi.org/10.1007/s00464-018-6079-2
doi: 10.1007/s00464-018-6079-2
pubmed: 29442240
Perez RE, Schwaitzberg SD (2019) Robotic surgery: finding value in 2019 and beyond. Ann Laparosc Endosc Surg. https://doi.org/10.21037/ales.2019.05.02
doi: 10.21037/ales.2019.05.02
Sheetz KH, Claflin J, Dimick JB (2020) Trends in the adoption of robotic surgery for common surgical procedures. JAMA Netw Open 3(1):e1918911. https://doi.org/10.1001/jamanetworkopen.2019.18911
doi: 10.1001/jamanetworkopen.2019.18911
pubmed: 31922557
pmcid: 6991252
Undre S, Sevdalis N, Healey AN, Darzi A, Vincent CA (2007) Observational teamwork assessment for surgery (OTAS): refinement and application in urological surgery. World J Surg 31(7):1373–1381. https://doi.org/10.1007/s00268-007-9053-z
doi: 10.1007/s00268-007-9053-z
pubmed: 17487527
Yule S, Flin R, Maran N, Rowley D, Youngson G, Paterson-Brown S (2008) Surgeons’ non-technical skills in the operating room: reliability testing of the NOTSS behavior rating system. World J Surg 32(4):548–556. https://doi.org/10.1007/s00268-007-9320-z
doi: 10.1007/s00268-007-9320-z
pubmed: 18259809
Robertson JM, Dias RD, Yule S, Smink DS (2017) Operating room team training with simulation: a systematic review. J Laparoendosc Adv Surg Tech A 27(5):475–480. https://doi.org/10.1089/lap.2017.0043
doi: 10.1089/lap.2017.0043
pubmed: 28294695
Yule S, Parker SH, Wilkinson J, McKinley A, MacDonald J, Neill A et al (2015) Coaching non-technical skills improves surgical residents’ performance in a simulated operating room. J Surg Educ 72(6):1124–1130. https://doi.org/10.1016/j.jsurg.2015.06.012
doi: 10.1016/j.jsurg.2015.06.012
pubmed: 26610355
Ali MR, Loggins JP, Fuller WD, Miller BE, Hasser CJ, Yellowlees P et al (2008) 3-D telestration: a teaching tool for robotic surgery. J Laparoendosc Adv Surg Tech A 18(1):107–112. https://doi.org/10.1089/lap.2007.0051
doi: 10.1089/lap.2007.0051
pubmed: 18266586
Russ S, Hull L, Rout S, Vincent C, Darzi A, Sevdalis N (2012) Observational teamwork assessment for surgery: feasibility of clinical and nonclinical assessor calibration with short-term training. Ann Surg 255(4):804–809. https://doi.org/10.1097/SLA.0b013e31824a9a02
doi: 10.1097/SLA.0b013e31824a9a02
pubmed: 22367446
Manuguerra A, Mazeaud C, Hubert N, Eschwège P, Roumiguié M, Salleron J et al (2020) Non-technical skills in robotic surgery and impact on near-miss events: a multi-center study. Surg Endosc. https://doi.org/10.1007/s00464-020-07988-5
doi: 10.1007/s00464-020-07988-5
pubmed: 32968920
Manser T (2009) Teamwork and patient safety in dynamic domains of healthcare: a review of the literature. Acta Anaesthesiol Scand 53(2):143–151. https://doi.org/10.1111/j.1399-6576.2008.01717.x
doi: 10.1111/j.1399-6576.2008.01717.x
pubmed: 19032571
Leonard M, Graham S, Bonacum D (2004) The human factor: the critical importance of effective teamwork and communication in providing safe care. Qual Saf Health Care 13(Suppl 1):i85-90. https://doi.org/10.1136/qhc.13.suppl_1.i85
doi: 10.1136/qhc.13.suppl_1.i85
pubmed: 15465961
pmcid: 1765783
Rao PP (2018) Robotic surgery: new robots and finally some real competition! World J Urol 36(4):537–41. https://doi.org/10.1007/s00345-018-2213-y
doi: 10.1007/s00345-018-2213-y
pubmed: 29427003
Brodie A, Vasdev N (2018) The future of robotic surgery. Ann R Coll Surg Engl 100(Suppl 7):4–13. https://doi.org/10.1308/rcsann.supp2.4
doi: 10.1308/rcsann.supp2.4
pubmed: 30179048
pmcid: 6216754
Paradis E, Sutkin G (2017) Beyond a good story: from Hawthorne effect to reactivity in health professions education research. Med Educ 51(1):31–39. https://doi.org/10.1111/medu.13122
doi: 10.1111/medu.13122
pubmed: 27580703
Lynch A (2020) Simulation-based acquisition of non-technical skills to improve patient safety. Semin Pediatr Surg 29(2):150906. https://doi.org/10.1016/j.sempedsurg.2020.150906
doi: 10.1016/j.sempedsurg.2020.150906
pubmed: 32423595
Nicksa GA, Anderson C, Fidler R, Stewart L (2015) Innovative approach using interprofessional simulation to educate surgical residents in technical and nontechnical skills in high-risk clinical scenarios. JAMA Surg 150(3):201–207. https://doi.org/10.1001/jamasurg.2014.2235
doi: 10.1001/jamasurg.2014.2235
pubmed: 25565037
Weaver S, Dy S, Rosen M (2014) Team-training in healthcare: a narrative synthesis of the literature. BMJ Qual Saf 23(5):359–372. https://doi.org/10.1136/bmjqs-2013-001848
doi: 10.1136/bmjqs-2013-001848
pubmed: 24501181
pmcid: 3995248
Koch A, Burns J, Catchpole K, Weigl M (2020) Associations of workflow disruptions in the operating room with surgical outcomes: a systematic review and narrative synthesis. BMJ Qual Saf. https://doi.org/10.1136/bmjqs-2019-010639
doi: 10.1136/bmjqs-2019-010639
pubmed: 32447319
Catchpole KR, Dale TJ, Hirst DG, Smith JP, Giddings TA (2010) A multicenter trial of aviation-style training for surgical teams. J Patient Saf 6(3):180–186. https://doi.org/10.1097/PTS.0b013e3181f100ea
doi: 10.1097/PTS.0b013e3181f100ea
pubmed: 20802280
Stewart-Parker E, Galloway R, Vig S (2017) S-TEAMS: a truly multiprofessional course focusing on nontechnical skills to improve patient safety in the operating theater. J Surg Educ 74(1):137–144. https://doi.org/10.1016/j.jsurg.2016.06.020
doi: 10.1016/j.jsurg.2016.06.020
pubmed: 27663080