Development and validation of an objective virtual reality tool for assessing technical aptitude among potential candidates for surgical training.
Aptitude
Assessment
Selection
Surgical training
Technical skills
Validation
Journal
BMC medical education
ISSN: 1472-6920
Titre abrégé: BMC Med Educ
Pays: England
ID NLM: 101088679
Informations de publication
Date de publication:
14 Mar 2024
14 Mar 2024
Historique:
received:
17
06
2023
accepted:
26
02
2024
medline:
18
3
2024
pubmed:
15
3
2024
entrez:
15
3
2024
Statut:
epublish
Résumé
Good technical skills are crucial for surgeons. Yet although surgical training programs strive to assess technical aptitude when selecting surgical residents, valid assessments of such aptitude are still lacking. Surgical simulators have been proposed as a potentially effective tool for this purpose. The current study aims to develop a technical aptitude test using a virtual reality surgical simulator, and to validate its use for the selection of surgical residents. The study had three phases. In Phase 1, we developed an initial version of the technical aptitude test using the Lap-X-VR laparoscopic simulator. In Phases 2 and 3 we refined the test and collected empirical data to evaluate four main sources of validity evidence (content, response process, internal structure, and relationships with other variables), and to evaluate the feasibility and acceptability of the test. Specifically, Phase 2 comprised a review of the test by 30 senior surgeons, and in Phase 3 a revised version of the test was administered to 152 interns to determine its psychometric properties. Both the surgeons and interns rated the test as highly relevant for selecting surgical residents. Analyses of the data obtained from the trial administration of the test supported the appropriateness of the score calculation process and showed good psychometric properties, including reliability (α = 0.83) and task discrimination (mean discrimination = 0.5, SD = 0.1). The correlations between test scores and background variables revealed significant correlations with gender, surgical simulator experience, and video game experience (ps < 0.001). These variables, however, explained together only 10% of the variance in test scores. We describe the systematic development of an innovative virtual reality test for assessing technical aptitude in candidates for surgical training, and present evidence for its validity, feasibility and acceptability. Further validation is required to support the application of the test for selection, as well as to discern the impact of gender, surgical simulator experience, and video game experience on the fairness of test results. However, the test appears to be a promising tool that may help training programs assess the suitability of candidates for surgical training.
Sections du résumé
BACKGROUND
BACKGROUND
Good technical skills are crucial for surgeons. Yet although surgical training programs strive to assess technical aptitude when selecting surgical residents, valid assessments of such aptitude are still lacking. Surgical simulators have been proposed as a potentially effective tool for this purpose. The current study aims to develop a technical aptitude test using a virtual reality surgical simulator, and to validate its use for the selection of surgical residents.
METHODS
METHODS
The study had three phases. In Phase 1, we developed an initial version of the technical aptitude test using the Lap-X-VR laparoscopic simulator. In Phases 2 and 3 we refined the test and collected empirical data to evaluate four main sources of validity evidence (content, response process, internal structure, and relationships with other variables), and to evaluate the feasibility and acceptability of the test. Specifically, Phase 2 comprised a review of the test by 30 senior surgeons, and in Phase 3 a revised version of the test was administered to 152 interns to determine its psychometric properties.
RESULTS
RESULTS
Both the surgeons and interns rated the test as highly relevant for selecting surgical residents. Analyses of the data obtained from the trial administration of the test supported the appropriateness of the score calculation process and showed good psychometric properties, including reliability (α = 0.83) and task discrimination (mean discrimination = 0.5, SD = 0.1). The correlations between test scores and background variables revealed significant correlations with gender, surgical simulator experience, and video game experience (ps < 0.001). These variables, however, explained together only 10% of the variance in test scores.
CONCLUSIONS
CONCLUSIONS
We describe the systematic development of an innovative virtual reality test for assessing technical aptitude in candidates for surgical training, and present evidence for its validity, feasibility and acceptability. Further validation is required to support the application of the test for selection, as well as to discern the impact of gender, surgical simulator experience, and video game experience on the fairness of test results. However, the test appears to be a promising tool that may help training programs assess the suitability of candidates for surgical training.
Identifiants
pubmed: 38486166
doi: 10.1186/s12909-024-05228-1
pii: 10.1186/s12909-024-05228-1
pmc: PMC10941473
doi:
Types de publication
Review
Journal Article
Langues
eng
Sous-ensembles de citation
IM
Pagination
286Subventions
Organisme : Israel Science Foundation
ID : 1830/20
Informations de copyright
© 2024. The Author(s).
Références
Am J Surg. 2010 Jan;199(1):99-104
pubmed: 20103073
Scand J Surg. 2013;102(3):141-4
pubmed: 23963026
Psychol Sci. 2007 Oct;18(10):850-5
pubmed: 17894600
Minim Invasive Ther Allied Technol. 2014 Aug;23(4):190-7
pubmed: 24794855
IEEE Trans Haptics. 2011 Jan-Feb;4(1):51-66
pubmed: 26962955
ANZ J Surg. 2008 Apr;78(4):291-6
pubmed: 18366403
J Grad Med Educ. 2015 Jun;7(2):234-7
pubmed: 26221441
Surg Endosc. 2015 Nov;29(11):3065-73
pubmed: 25631116
Surgery. 2020 Feb;167(2):321-327
pubmed: 31753325
Surg Endosc. 2003 Jul;17(7):1082-5
pubmed: 12728373
J Laparoendosc Adv Surg Tech A. 2011 May;21(4):329-33
pubmed: 21563940
J Bone Joint Surg Am. 2011 Oct 5;93(19):e115(1-9)
pubmed: 22005876
Minim Invasive Ther Allied Technol. 2014 Oct;23(5):287-93
pubmed: 24773373
Surg Endosc. 2004 Oct;18(10):1514-8
pubmed: 15791380
J Grad Med Educ. 2015 Dec;7(4):539-48
pubmed: 26692964
Int J Surg. 2016 May;29:85-94
pubmed: 26992652
Surg Endosc. 2009 Mar;23(3):645-9
pubmed: 18622548
Surg Endosc. 2019 Jul;33(7):2093-2103
pubmed: 30327918
Child Dev. 2010 Sep-Oct;81(5):1417-30
pubmed: 20840231
Child Dev. 1985 Dec;56(6):1479-98
pubmed: 4075870
Int J Surg. 2013;11(9):773-8
pubmed: 23838344
ANZ J Surg. 2009 Mar;79(3):108-13
pubmed: 19317772
World J Surg. 2008 Sep;32(9):1917-21
pubmed: 18553192
J Surg Res. 2009 Oct;156(2):312-6
pubmed: 19631336
Am J Surg. 2014 Feb;207(2):263-70
pubmed: 24468026
Eur Urol. 2004 Mar;45(3):346-51; author reply 351
pubmed: 15036681
J Am Coll Surg. 2013 Mar;216(3):461-71
pubmed: 23266420
Surg Endosc. 2007 Aug;21(8):1413-7
pubmed: 17294307
J Surg Res. 2011 Oct;170(2):189-94
pubmed: 21612796
J Surg Educ. 2023 Jan;80(1):81-92
pubmed: 36175291
Surg Endosc. 2013 Jan;27(1):118-26
pubmed: 22773236
J Surg Educ. 2012 May-Jun;69(3):360-3
pubmed: 22483139
Surg Endosc. 2008 Jun;22(6):1440-4
pubmed: 17972134
N Engl J Med. 2013 Oct 10;369(15):1434-42
pubmed: 24106936
J Surg Educ. 2017 Nov - Dec;74(6):992-1000
pubmed: 28412270
World J Surg. 2014 Feb;38(2):296-304
pubmed: 24146198
Updates Surg. 2018 Sep;70(3):401-405
pubmed: 29987767
J Surg Educ. 2018 Nov;75(6):1650-1657
pubmed: 29650484
Ann Surg. 2008 Mar;247(3):540-3
pubmed: 18376201
Int J Med Educ. 2020 May 18;11:97-106
pubmed: 32425176
Ann Surg. 2016 Apr;263(4):673-91
pubmed: 26079898
Surg Endosc. 2023 Nov;37(11):8804-8809
pubmed: 37603102
Surg Endosc. 2007 Nov;21(11):1991-5
pubmed: 17593434
Med Educ. 2016 Apr;50(4):409-27
pubmed: 26995481
Mem Cognit. 1998 May;26(3):444-8
pubmed: 9610116
Am J Surg. 2009 Apr;197(4):447-9
pubmed: 19217604
J Surg Res. 2013 Dec;185(2):561-9
pubmed: 23910887
J Am Coll Surg. 2006 May;202(5):779-87
pubmed: 16648018
J Surg Educ. 2011 Jan-Feb;68(1):67-72
pubmed: 21292219
Curr Probl Surg. 2002 Jun;39(6):573-659
pubmed: 12037512
J Surg Educ. 2010 Jan-Feb;67(1):37-43
pubmed: 20421089
J Plast Reconstr Aesthet Surg. 2014 Sep;67(9):1303-4
pubmed: 24880578
Intern Med. 2017;56(12):1507-1513
pubmed: 28626175
J Surg Educ. 2010 May-Jun;67(3):184-9
pubmed: 20630431
Scand J Surg. 2009;98(4):209-13
pubmed: 20218416
Brain Cogn. 2009 Apr;69(3):451-9
pubmed: 18980790
ANZ J Surg. 2016 Dec;86(12):983-989
pubmed: 25645288
Br J Surg. 2001 Nov;88(11):1525-32
pubmed: 11683753
Surg Endosc. 2014 Jun;28(6):1830-7
pubmed: 24442679
Surg Endosc. 2009 Mar;23(3):641-4
pubmed: 18813975
BMC Med Educ. 2017 May 2;17(1):75
pubmed: 28464857
J Surg Educ. 2016 Jul-Aug;73(4):721-9
pubmed: 27133583
Surg Endosc. 2018 Oct;32(10):4087-4095
pubmed: 29541863
Curr Urol Rep. 2006 Mar;7(2):120-4
pubmed: 16526996
Am J Surg. 2001 Aug;182(2):110-6
pubmed: 11574079
Adv Health Sci Educ Theory Pract. 2014 May;19(2):203-17
pubmed: 23568181
J Am Coll Surg. 2016 Sep;223(3):530-6
pubmed: 27321389
Int Surg. 2015 Feb;100(2):350-7
pubmed: 25692441
Am J Surg. 2012 Mar;203(3):397-400; discussion 400
pubmed: 22364906
J Endourol. 2012 Nov;26(11):1506-11
pubmed: 22642549
JAMA Surg. 2023 Nov 1;158(11):1168-1175
pubmed: 37647076
Am J Surg. 2009 Nov;198(5):726-32
pubmed: 19887202
Med Educ. 2013 Aug;47(8):790-800
pubmed: 23837425
ANZ J Surg. 2008 Apr;78(4):282-90
pubmed: 18366402
J Minim Access Surg. 2017 Jul-Sep;13(3):165-169
pubmed: 27143695
Am J Surg. 2016 Aug;212(2):354-60
pubmed: 27018078
Am J Surg. 2007 Jun;193(6):797-804
pubmed: 17512301
Br J Surg. 2012 Dec;99(12):1610-21
pubmed: 23034658
Surg Endosc. 2008 Jan;22(1):31-7
pubmed: 17965919
Surg Endosc. 2017 Sep;31(9):3718-3727
pubmed: 28451813