Blending space and time to talk about cancer in extended reality.
Journal
NPJ digital medicine
ISSN: 2398-6352
Titre abrégé: NPJ Digit Med
Pays: England
ID NLM: 101731738
Informations de publication
Date de publication:
29 Sep 2024
29 Sep 2024
Historique:
received:
17
12
2023
accepted:
18
09
2024
medline:
30
9
2024
pubmed:
30
9
2024
entrez:
29
9
2024
Statut:
epublish
Résumé
We introduce a proof-of-concept extended reality (XR) environment for discussing cancer, presenting genomic information from multiple tumour sites in the context of 3D tumour models generated from CT scans. This tool enhances multidisciplinary discussions. Clinicians and cancer researchers explored its use in oncology, sharing perspectives on XR's potential for use in molecular tumour boards, clinician-patient communication, and education. XR serves as a universal language, fostering collaborative decision-making in oncology.
Identifiants
pubmed: 39343807
doi: 10.1038/s41746-024-01262-x
pii: 10.1038/s41746-024-01262-x
doi:
Types de publication
Journal Article
Review
Langues
eng
Pagination
261Subventions
Organisme : Manatu Hauora | Health Research Council of New Zealand (HRC)
ID : 20/765
Organisme : Manatu Hauora | Health Research Council of New Zealand (HRC)
ID : 20/765
Organisme : Manatu Hauora | Health Research Council of New Zealand (HRC)
ID : 20/765
Organisme : Manatu Hauora | Health Research Council of New Zealand (HRC)
ID : 20/765
Organisme : Manatu Hauora | Health Research Council of New Zealand (HRC)
ID : 20/765
Organisme : Manatu Hauora | Health Research Council of New Zealand (HRC)
ID : 20/765
Organisme : Manatu Hauora | Health Research Council of New Zealand (HRC)
ID : 20/765
Organisme : Manatu Hauora | Health Research Council of New Zealand (HRC)
ID : 20/765
Organisme : Manatu Hauora | Health Research Council of New Zealand (HRC)
ID : 20/765
Organisme : Manatu Hauora | Health Research Council of New Zealand (HRC)
ID : 20/765
Organisme : Manatu Hauora | Health Research Council of New Zealand (HRC)
ID : 20/765
Informations de copyright
© 2024. The Author(s).
Références
Sharp, P. A. & Langer, R. Promoting convergence in biomedical science. Science 333, 527–527 (2011).
pubmed: 21798916
doi: 10.1126/science.1205008
Rolfo, C. et al. Multidisciplinary molecular tumour board: a tool to improve clinical practice and selection accrual for clinical trials in patients with cancer. ESMO Open 3, e000398 (2018).
pubmed: 30094075
pmcid: 6069914
doi: 10.1136/esmoopen-2018-000398
Moore, D. A. et al. Prospective analysis of 895 patients on a UK Genomics Review Board. ESMO Open 4, e000469 (2019).
pubmed: 31245058
pmcid: 6557082
doi: 10.1136/esmoopen-2018-000469
Dillekås, H., Rogers, M. S. & Straume, O. Are 90% of deaths from cancer caused by metastases? Cancer Med 8, 5574–5576 (2019).
pubmed: 31397113
pmcid: 6745820
doi: 10.1002/cam4.2474
Perakis, S. O. et al. Comparison of three commercial decision support platforms for matching of next-generation sequencing results with therapies in patients with cancer. ESMO Open 5, e000872 (2020).
pubmed: 32967919
pmcid: 7513637
doi: 10.1136/esmoopen-2020-000872
Lazebnik, Y. Can a biologist fix a radio? Or, what I learned while studying apoptosis. Cancer Cell 2, 179–182 (2002).
pubmed: 12242150
doi: 10.1016/S1535-6108(02)00133-2
Gerlinger, M. et al. Genomic architecture and evolution of clear cell renal cell carcinomas defined by multiregion sequencing. Nat. Genet. 46, 225–233 (2014).
pubmed: 24487277
pmcid: 4636053
doi: 10.1038/ng.2891
Robb, T. J. et al. Complex patterns of genomic heterogeneity identified in 42 tumor samples and ctDNA of a pulmonary atypical carcinoid patient. Cancer Res. Commun. 3, 31–42 (2023).
pubmed: 36968225
pmcid: 10035512
doi: 10.1158/2767-9764.CRC-22-0101
Hood, L. Systems biology and p4 medicine: past, present, and future. Rambam Maimonides Med. J. 4, e0012 (2013).
pubmed: 23908862
pmcid: 3678833
doi: 10.5041/RMMJ.10112
Blenkiron, C. et al. Tailoring a rapid autopsy protocol to explore cancer evolution: a patient collaboration. N. Z. Med. J. 132, 83–92 (2019).
pubmed: 31581185
Robb, T. J., Tse, R. & Blenkiron, C. Reviving the autopsy for modern cancer evolution research. Cancers 13, 409 (2021).
pubmed: 33499137
pmcid: 7866143
doi: 10.3390/cancers13030409
Pires, F., Costa, C. & Dias, P. On the use of virtual reality for medical imaging visualization. J. Digit Imaging 34, 1034–1048 (2021).
pubmed: 34327628
pmcid: 8455774
doi: 10.1007/s10278-021-00480-z
Yuan, J. et al. Extended reality for biomedicine. Nat. Rev. Methods Prim. 3, 14 (2023).
doi: 10.1038/s43586-023-00198-y
Wang, S. et al. Augmented reality as a telemedicine platform for remote procedural training. Sensors 17, 2294 (2017).
pubmed: 28994720
pmcid: 5676722
doi: 10.3390/s17102294
Pratt, P. et al. Through the HoloLens™ looking glass: augmented reality for extremity reconstruction surgery using 3D vascular models with perforating vessels. Eur. Radiol. Exp. 2, 2 (2018).
pubmed: 29708204
pmcid: 5909360
doi: 10.1186/s41747-017-0033-2
Ortiz-Catalan, M., Sander, N., Kristoffersen, M., Håkansson, B. & Brånemark, R. Treatment of phantom limb pain (PLP) based on augmented reality and gaming controlled by myoelectric pattern recognition: a case study of a chronic PLP patient. Front. Neurosci. 8, 71345 (2014).
doi: 10.3389/fnins.2014.00024
Sivananthan, A. et al. A feasibility trial of HoloLens 2™; using mixed reality headsets to deliver remote bedside teaching during COVID-19. Res. Sq. 6, e35674 (2021).
Lau, C. W. et al. Virtual reality for the observation of oncology models (VROOM): immersive analytics for oncology patient cohorts. Sci. Rep. 12, 11337 (2022).
pubmed: 35790803
pmcid: 9256599
doi: 10.1038/s41598-022-15548-1
Pirch, S. et al. The VRNetzer platform enables interactive network analysis in Virtual Reality. Nat. Commun. 12, 2432 (2021).
pubmed: 33893283
pmcid: 8065164
doi: 10.1038/s41467-021-22570-w
Robb, T. J. Tumour evolution in a single patient PhD thesis, University of Auckland, (2021).
Cerami, E. et al. The cBio cancer genomics portal: an open platform for exploring multidimensional cancer genomics data. Cancer Discov. 2, 401–404 (2012).
pubmed: 22588877
doi: 10.1158/2159-8290.CD-12-0095
Goldman, M. J. et al. Visualizing and interpreting cancer genomics data via the Xena platform. Nat. Biotechnol. 38, 675–678 (2020).
pubmed: 32444850
pmcid: 7386072
doi: 10.1038/s41587-020-0546-8
Goldman, M. J. et al. A user guide for the online exploration and visualization of PCAWG data. Nat. Commun. 11, 3400 (2020).
pubmed: 32636365
pmcid: 7340791
doi: 10.1038/s41467-020-16785-6
Qu, Z., Lau, C. W., Nguyen, Q. V., Zhou, Y. & Catchpoole, D. R. Visual analytics of genomic and cancer data: a systematic review. Cancer Inform. 18, 1176935119835546 (2019).
pubmed: 30890859
pmcid: 6416684
doi: 10.1177/1176935119835546
Saadatmand, S., Bretveld, R., Siesling, S. & Tilanus-Linthorst, M. M. A. Influence of tumour stage at breast cancer detection on survival in modern times: population based study in 173 797 patients. BMJ 351, h4901 (2015).
pubmed: 26442924
pmcid: 4595560
doi: 10.1136/bmj.h4901
Erdmann, J. All aboard: will molecular tumor boards help cancer patients? Nat. Med. 21, 655–656 (2015).
pubmed: 26151320
doi: 10.1038/nm0715-655
Kato, S. et al. Real-world data from a molecular tumor board demonstrates improved outcomes with a precision N-of-One strategy. Nat. Commun. 11, 4965 (2020).
pubmed: 33009371
pmcid: 7532150
doi: 10.1038/s41467-020-18613-3
Repetto, M. et al. Molecular tumour board at European Institute of Oncology: report of the first three year activity of an Italian precision oncology experience. Eur. J. Cancer 183, 79–89 (2023).
pubmed: 36801623
doi: 10.1016/j.ejca.2023.01.019
Fittall, M. W. & Van Loo, P. Translating insights into tumor evolution to clinical practice: promises and challenges. Genome Med. 11, 20 (2019).
pubmed: 30925887
pmcid: 6440005
doi: 10.1186/s13073-019-0632-z
Pelanis, E. et al. Use of mixed reality for improved spatial understanding of liver anatomy. Minim. Invasive Ther. Allied Technol. 29, 154–160 (2020).
pubmed: 31116053
doi: 10.1080/13645706.2019.1616558
Pishvaian, M. J. et al. A virtual molecular tumor board to improve efficiency and scalability of delivering precision oncology to physicians and their patients. JAMIA Open 2, 505–515 (2019).
pubmed: 32025647
pmcid: 6994017
doi: 10.1093/jamiaopen/ooz045
Russo, A. et al. The challenge of the Molecular Tumor Board empowerment in clinical oncology practice: a position paper on behalf of the AIOM- SIAPEC/IAP-SIBioC-SIC-SIF-SIGU-SIRM Italian Scientific Societies. Crit. Rev. Oncol. Hematol. 169, 103567 (2022).
pubmed: 34896250
doi: 10.1016/j.critrevonc.2021.103567
Dodds, H. E. Immersive learning environments: designing XR into higher education.A Practitioner’s. Guide Instr. Des. High. Educ. 1, 69–79 (2021).
doi: 10.59668/164.4220
Zweifach, S. M. & Triola, M. M. Extended reality in medical education: driving adoption through provider-centered design. Digital Biomark. 3, 14–21 (2019).
doi: 10.1159/000498923
Lee, Y. & Yoo, B. XR collaboration beyond virtual reality: work in the real world. J. Comput. Des. Eng. 8, 756–772 (2021).
Khader, J. Improving cancer outcomes through international collaboration in developing countries: King Hussein Cancer Center as a unique experience. J. Glob. Oncol. 4, 161s–161s (2018).
doi: 10.1200/jgo.18.43300
Carlin, L. E., Smith, H. E. & Henwood, F. To see or not to see: a qualitative interview study of patients’ views on their own diagnostic images. BMJ open 4, e004999 (2014).
pubmed: 25082418
pmcid: 4120403
doi: 10.1136/bmjopen-2014-004999
Phelps, E. E., Wellings, R., Griffiths, F., Hutchinson, C. & Kunar, M. Do medical images aid understanding and recall of medical information? An experimental study comparing the experience of viewing no image, a 2D medical image and a 3D medical image alongside a diagnosis. Patient Educ. Couns. 100, 1120–1127 (2017).
pubmed: 28081938
doi: 10.1016/j.pec.2016.12.034
Mulsow, J. J. W., Feeley, T. M. & Tierney, S. Beyond consent—improving understanding in surgical patients. Am. J. Surg. 203, 112–120 (2012).
pubmed: 21641573
doi: 10.1016/j.amjsurg.2010.12.010
Falagas, M. E., Korbila, I. P., Giannopoulou, K. P., Kondilis, B. K. & Peppas, G. Informed consent: how much and what do patients understand? Am. J. Surg. 198, 420–435 (2009).
pubmed: 19716887
doi: 10.1016/j.amjsurg.2009.02.010
Perin, A. et al. Informed consent through 3D virtual reality: a randomized clinical trial. Acta Neurochir. 163, 301–308 (2021).
pubmed: 32242272
doi: 10.1007/s00701-020-04303-y
Miller, S. M. Monitoring versus blunting styles of coping with cancer influence the information patients want and need about their disease. Implications for cancer screening and management. Cancer 76, 167–177 (1995).
pubmed: 8625088
doi: 10.1002/1097-0142(19950715)76:2<167::AID-CNCR2820760203>3.0.CO;2-K
Stepan, K. et al. Immersive virtual reality as a teaching tool for neuroanatomy. Int. Forum Allergy Rhinol. 7, 1006–1013 (2017).
pubmed: 28719062
doi: 10.1002/alr.21986
Makransky, G., Terkildsen, T. S. & Mayer, R. E. Adding immersive virtual reality to a science lab simulation causes more presence but less learning. Learn. Instr. 60, 225–236 (2019).
doi: 10.1016/j.learninstruc.2017.12.007
Robertson, S. P. et al. Genomic medicine must reduce, not compound, health inequities: the case for hauora-enhancing genomic resources for New Zealand. N. Z. Med. J. 131, 81–89 (2018).
pubmed: 30116069
Madhusoodanan, J. Health-care inequality could deepen with precision oncology. Nature 585, S13–S13 (2020).
doi: 10.1038/d41586-020-02678-7
Henare, K. L. et al. Mapping a route to Indigenous engagement in cancer genomic research. Lancet Oncol. 20, e327–e335 (2019).
pubmed: 31162106
doi: 10.1016/S1470-2045(19)30307-9
Dal Mas, F., Massaro, M., Rippa, P. & Secundo, G. The challenges of digital transformation in healthcare: An interdisciplinary literature review, framework, and future research agenda. Technovation 123, 102716 (2023).
doi: 10.1016/j.technovation.2023.102716
Ting, D. S. W., Carin, L., Dzau, V. & Wong, T. Y. Digital technology and COVID-19. Nat. Med. 26, 459–461 (2020).
pubmed: 32284618
pmcid: 7100489
doi: 10.1038/s41591-020-0824-5
NHS England & NHS Improvement. Science in healthcare: Delivering the NHS long term plan. The Chief Scientific Officer’s strategy, https://www.england.nhs.uk/wp-content/uploads/2020/03/science-in-healthcare-delivering-the-nhs-long-term-plan.pdf (2020).
Vlčková, J. & Klimková, V. The digital transformation of Czech healthcare: trends and COVID-19 impact. Int. J. Electron. Healthc. 13, 15–32 (2023).
doi: 10.1504/IJEH.2023.128604
Nan, C., Pradosh, K. & Viktor, G. Augmented reality with Microsoft HoloLens holograms for near infrared fluorescence based image guided surgery. Proc.SPIE 10049, https://doi.org/10.1117/12.2251625, (2017).
Poothicottu Jacob, G. A HoloLens Framework for Augmented Reality Applications in Breast Cancer Surgery Master of Applied Science thesis, University of Waterloo, (2018).
Zafar, S. & Zachar, J. J. Evaluation of HoloHuman augmented reality application as a novel educational tool in dentistry. Eur. J. Dent. Educ. 24, 259–265 (2020).
pubmed: 31899840
doi: 10.1111/eje.12492
Hillary, L. et al. HoloLens in suturing training. Proc.SPIE 10576, https://doi.org/10.1117/12.2293934, (2018).
Luck, J., Gosling, N. & Saour, S. Undergraduate surgical education during COVID-19: could augmented reality provide a solution? Br. J. Surg. 108, e129–e130 (2021).
pubmed: 33793706
doi: 10.1093/bjs/znaa159
Tu, P. et al. Augmented reality based navigation for distal interlocking of intramedullary nails utilizing Microsoft HoloLens 2. Comput. Biol. Med. 133, 104402 (2021).
pubmed: 33895460
doi: 10.1016/j.compbiomed.2021.104402
Liimatainen, K., Latonen, L., Valkonen, M., Kartasalo, K. & Ruusuvuori, P. Virtual reality for 3D histology: multi-scale visualization of organs with interactive feature exploration. BMC Cancer 21, 1133 (2021).
pubmed: 34686173
pmcid: 8539837
doi: 10.1186/s12885-021-08542-9