Toward widespread use of virtual trials in medical imaging innovation and regulatory science.
computational phantoms
credibility
imaging simulators
in silico trials
informatics
medical imaging simulations
reproducibility
simulations
virtual imaging trials
Journal
Medical physics
ISSN: 2473-4209
Titre abrégé: Med Phys
Pays: United States
ID NLM: 0425746
Informations de publication
Date de publication:
06 Oct 2024
06 Oct 2024
Historique:
revised:
06
09
2024
received:
17
04
2024
accepted:
18
09
2024
medline:
7
10
2024
pubmed:
7
10
2024
entrez:
6
10
2024
Statut:
aheadofprint
Résumé
The rapid advancement in the field of medical imaging presents a challenge in keeping up to date with the necessary objective evaluations and optimizations for safe and effective use in clinical settings. These evaluations are traditionally done using clinical imaging trials, which while effective, pose several limitations including high costs, ethical considerations for repetitive experiments, time constraints, and lack of ground truth. To tackle these issues, virtual trials (aka in silico trials) have emerged as a promising alternative, using computational models of human subjects and imaging devices, and observer models/analysis to carry out experiments. To facilitate the widespread use of virtual trials within the medical imaging research community, a major need is to establish a common consensus framework that all can use. Based on the ongoing efforts of an AAPM Task Group (TG387), this article provides a comprehensive overview of the requirements for establishing virtual imaging trial frameworks, paving the way toward their widespread use within the medical imaging research community. These requirements include credibility, reproducibility, and accessibility. Credibility assessment involves verification, validation, uncertainty quantification, and sensitivity analysis, ensuring the accuracy and realism of computational models. A proper credibility assessment requires a clear context of use and the questions that the study is intended to objectively answer. For reproducibility and accessibility, this article highlights the need for detailed documentation, user-friendly software packages, and standard input/output formats. Challenges in data and software sharing, including proprietary data and inconsistent file formats, are discussed. Recommended solutions to enhance accessibility include containerized environments and data-sharing hubs, along with following standards such as CDISC (Clinical Data Interchange Standards Consortium). By addressing challenges associated with credibility, reproducibility, and accessibility, virtual imaging trials can be positioned as a powerful and inclusive resource, advancing medical imaging innovation and regulatory science.
Types de publication
Journal Article
Review
Langues
eng
Sous-ensembles de citation
IM
Subventions
Organisme : NIH HHS
ID : R01HL155293
Pays : United States
Organisme : NIH HHS
ID : P41EB028744
Pays : United States
Organisme : NIH HHS
ID : R01EB001838
Pays : United States
Organisme : NIH HHS
ID : R01CA259048
Pays : United States
Organisme : Royal Academy of Engineering
ID : INSILEX CiET1919/19
Organisme : ERC Advanced
ID : INSILICO EP/Y030494/1
Informations de copyright
© 2024 American Association of Physicists in Medicine.
Références
Abadi E, Segars WP, Tsui BM, et al. Virtual clinical trials in medical imaging: a review. J Med Imaging. 2020;7(4):042805.
Badano A. In silico imaging clinical trials: cheaper, faster, better, safer, and more scalable. Trials. 2021;22(1):64.
Marshall N, Bosmans H. Performance evaluation of digital breast tomosynthesis systems: comparison of current virtual clinical trial methods. Phys Med Biol. 2022;67(22):22TR04.
Sizikova E, Saharkhiz N, Sharma D, et al. Knowledge‐based in silico models and datasets for the comparative evaluation of mammography AI for a range of breast characteristics, lesion conspicuities, and doses. Oh A., Naumann T., Globerson A., Saenko K., Hardt M., Levine S. eds. Advances in Neural Information Processing Systems Curran Associates, Inc.. 2023;36:37401‐37412. https://proceedings.neurips.cc/paper_files/paper/2023/file/75d0956c9594f47bfb86a07bef58d4b0‐Paper‐Datasets_and_Benchmarks.pdf
Tushar FI, Abadi E, Sotoudeh‐Paima S, et al. Virtual vs. reality: external validation of COVID‐19 classifiers using XCAT phantoms for chest computed tomography. In: Proceedings of SPIE Medical Imaging: Computer‐Aided Diagnosis, SPIE—International Society for Optics and Photonics; 2022:32‐38.
Oberkampf WL, Roy CJ. Verification and validation in scientific computing. Cambridge University Press; 2010.
Steele MJ. Standard for models and simulations. NASA Technical Standards System; 2016.
V&V40 ASME. Assessing Credibility of Computational Modeling through Verification and Validation: Application to Medical Devices. The American Society of Mechanical Engineers; 2018.
Pathmanathan P, Aycock K, Badal A, et al. Credibility assessment of in silico clinical trials for medical devices. PLOS Comput Biol. 2024;20(8):e1012289.
US Food &Drug Administration. Assessing the Credibility of Computational Modeling and Simulation in Medical Device Submissions. US Food &Drug Administration; 2024. Accessed 28 March 2024. https://www.fda.gov/media/154985/download
Sechopoulos I, Ali ES, Badal A, et al. Monte Carlo reference data sets for imaging research: executive summary of the report of AAPM Research Committee Task Group 195. Med Phys. 2015;42(10):5679‐5691.
Soize C. Uncertainty Quantification. Springer; 2017.
Feng SSJ, Sechopoulos I. Clinical digital breast tomosynthesis system: dosimetric characterization. Radiology. 2012;263(1):35‐42.
Barufaldi B, Bakic PR, Pokrajac DD, Lago MA, Maidment AD. Developing populations of software breast phantoms for virtual clinical trials. In: Proceedings of the 14th International Workshop on Breast Imaging (IWBI 2018). SPIE; 2018:481‐489.
Erdemir A, Mulugeta L, Ku JP, et al. Credible practice of modeling and simulation in healthcare: ten rules from a multidisciplinary perspective. J Transl Med. 2020;18(1):369.
Viceconti M, Pappalardo F, Rodriguez B, Horner M, Bischoff J, Musuamba Tshinanu F. In silico trials: verification, validation and uncertainty quantification of predictive models used in the regulatory evaluation of biomedical products. Methods. 2021;185:120‐127.
Badano A, Lago MA, Sizikova E, et al. The stochastic digital human is now enrolling for in silico imaging trials—methods and tools for generating digital cohorts. Prog Biomed Eng. 2023;5(4):042002.
Kim CH, Yeom Y, Petoussi‐Henß N, et al. ICRP publication 145: adult mesh‐type reference computational phantoms. Anna ICRP. 2020;49(3):13‐201.
Plesser HE. Reproducibility vs. replicability: a brief history of a confused terminology. Front Neuroinform. 2018;11:76.
Sharma S, Kapadia A, Ria F, Segars WP, Samei E. Dose coefficients for organ dosimetry in tomosynthesis imaging of adults and pediatrics across diverse protocols. Med Phys. 2022;49(8):5439‐5450.
Herraiz JL, Lopez‐Montes A, Badal A. MCGPU‐PET: an open‐source real‐time Monte Carlo PET simulator. Comput Phys Commun. 2024;296:109008.
Barufaldi B, Higginbotham D, Bakic PR, Maidment AD. OpenVCT: a GPU‐accelerated virtual clinical trial pipeline for mammography and digital breast tomosynthesis. In: Proceeding of SPIE 10573. SPIE; 2018:1333‐1340.
Badal A, Badano A. Accelerating Monte Carlo simulations of photon transport in a voxelized geometry using a massively parallel graphics processing unit. Med Phys. 2009;36(11):4878‐4880.
Abadi E, Harrawood B, Sharma S, Kapadia A, Segars WP, Samei E. DukeSim: a realistic, rapid, and scanner‐specific simulation framework in computed tomography. IEEE Trans Med Imaging. 2018;38(6):1457‐1465.
Sharma D, Graff CG, Badal A, et al. Technical Note: In silico imaging tools from the VICTRE clinical trial. Med Phys. 2019;46(9):3924‐3928.
Segars WP, Tsui BMW, Cai J, Yin F‐F, Fung GS, Samei E. Application of the 4‐D XCAT phantoms in biomedical imaging and beyond. IEEE Trans Med Imaging. 2018;37(3):680‐692.
Lewellen T. The simset program. In: Monte Carlo Calculations in Nuclear Medicine. 2nd ed. CRC Press; 1998:77‐92.
Lee C, Lodwick D, Hurtado J, Pafundi D, Williams JL, Bolch WE. The UF family of reference hybrid phantoms for computational radiation dosimetry. Phys Med Biol. 2010;55(2):339.
Comtat C, Kinahan PE, Defrise M, Michel C, Lartizien C, Townsend D. Simulating whole‐body PET scanning with rapid analytical methods. In: 1999 IEEE Nuclear Science Symposium. Conference Record. 1999 Nuclear Science Symposium and Medical Imaging Conference (Cat. No. 99CH37019). IEEE; 1999:1260‐1264.
Wu M, FitzGerald P, Zhang J, et al. Xcist—an open access x‐ray/CT simulation toolkit. Phys Med Biol. 2022;67(19):194002.
Fung GS, Stierstorfer K, Segars WP, Taguchi K, Flohr TG, Tsui BMW. XCAT/DRASIM: a realistic CT/human‐model simulation package. In: Proceedings of SPIE ‐ The International Society for Optical Engineering. SPIE; 2011:962‐967.
De Man B, Basu S, Chandra N, et al. CATSIM : a new computer assisted tomography SIMulation environment. In: Proceedings of SPIE ‐ The International Society for Optical Engineering. SPIE; 2007:856‐863.
Kavuri A, Fryling D, Felice N, et al. A dockerized CT simulator with a user‐friendly graphical user interface: development and initial demonstration. Med Phys. 2023;50(6):e312.
Souza T, Kush R, Evans JP. Global clinical data interchange standards are here!. Drug Discov Today. 2007;12(3‐4):174‐181.
Data Standards and Terminology Standards for Information Submitted to CDRH. FDA; 2024. Accessed 28 March 2024. https://www.fda.gov/medical‐devices/device‐advice‐comprehensive‐regulatory‐assistance/data‐standards‐and‐terminology‐standards‐information‐submitted‐cdrh
Aydin B, Kiely É, Ohmann C. Feasibility assessment of using CDISC data standards for in silico medical device trials. JSCDM. 2023;4(1).https://doi.org/10.47912/jscdm.230
Becnel LB, Hastak S, Ver Hoef W, et al. BRIDG: a domain information model for translational and clinical protocol‐driven research. J Am Med Inform Assoc. 2017;24(5):882‐890.
AAPM. Task Group No. 387 ‐ Consensus Recommendations on reliable development and use of Virtual Imaging Trials. AAPM; 2024. https://www.aapm.org/org/structure/default.asp?committee_code=TG387
Vent TL, Barufaldi B, Maidment AD. Simulation and experimental validation of high‐resolution test objects for evaluating a next‐generation digital breast tomosynthesis prototype. In: Proceedings of the SPIE. SPIE; 2019:145‐156.
Choi CJ, Vent TL, Geagan MJ, Noël PB, Maidment AD. Investigation of optimal angular range and projection density for next generation tomosynthesis. In Medical Imaging 2022: Physics of Medical Imaging. SPIE; 2022;12031:1112‐1126.
Shankar SS, Felice N, Hoffman EA, et al. Task‐based validation and application of a scanner‐specific CT simulator using an anthropomorphic phantom. Med Phys. 2022;49(12):7447‐7457.
Abadi E, McCabe C, Harrawood B, Sotoudeh‐Paima S, Segars WP, Samei E. Development and clinical applications of a virtual imaging framework for optimizing photon‐counting CT. In Medical Imaging 2022: Physics of Medical Imaging. SPIE, 2022;12031:426‐432.
Abadi E, Harrawood B, Rajagopal JR, et al. Development of a scanner‐specific simulation framework for photon‐counting computed tomography. Biomed Phys Eng Express. 2019;5(5):055008.
Borges LR, Barufaldi B, Caron RF, et al. Technical note: noise models for virtual clinical trials of digital breast tomosynthesis. Med Phys. 2019;46(6):2683‐2689.
Badal A, Sharma D, Graff CG, Zeng R, Badano A. Mammography and breast tomosynthesis simulator for virtual clinical trials. Comput Phys Commun. 2021;261:107779.
Vancoillie L, Marshall N, Cockmartin L, Vignero J, Zhang G, Bosmans H. Verification of the accuracy of a hybrid breast imaging simulation framework for virtual clinical trial applications. J Med Imaging. 2020;7(4):042804‐042804.
de Carvalho PM. Low‐Dose 3d Quantitative Vascular X‐Ray Imaging of The Breast. Université Paris Sud‐Paris XI; 2014.
Elangovan P, Warren LM, Mackenzie A, et al. Development and validation of a modelling framework for simulating 2D‐mammography and breast tomosynthesis images. Phys Med Biol. 2014;59(15):4275‐4293.
Jadick G, Abadi E, Harrawood B, Sharma S, Segars WP, Samei E. A scanner‐specific framework for simulating CT images with tube current modulation. Phys Med Biol. 2021;66(18):185010.
Bliznakova K, Russo P, Mettivier G, et al. A software platform for phase contrast x‐ray breast imaging research. Comput Biol Med. 2015;61:62‐74.
Lorenzana DJ, Solomon J, French RJ, et al. Comparison of simulated low‐dose and conventional‐dose CT for preoperative planning in shoulder arthroplasty. J Bone Joint Surg Am. 2022;104(11):1004‐1014.
Acciavatti RJ, Cohen EA, Maghsoudi OH, et al. Calculation of radiomic features to validate the textural realism of physical anthropomorphic phantoms for digital mammography. In 15th International Workshop on Breast Imaging (IWBI2020). SPIE; 2020;11513:65‐71.
Merken K, Monnens J, Marshall N, et al. Development and validation of a 3D anthropomorphic phantom for dental CBCT imaging research. Med Phys. 2023;50(11):6714‐6736.
Sauer TJ, McCabe C, Abadi E, Samei E, Segars WP. Surface‐based anthropomorphic bone structures for use in high‐resolution simulated medical imaging. Phys Med Biol. 2023;69(1):015023.
Abbey CK, Bakic PR, Pokrajac DD, Maidment AD, Eckstein MP, Boone JM. Evaluation of non‐Gaussian statistical properties in virtual breast phantoms. J Med Imaging. 2019;6(2):025502.
Sarno A, Mettivier G, di Franco F, et al. Dataset of patient‐derived digital breast phantoms for in silico studies in breast computed tomography, digital breast tomosynthesis, and digital mammography. Med Phys. 2021;48(5):2682‐2693.
Baneva Y, Bliznakova K, Cockmartin L, et al. Evaluation of a breast software model for 2D and 3D x‐ray imaging studies of the breast. Phys Med. 2017;41:78‐86.
Marinov S, Carton A‐K, Cockmartin L, et al. Evaluation of the visual realism of breast texture phantoms in digital mammography. In: Proceedings of the 15th International Workshop on Breast Imaging (IWBI 2020). SPIE; 2020:293‐299.
Shaheen E, Van Ongeval C, Zanca F, et al. The simulation of 3D microcalcification clusters in 2D digital mammography and breast tomosynthesis. Med Phys. 2011;38(12):6659‐6671.
Shaheen E, De Keyzer F, Bosmans H, Dance DR, Young KC, Ongeval CV. The simulation of 3D mass models in 2D digital mammography and breast tomosynthesis. Med Phys. 2014;41(8):081913.
Robins M, Solomon J, Koweek LMH, Christensen J, Samei E. Validation of lesion simulations in clinical CT data for anonymized chest and abdominal CT databases. Med Phys. 2019;46(4):1931‐1937.
Sauer TJ, Bejan A, Segars P, Samei E. Development and CT image‐domain validation of a computational lung lesion model for use in virtual imaging trials. Med Phys. 2023;50(7):4366‐4378.
Rodríguez Pérez S, Coolen J, Marshall NW, et al. Methodology to create 3D models of COVID‐19 pathologies for virtual clinical trials. J Med Imaging. 2021;8(Suppl 1):013501.
Abadi E, Jadick G, Lynch DA, Segars WP, Samei E. Emphysema quantifications with CT scan: assessing the effects of acquisition protocols and imaging parameters using virtual imaging trials. Chest. 2023;163(5):1084‐1100.
Elangovan P, Mackenzie A, Dance DR, et al. Design and validation of realistic breast models for use in multiple alternative forced choice virtual clinical trials. Phys Med Biol. 2017;62(7):2778‐2794.
Mettivier G, Bliznakova K, Sechopoulos I, et al. Evaluation of the BreastSimulator software platform for breast tomography. Phys Med Biol. 2017;62(16):6446‐6466.
Rafferty EA, Park JM, Philpotts LE, et al. Assessing radiologist performance using combined digital mammography and breast tomosynthesis compared with digital mammography alone: results of a multicenter, multireader trial. Radiology. 2013;266(1):104‐113.
Barufaldi B, Maidment AD, Dustler M, et al. Virtual clinical trials in medical imaging system evaluation and optimisation. Radiat Prot Dosimetry. 2021;195(3‐4):363‐371.
Badano A, Graff CG, Badal A, et al. Evaluation of digital breast tomosynthesis as replacement of full‐field digital mammography using an in silico imaging trial. JAMA Netw Open. 2018;1(7):e185474.
Harrison RL, Lewellen TK. The simset program. In: Monte Carlo Calculations in Nuclear Medicine: Applications in Diagnostic Imaging. CRC Press; 2012;87.
Poon JK, Dahlbom ML, Casey ME, Qi J, Cherry SR, Badawi RD. Validation of the SimSET simulation package for modeling the Siemens Biograph mCT PET scanner. Phys Med Biol. 2015;60(3):N35‐N45.
Raptis E, Parkes LM, Anton‐Rodriguez J, Carter SF, Herholz K, Matthews JC. Validation of a realistic simulation of the HRRT using SimSET. In: 2017 IEEE Nuclear Science Symposium and Medical Imaging Conference. IEEE; 2017:1‐5.
Poon JK, Dahlbom M, Qi J, Cherry SR, Badawi RD. Validation of SimSET Monte Carlo simulations of the Siemens Biograph mCT PET scanner. In: Nuclear Science Symposium and Medical Imaging Conference (NSS/MIC). IEEE; 2012:2681‐2684.
MacDonald L, Schmitz R, Alessio A, et al. Measured count‐rate performance of the Discovery STE PET/CT scanner in 2D, 3D and partial collimation acquisition modes. Phys Med Biol. 2008;53(14):3723‐3738.
Kemp BJ, Hruska CB, McFarland AR, Lenox MW, Lowe VJ. NEMA NU 2–2007 performance measurements of the Siemens Inveon™ preclinical small animal PET system. Phys Med Biol. 2009;54(8):2359‐2376.
Gonias P, Bertsekas N, Karakatsanis N, et al. Validation of a GATE model for the simulation of the Siemens biograph™ 6 PET scanner. NIM‐A. 2007;571(1‐2):263‐266.
Rezaei H, Sheikhzadeh P, Ghafarian P, Zaidi H, Ay MR. Accurate modeling and performance evaluation of a total‐body pet scanner using Monte Carlo simulations. Med Phys. 2023;50(11):6815‐6827.
Paredes‐Pacheco J, López‐González FJ, Silva‐Rodríguez J, et al. SimPET—an open online platform for the Monte Carlo simulation of realistic brain PET data. Validation for 18F‐FDG scans. Med Phys. 2021;48(5):2482‐2493.
O'Briain TB, Uribe C, Sechopoulos I, Michel C, Bazalova‐Carter M. Publicly available framework for simulating and experimentally validating clinical PET systems. Med Phys. 2023;50(3):1549‐1559.
Elston B, Comtat C, Harrison RL, Kinahan P. ASIM: An Analytic PET Simulator. CRC Press; 2012.
Lai Y, Zhong Y, Chalise A, et al. gPET: a GPU‐based, accurate and efficient Monte Carlo simulation tool for PET. Phys Med Biol. 2019;64(24):245002.
Wangerin KA, Ahn S, Wollenweber S, Ross SG, Kinahan PE, Manjeshwar RM. Evaluation of lesion detectability in positron emission tomography when using a convergent penalized likelihood image reconstruction method. J Med Imaging. 2017;4(1):011002.
Hamdi M, Natsuaki Y, Wangerin KA, et al. Evaluation of attenuation correction in PET/MRI with synthetic lesion insertion. J Med Imaging. 2021;8(5):056001.
Wang K, Teoh E, Jaros J, Treeby BE. Modelling nonlinear ultrasound propagation in absorbing media using the k‐Wave toolbox: experimental validation. In: Ultrasonics Symposium (IUS). IEEE; 2012:523‐526.
Harris GR, Howard SM, Hurrell AM, et al. Hydrophone measurements for biomedical ultrasound applications: a review. IEEE Trans Ultrason Ferroelectr Freq Control. 2023;70(2):85‐100.
Du Y, Jensen H, Jensen JA. Investigation of an angular spectrum approach for pulsed ultrasound fields. Ultrasonics. 2013;53(6):1185‐1191.
Angla C, Larrat B, Jl Gennisson, Chatillon S. Transcranial ultrasound simulations: a review. Med Phys. 2023;50(2):1051‐1072.
Garcia D. SIMUS: an open‐source simulator for medical ultrasound imaging. Part I: theory & examples. Comput Methods Programs Biomed. 2022;218:106726.
Makūnaitė M, Jurkonis R, Lukoševičius A, Baranauskas M. Main uncertainties in the RF ultrasound scanning simulation of the standard ultrasound phantoms. Sensors. 2021;21(13):4420.
Baiu C, Garra BS, Hangiandreou N, et al. AIUM Routine Quality Assurance of Clinical Ultrasound Equipment v2.0. American Institute of Ultrasound in Medicine; 2024. Accessed 28 March 2024. http://aium.s3.amazonaws.com/resourceLibrary/rqa2.pdf
Hoskins PR. Simulation and validation of arterial ultrasound imaging and blood flow. Ultrasound Med Biol. 2008;34(5):693‐717.
Ekroll IK, Saris AE, Avdal J. FLUST: a fast, open source framework for ultrasound blood flow simulations. Comput Methods Programs Biomed. 2023;238:107604.
Wang K, Matthews T, Anis F, Li C, Duric N, Anastasio MA. Waveform inversion with source encoding for breast sound speed reconstruction in ultrasound computed tomography. IEEE Trans Ultrason Ferroelectr Freq Control. 2015;62(3):475‐493.
Trutna CA, Rouze NC, Palmeri ML, Nightingale KR. Measurement of viscoelastic material model parameters using fractional derivative group shear wave speeds in simulation and phantom data. IEEE Trans Ultrason Ferroelectr Freq Control. 2020;67(2):286‐295.
Tehrani AK, Amiri M, Rosado‐Mendez IM, Hall TJ, Rivaz H. Ultrasound scatterer density classification using convolutional neural networks and patch statistics. IEEE Trans Ultrason Ferroelectr Freq Control. 2021;68(8):2697‐2706.
Yeats E, Gupta D, Xu Z, Hall TL. Effects of phase aberration on transabdominal focusing for a large aperture, low f‐number histotripsy transducer. Phys Med Biol. 2022;67(15):155004.
Mast TD, Hinkelman LM, Orr MJ, Sparrow VW, Waag RC. Simulation of ultrasonic pulse propagation through the abdominal wall. J Acoust Soc Am. 1997;102(2 Pt 1):1177‐1190.
Pinton GF, Dahl J, Rosenzweig S, Trahey GE. A heterogeneous nonlinear attenuating full‐wave model of ultrasound. IEEE Trans Ultrason Ferroelectr Freq Control. 2009;56(3):474‐488.
Lartizien C, Kinahan PE, Comtat C. Volumetric model and human observer comparisons of tumor detection for whole‐body positron emission tomography1. Acad Radiol. 2004;11(6):637‐648.
Wangerin KA, Ahn S, Ross SG, Kinahan PE, Manjeshwar RM, Ravindra M. Improving lesion detectability in PET imaging with a penalized likelihood reconstruction algorithm. In: Proceedings of the SPIE. SPIE; 2015:207‐214.
Fan F, Ahn S, De Man B, et al. Deep learning‐based model observers that replicate human observers for PET imaging. In: Image Perception, Observer Performance, and Technology Assessment. SPIE; 2020:53‐58.
Lago MA, Abbey CK, Eckstein MP. Foveated model observers for visual search in 3D medical images. IEEE Trans Med Imaging. 2020;40(3):1021‐1031.