Impact of Different Mammography Systems on Artificial Intelligence Performance in Breast Cancer Screening.

Artificial intelligence (AI) tools may assist breast screening mammography programs, but limited evidence supports their generalizability to new settings. This retrospective study used a 3-year dataset (April 1, 2016-March 31, 2019) from a U.K. regional screening program. The performance of a commercially available breast screening AI algorithm was assessed with a prespecified and site-specific decision threshold to evaluate whether its performance was transferable to a new clinical site. The dataset consisted of women (aged approximately 50-70 years) who attended routine screening, excluding self-referrals, those with complex physical requirements, those who had undergone a previous mastectomy, and those who underwent screening that had technical recalls or did not have the four standard image views. In total, 55 916 screening attendees (mean age, 60 years ± 6 [SD]) met the inclusion criteria. The prespecified threshold resulted in high recall rates (48.3%, 21 929 of 45 444), which reduced to 13.0% (5896 of 45 444) following threshold calibration, closer to the observed service level (5.0%, 2774 of 55 916). Recall rates also increased approximately threefold following a software upgrade on the mammography equipment, requiring per-software version thresholds. Using software-specific thresholds, the AI algorithm would have recalled 277 of 303 (91.4%) screen-detected cancers and 47 of 138 (34.1%) interval cancers. AI performance and thresholds should be validated for new clinical settings before deployment, while quality assurance systems should monitor AI performance for consistency.

Informations de copyright

© 2023 by the Radiological Society of North America, Inc.

Déclaration de conflit d'intérêts

Disclosures of conflicts of interest: C.F.d.V. No relevant relationships. S.J.C. No relevant relationships. R.T.S. No relevant relationships. J.A.D. No relevant relationships. J.Y. Employed by Kheiron Medical Technologies; support for attending meetings/travel from Kheiron Medical Technologies; patents planned, issued, or pending with Kheiron Medical Technologies; stock or stock options in Kheiron Medical Technologies. D.D. Full-time employee of Kheiron Medical Technologies, supplier of the medical device evaluated in this project; grant from Innovate UK via iCAIRD, the industrial center for AI research in digital diagnostics, all parties received grant monies for the work done; associate member of the Faculty for Clinical Informatics and a health executive in residence for the UCL Global Business School for Health (unpaid, volunteer role); stock or stock options in Kheiron Medical Technologies (employee share options benefit scheme). M.B. iCAIRD funded by Innovate UK, under the UK Research and Innovation (UKRI) Industrial Strategy Challenge Fund “From Data to Early Diagnosis in Precision Medicine” challenge. D.J.H. Receipt of research award (chief investigator) from Innovate UK/UKRI, this funding underpinned the research infrastructure and some staff time. L.A.A. Funding from Innovate UK. G.L. No relevant relationships.