Mammography self-evaluation online test for screening readers: an Italian Society of Medical Radiology (SIRM) initiative.
Breast neoplasms
Diagnostic self-evaluation
Mammography
Mass screening
Journal
European radiology
ISSN: 1432-1084
Titre abrégé: Eur Radiol
Pays: Germany
ID NLM: 9114774
Informations de publication
Date de publication:
Mar 2022
Mar 2022
Historique:
received:
12
03
2021
accepted:
23
07
2021
revised:
06
07
2021
pubmed:
5
9
2021
medline:
15
2
2022
entrez:
4
9
2021
Statut:
ppublish
Résumé
To report and analyse the characteristics and performance of the first cohort of Italian radiologists completing the national mammography self-evaluation online test established by the Italian Society of Medical Radiology (SIRM). A specifically-built dataset of 132 mammograms (24 with screen-detected cancers and 108 negative cases) was preliminarily tested on 48 radiologists to define pass thresholds (62% sensitivity and 86% specificity) and subsequently made available online to SIRM members during a 13-month timeframe between 2018 and 2019. Associations between participants' characteristics, pass rates, and diagnostic accuracy were then investigated with descriptive statistics and univariate and multivariable regression analyses. A total of 342 radiologists completed the test, 151/342 (44.2%) with success. All individual variables, except gender, showed a significant correlation with pass rates and diagnostic sensitivity, confirmed by univariate logistic regression, while only involvement in organised screening programs and number of mammograms read per year showed a positive association with specificity at univariate logistic regression. In the multivariable regression analysis, fewer variables remained significant: > 3000 mammograms read per year for success rate; female gender, public practice setting, and higher experience self-judgement for sensitivity; no variables were significantly associated with specificity. This national self-evaluation test effectively differentiated multiple aspects of mammographic reading experience, but specific breast imaging experience was shown not to strictly guarantee good diagnostic accuracy. Due to its easy use and the validity of obtained results, this test could be extended to all Italian breast radiologists, regardless of their experience, also as a Breast Unit accreditation criterion. • This self-evaluation test was found to be able to differentiate various degrees of mammographic interpretation experience. • Breast cancer screening readers should undergo a self-assessment test, since experience parameters alone do not guarantee diagnostic ability.
Identifiants
pubmed: 34480624
doi: 10.1007/s00330-021-08241-w
pii: 10.1007/s00330-021-08241-w
doi:
Types de publication
Journal Article
Langues
eng
Sous-ensembles de citation
IM
Pagination
1624-1633Informations de copyright
© 2021. European Society of Radiology.
Références
Perry N, Broeders M, de Wolf C, Tornberg S, Holland R, von Karsa L (2007) European guidelines for quality assurance in breast cancer screening and diagnosis. Fourth edition–summary document. Ann Oncol 19:614–622. https://doi.org/10.1093/annonc/mdm481
pubmed: 18024988
doi: 10.1093/annonc/mdm481
Giordano L, Giorgi D, Frigerio A et al (2006) Process indicators and standards for the evaluation of breast cancer screening programmes. Epidemiol Prev 30(5–9):11–47
Wilson ARM, Marotti L, Bianchi S et al (2013) The requirements of a specialist Breast Centre. Eur J Cancer 49:3579–3587. https://doi.org/10.1016/j.ejca.2013.07.017
pubmed: 23968730
doi: 10.1016/j.ejca.2013.07.017
Torre LA, Islami F, Siegel RL, Ward EM, Jemal A (2017) Global cancer in women: burden and trends. Cancer Epidemiol Biomarkers Prev 26:444–457. https://doi.org/10.1158/1055-9965.EPI-16-0858
pubmed: 28223433
doi: 10.1158/1055-9965.EPI-16-0858
AIRTUM Working Group (2019) I numeri del cancro in Italia. Associazione Italiana Registri Tumori, Milano. Available via https://www.aiom.it/wp-content/uploads/2019/09/2019_Numeri_Cancro-operatori-web.pdf . Accessed 04 July 2021
Lauby-Secretan B, Scoccianti C, Loomis D et al (2015) Breast-Cancer Screening — Viewpoint of the IARC Working Group. N Engl J Med 372:2353–2358. https://doi.org/10.1056/NEJMsr1504363
pubmed: 26039523
doi: 10.1056/NEJMsr1504363
Trimboli RM, Giorgi Rossi P, Battisti NML et al (2020) Do we still need breast cancer screening in the era of targeted therapies and precision medicine? Insights Imaging 11:105. https://doi.org/10.1186/s13244-020-00905-3
pubmed: 32975658
pmcid: 7519022
doi: 10.1186/s13244-020-00905-3
Muratov S, Canelo-Aybar C, Tarride J-E et al (2020) Monitoring and evaluation of breast cancer screening programmes: selecting candidate performance indicators. BMC Cancer 20:795. https://doi.org/10.1186/s12885-020-07289-z
pubmed: 32831048
pmcid: 7444070
doi: 10.1186/s12885-020-07289-z
Giordano L, Castagno R, Giorgi D et al (2015) Breast cancer screening in Italy: evaluating key performance indicators for time trends and activity volumes. Epidemiol Prev 39:30–39
pubmed: 26405774
Giorgi D, Giordano L, Ventura L, Frigerio A, Paci E, Zappa M (2012) Mammography breast cancer screening in Italy: 2010 survey. Epidemiol Prev 36:8–27
pubmed: 23293268
Osservatorio Nazionale Screening (2019) Rapporto 2019. Osservatorio Nazionale Screening, Firenze. Available via https://www.osservatorionazionalescreening.it/content/rapporto . Accessed 04 July 2021
Hofvind S, Bennett R, Brisson J et al (2016) Audit feedback on reading performance of screening mammograms: an international comparison. J Med Screen 23:150–159. https://doi.org/10.1177/0969141315610790
pubmed: 26892191
doi: 10.1177/0969141315610790
Rutter CM, Taplin S (2000) Assessing mammographers’ accuracy. J Clin Epidemiol 53:443–450. https://doi.org/10.1016/S0895-4356(99)00218-8
pubmed: 10812315
doi: 10.1016/S0895-4356(99)00218-8
Qenam BA, Li T, Tapia K, Brennan PC (2020) The roles of clinical audit and test sets in promoting the quality of breast screening: a scoping review. Clin Radiol 75:794.e1-794.e6. https://doi.org/10.1016/j.crad.2020.01.015
doi: 10.1016/j.crad.2020.01.015
Soh BP, Lee W, Kench PL et al (2012) Assessing reader performance in radiology, an imperfect science: lessons from breast screening. Clin Radiol 67:623–628. https://doi.org/10.1016/j.crad.2012.02.007
pubmed: 22486992
doi: 10.1016/j.crad.2012.02.007
Gale AG (2003) Performs: a self-assessment scheme for radiologists in breast screening. Semin Breast Dis 6:148–152. https://doi.org/10.1053/j.sembd.2004.03.006
doi: 10.1053/j.sembd.2004.03.006
Brennan PC, Trieu PD, Tapia K, Ryan J, Mello-Thoms C, Lee W (2014) BREAST: a novel strategy to improve the detection of breast cancer. In: Fujita H, Hara T, Muramatsu C (eds) Breast imaging. IWDM 2014. Lecture Notes in Computer Science. Springer, Cham.
Trieu PDY, Tapia K, Frazer H, Lee W, Brennan P (2019) Improvement of cancer detection on mammograms via BREAST test sets. Acad Radiol 26:e341–e347. https://doi.org/10.1016/j.acra.2018.12.017
pubmed: 30826148
doi: 10.1016/j.acra.2018.12.017
Scott HJ, Gale AG (2006) Breast screening: PERFORMS identifies key mammographic training needs. Br J Radiol 79:S127–S133. https://doi.org/10.1259/bjr/25049149
pubmed: 17209118
doi: 10.1259/bjr/25049149
Timmers JMH, Verbeek ALM, Pijnappel RM, Broeders MJM, den Heeten GJ (2014) Experiences with a self-test for Dutch breast screening radiologists: lessons learnt. Eur Radiol 24:294–304. https://doi.org/10.1007/s00330-013-3018-4
pubmed: 24057201
doi: 10.1007/s00330-013-3018-4
Soh BP, Lee WB, McEntee MF et al (2014) Mammography test sets: reading location and prior images do not affect group performance. Clin Radiol 69:397–402. https://doi.org/10.1016/j.crad.2013.11.010
pubmed: 24418670
doi: 10.1016/j.crad.2013.11.010
Suleiman WI, Rawashdeh MA, Lewis SJ et al (2016) Impact of breast reader assessment strategy on mammographic radiologists’ test reading performance. J Med Imaging Radiat Oncol 60:352–358. https://doi.org/10.1111/1754-9485.12461
pubmed: 27062490
doi: 10.1111/1754-9485.12461
Scott HJ, Evans A, Gale AG, Murphy A, Reed J (2009) The relationship between real life breast screening and an annual self assessment scheme. In: Sahiner B, Manning DJ (eds) Medical Imaging 2009: Image Perception, Observer Performance, and Technology Assessment. SPIE, Bellingham, WA, USA
Ekpo EU, Alakhras M, Brennan P (2018) Errors in mammography cannot be solved through technology alone. Asian Pacific J Cancer Prev 19:291–301. https://doi.org/10.22034/APJCP.2018.19.2.291
doi: 10.22034/APJCP.2018.19.2.291
Molins E, Macià F, Ferrer F, Maristany M-T, Castells X (2008) Association between radiologists’ experience and accuracy in interpreting screening mammograms. BMC Health Serv Res 8:91. https://doi.org/10.1186/1472-6963-8-91
pubmed: 18439248
pmcid: 2375876
doi: 10.1186/1472-6963-8-91
Miglioretti DL, Ichikawa L, Smith RA et al (2017) Correlation between screening mammography interpretive performance on a test set and performance in clinical practice. Acad Radiol 24:1256–1264. https://doi.org/10.1016/j.acra.2017.03.016
pubmed: 28551400
pmcid: 5591765
doi: 10.1016/j.acra.2017.03.016
Soh BP, Lee W, McEntee MF et al (2013) Screening mammography: test set data can reasonably describe actual clinical reporting. Radiology 268:46–53. https://doi.org/10.1148/radiol.13122399
pubmed: 23481165
pmcid: 3689446
doi: 10.1148/radiol.13122399
Nodine CF, Kundel HL, Mello-Thoms C et al (1999) How experience and training influence mammography expertise. Acad Radiol 6:575–585. https://doi.org/10.1016/S1076-6332(99)80252-9
pubmed: 10516859
doi: 10.1016/S1076-6332(99)80252-9
Beam CA, Conant EF, Sickles EA (2003) Association of volume and volume-independent factors with accuracy in screening mammogram interpretation. J Natl Cancer Inst 95:282–290. https://doi.org/10.1093/jnci/95.4.282
pubmed: 12591984
doi: 10.1093/jnci/95.4.282
Ciatto S, Ambrogetti D, Morrone D, Del Turco MR (2006) Analysis of the results of a proficiency test in screening mammography at the CSPO of Florence: review of 705 tests. Radiol Med 111:797–803. https://doi.org/10.1007/s11547-006-0073-9
pubmed: 16896561
doi: 10.1007/s11547-006-0073-9
Kim SH, Lee EH, Jun JK et al (2019) Interpretive performance and inter-observer agreement on digital mammography test sets. Korean J Radiol 20:218. https://doi.org/10.3348/kjr.2018.0193
pubmed: 30672161
doi: 10.3348/kjr.2018.0193
Suleiman WI, Lewis SJ, Georgian-Smith D, Evanoff MG, McEntee MF (2014) Number of mammography cases read per year is a strong predictor of sensitivity. J Med Imaging 1:015503. https://doi.org/10.1117/1.JMI.1.1.015503
doi: 10.1117/1.JMI.1.1.015503
Elmore JG, Wells CK, Howard DH (1998) Does diagnostic accuracy in mammography depend on radiologists’ experience? J Women’s Heal 7:443–449. https://doi.org/10.1089/jwh.1998.7.443
doi: 10.1089/jwh.1998.7.443
Esserman L, Cowley H, Eberle C et al (2002) Improving the accuracy of mammography: volume and outcome relationships. J Natl Cancer Inst 94:369–375. https://doi.org/10.1093/jnci/94.5.369
pubmed: 11880475
doi: 10.1093/jnci/94.5.369
Rawashdeh MA, Lee WB, Bourne RM et al (2013) Markers of good performance in mammography depend on number of annual readings. Radiology 269:61–67. https://doi.org/10.1148/radiol.13122581
pubmed: 23737538
doi: 10.1148/radiol.13122581
Reed WM, Lee WB, Cawson JN, Brennan PC (2010) Malignancy detection in digital mammograms. Acad Radiol 17:1409–1413. https://doi.org/10.1016/j.acra.2010.06.016
pubmed: 20719545
doi: 10.1016/j.acra.2010.06.016
Gur D, Bandos AI, Cohen CS et al (2008) The “laboratory” effect: comparing radiologists’ performance and variability during prospective clinical and laboratory mammography interpretations. Radiology 249:47–53. https://doi.org/10.1148/radiol.2491072025
pubmed: 18682584
pmcid: 2607194
doi: 10.1148/radiol.2491072025