A Performance-Based Competency Assessment of Pediatric Chest Radiograph Interpretation Among Practicing Physicians.
Journal
The Journal of continuing education in the health professions
ISSN: 1554-558X
Titre abrégé: J Contin Educ Health Prof
Pays: United States
ID NLM: 8805847
Informations de publication
Date de publication:
21 Dec 2022
21 Dec 2022
Historique:
received:
13
07
2022
accepted:
19
10
2022
entrez:
2
2
2023
pubmed:
3
2
2023
medline:
3
2
2023
Statut:
aheadofprint
Résumé
There is limited knowledge on pediatric chest radiograph (pCXR) interpretation skill among practicing physicians. We systematically determined baseline interpretation skill, the number of pCXR cases physicians required complete to achieve a performance benchmark, and which diagnoses posed the greatest diagnostic challenge. Physicians interpreted 434 pCXR cases via a web-based platform until they achieved a performance benchmark of 85% accuracy, sensitivity, and specificity. Interpretation difficulty scores for each case were derived by applying one-parameter item response theory to participant data. We compared interpretation difficulty scores across diagnostic categories and described the diagnoses of the 30% most difficult-to-interpret cases. 240 physicians who practice in one of three geographic areas interpreted cases, yielding 56,833 pCXR case interpretations. The initial diagnostic performance (first 50 cases) of our participants demonstrated an accuracy of 68.9%, sensitivity of 69.4%, and a specificity of 68.4%. The median number of cases completed to achieve the performance benchmark was 102 (interquartile range 69, 176; min, max, 54, 431). Among the 30% most difficult-to-interpret cases, 39.2% were normal pCXR and 32.3% were cases of lobar pneumonia. Cases with a single trauma-related imaging finding, cardiac, hilar, and diaphragmatic pathologies were also among the most challenging. At baseline, practicing physicians misdiagnosed about one-third of pCXR and there was up to an eight-fold difference between participants in number of cases completed to achieve the standardized performance benchmark. We also identified the diagnoses with the greatest potential for educational intervention.
Sections du résumé
BACKGROUND
BACKGROUND
There is limited knowledge on pediatric chest radiograph (pCXR) interpretation skill among practicing physicians.
OBJECTIVES
OBJECTIVE
We systematically determined baseline interpretation skill, the number of pCXR cases physicians required complete to achieve a performance benchmark, and which diagnoses posed the greatest diagnostic challenge.
METHODS
METHODS
Physicians interpreted 434 pCXR cases via a web-based platform until they achieved a performance benchmark of 85% accuracy, sensitivity, and specificity. Interpretation difficulty scores for each case were derived by applying one-parameter item response theory to participant data. We compared interpretation difficulty scores across diagnostic categories and described the diagnoses of the 30% most difficult-to-interpret cases.
RESULTS
RESULTS
240 physicians who practice in one of three geographic areas interpreted cases, yielding 56,833 pCXR case interpretations. The initial diagnostic performance (first 50 cases) of our participants demonstrated an accuracy of 68.9%, sensitivity of 69.4%, and a specificity of 68.4%. The median number of cases completed to achieve the performance benchmark was 102 (interquartile range 69, 176; min, max, 54, 431). Among the 30% most difficult-to-interpret cases, 39.2% were normal pCXR and 32.3% were cases of lobar pneumonia. Cases with a single trauma-related imaging finding, cardiac, hilar, and diaphragmatic pathologies were also among the most challenging.
CONCLUSIONS
CONCLUSIONS
At baseline, practicing physicians misdiagnosed about one-third of pCXR and there was up to an eight-fold difference between participants in number of cases completed to achieve the standardized performance benchmark. We also identified the diagnoses with the greatest potential for educational intervention.
Identifiants
pubmed: 36728983
doi: 10.1097/CEH.0000000000000481
pii: 00005141-990000000-00043
doi:
Types de publication
Journal Article
Langues
eng
Sous-ensembles de citation
IM
Informations de copyright
Copyright © 2022 The Alliance for Continuing Education in the Health Professions, the Association for Hospital Medical Education, and the Society for Academic Continuing Medical Education.
Références
Taves J, Skitch S, Valani R. Determining the clinical significance of errors in pediatric radiograph interpretation between emergency physicians and radiologists. CJEM 2018;20:420–424.
Klein EJ, Koenig M, Diekema DS, et al. Discordant radiograph interpretation between emergency physicians and radiologists in a pediatric emergency department. Pediatr Emerg Care. 1999;15:245–248.
Fabre C, Proisy M, Chapuis C, et al. Radiology residents' skill level in chest x-ray reading. Diagn Interv Imaging. 2018;99:361–370.
Higginson I, Vogel S, Thompson J, et al. Do radiographs requested from a paediatric emergency department in New Zealand need reporting? Emerg Med Australas. 2004;16:288–294.
Johnson J, Kline JA. Intraobserver and interobserver agreement of the interpretation of pediatric chest radiographs. Emerg Radiol. 2010;17:285–290.
Walsh-Kelly CM, Melzer-Lange MD, Hennes HM, et al. Clinical impact of radiograph misinterpretation in a pediatric ED and the effect of physician training level. American Emerg Med. 1995;13:262–264.
Walsh-Kelly CM, Hennes HM, Melzer-Lange MD. False-positive preliminary radiograph interpretations in a pediatric emergency department: clinical and economic impact. Am J Emerg Med. 1997;15:354–356.
Simon HK, Khan NS, Nordenberg DF, et al. Pediatric emergency physician interpretation of plain radiographs: is routine review by a radiologist necessary and cost-effective? Ann Emerg Med. 1996;27:295–298.
Soudack M, Raviv-Zilka L, Ben-Shlush A, et al. Who should be reading chest radiographs in the pediatric emergency department? Pediatr Emerg Care. 2012;28:1052–1054.
Edwards BL, Dorfman D. High-risk pediatric emergencies. Emerg Med Clin North Am. 2020;38:383–400.
Margolis SA, Nilsson KA, Reed RL. Performance in reading radiographs: does level of education predict skill? J Contin Educ Health Prof. 2003;23:48–53.
Gatt ME, Spectre G, Paltiel O, et al. Chest radiographs in the emergency department: is the radiologist really necessary? Postgrad Med J. 2003;79:214–217.
Brunswick JE, Ilkhanipour K, Fuchs S, et al. Emergency medicine resident interpretation of pediatric radiographs. Acad Emerg Med. 1996;3:790–793.
Al Shammari M, Hassan A, AlShamlan N, et al. Family medicine residents' skill levels in emergency chest X-ray interpretation. BMC Fam Pract. 2021;22:39.
Satia I, Bashagha S, Bibi A, et al. Assessing the accuracy and certainty in interpreting chest X-rays in the medical division. Clin Med (London, England) 2013;13:349–352.
Jimah BB, Baffour Appiah A, Sarkodie BD, et al. Competency in chest radiography interpretation by junior doctors and final year medical students at a teaching hospital. Radiol Res Pract. 2020;2020:1–7.
Communicating Radiation Risks in Paediatric Imaging: Information to Support Health Care Discussions about Benefit and Risk. Geneva, Switzerland: World Health Organization. 2016. Available at: https://www.who.int/publications/i/item/978924151034. Accessed May 5 2022.
Frank JR, Snell LS, Cate OT, et al. Competency-based medical education: theory to practice. Med Teach. 2010;32:638–645.
Peck C, McCall M, McLaren B, et al. Continuing medical education and continuing professional development: international comparisons. BMJ 2000;320:432–435.
Leung JS, Brar M, Eltorki M, et al. Development of an in situ simulation-based continuing professional development curriculum in pediatric emergency medicine. Adv Simulation (London). 2020;5:12.
Campbell CM, Parboosingh J. The Royal College experience and plans for the maintenance of certification program. J Contin Educ Health Prof. 2013;33:S36–S47.
O'Grady KAF, Torzillo PJ, Frawley K, et al. The radiological diagnosis of pneumonia in children. Pneumonia. 2014;5:38–51.
Jash D, Maji A, Patra A, et al. Approach to unequal hilum on chest X-ray. J Assoc Chest Physic. 2013;1:32–37.
Yoo S, MacDonald C, Babyn P. Pulmonary vascularity. In: Chest Radiographic Interpretation in Pediatric Cardiac Patients. Stuttgart, Germany: Georg Thieme Verlag, 2010.
Carter BW, Okumura M, Detterbeck FC, et al. Approaching the patient with an anterior mediastinal mass: a guide for radiologists. J Thorac Oncol. 2014;9:S110–S118.
Jetley NK, Al-Assiri AH, Al-Helal AS, et al. Down's syndrome as a factor in the diagnosis, management, and outcome in patients of Morgagni hernia. J Pediatr Surg. 2011;46:636–639.
Cherian T, Mulholland EK, Carlin JB, et al. Standardized interpretation of paediatric chest radiographs for the diagnosis of pneumonia in epidemiological studies. Bull World Health Organ. 2005;83:353–359.
Thau E, Perez M, Pusic MV, et al. Image interpretation: learning analytics–informed education opportunities. AEM Educ Train. 2021;5:e10592.
Boutis K, Pecaric M, Seeto B, et al. Using signal detection theory to model changes in serial learning of radiological image interpretation. Adv Health Sci Educ. 2010;15:647–658.
Lee MS, Pusic M, Carrière B, et al. Building emergency medicine trainee competency in pediatric musculoskeletal radiograph interpretation: a multicenter prospective cohort study. AEM Educ Train. 2019;3:269–279.
Albaum MN, Hill LC, Murphy M, et al. Interobserver reliability of the chest radiograph in community-acquired pneumonia. Chest 1996;110:343–350.
Lee MS, Pusic MV, Camp M, et al. A target population derived method for developing a competency standard in radiograph interpretation. Teach Learn Med. 2022;34:167–177.
Downing SM. Item response theory: applications of modern test theory in medical education. Med Educ. 2003;37:739–745.
Bond TG, Fox CM. Applying the Rasch Model. London, UK: Lawrence Erlbaum Associates Inc. 2007.
Irby DM, O'Brien BC, Shulman LS, et al. Educating Physicians: A Call for Reform of Medical School and Residency. San Francisco, CA: Joey-Bass, 2010.
Frank J, Snell L, Sherbino J. CanMEDS 2015 Physician Competency Framework. Ottawa, Canada: Royal College of Physicians and Surgeons of Canada 2015. Available at: https://canmeds.royalcollege.ca/uploads/en/framework/CanMEDS%202015%20Framework_EN_Reduced.pdf
Englander R, Carraccio C. From theory to practice: making entrustable professional activities come to life in the context of milestones. Acad Med. 2014;89:1321–1323.
Frank JR, Mungroo R, Ahmad Y, et al. Toward a definition of competency-based education in medicine: a systematic review of published definitions. Med Teach. 2010;32:631–637.
Khanduja PK, Bould MD, Naik VN, et al. The role of simulation in continuing medical education for acute care physicians: a systematic review. Crit Care Med. 2015;43:186–193.
Fawole OA, Kelly MS, Steenhoff AP, et al. Interpretation of pediatric chest radiographs by non-radiologist clinicians in Botswana using World Health Organization criteria for endpoint pneumonia. Pediatr Radiol. 2020;50:913–922.
Sargeant J, Wong BM, Campbell CM. CPD of the future: a partnership between quality improvement and competency‐based education. Med Education. 2018;52:125–135.
Iqbal MZ, Al-Eraky MM, AlSheikh MH. Designing entrustable professional activities for training health professional educators: a review of current practices. Med Sci Educator. 2018;28:797–802.
Lockyer J, Bursey F, Richardson D, et al. Competency-based medical education and continuing professional development: a conceptualization for change. Med Teach. 2017;39:617–622.
Hatala R, Gutman J, Lineberry M, et al. How well is each learner learning? Validity investigation of a learning curve-based assessment approach for ECG interpretation. Adv Health Sci Educ. 2018;24:45–63.
Kwan C, Pusic M, Pecaric M, et al. The variable journey in learning to interpret pediatric point of care ultrasound images: a multicenter prospective cohort study. AEM Education Train. 2020;4:111–122.
Cohen E, Berry JG, Sanders L, et al. Status complexicus? the emergence of pediatric complex care. Pediatrics (Evanston). 2018;141:S202–S211.
Jacobs RF. Judicious use of antibiotics for common pediatric respiratory infections. Pediatr Infect Dis J. 2000;19:938–943.
Islam S, Mannix MK, Breuer RK, et al. Guideline adherence and antibiotic utilization by community pediatricians, private urgent care centers, and a pediatric emergency department. Clin Pediatr. 2020;59:21–30.
Phuong NTK, Hoang TT, Van PH, et al. Encouraging rational antibiotic use in childhood pneumonia: a focus on Vietnam and the Western Pacific Region. Pneumonia. 2017;9:7.
Dasgupta S, Kelleman M, Slesnick T, et al. Cardiomegaly on chest radiographs as a predictor of heart disease in the pediatric population. Am J Emerg Med. 2020;38:855–859.
Satou GM, Lacro RV, Chung T, et al. Heart size on chest x-ray as a predictor of cardiac enlargement by echocardiography in children. Pediatr Cardiol. 2001;22:218–222.
Sreeramareddy CT, Panduru KV, Menten J, et al. Time delays in diagnosis of pulmonary tuberculosis: a systematic review of literature. BMC Infect Dis. 2009;9:91.
Storla DG, Yimer S, Bjune GA. A systematic review of delay in the diagnosis and treatment of tuberculosis. BMC Public Health. 2008;8:15.
Mathur P, Sacks L, Auten G, et al. Delayed diagnosis of pulmonary tuberculosis in city hospitals. Arch Intern Med. 1994;154:306–310.
Golub JE, Bur S, Cronin WA, et al. Patient and health care system delays in pulmonary tuberculosis diagnosis in a low-incidence state. Int J Tuberc Lung Dis. 2005;9:992–998.
Elhalaby E, Abo Sikeena M. Delayed presentation of congenital diaphragmatic hernia. Ped Surg Int. 2002;18:480–485.
Fotter R, Schimpl G, Sorantin E, et al. Delayed presentation of congenital diaphragmatic hernia. Pediatr Radiol. 1992;22:187–191.
Topol EJ. High-performance medicine: the convergence of human and artificial intelligence. Nat Med. 2019;25:44–56.
Kahneman D, Sibony O, Sunstein CR. Noise. London, UK: William Collins, 2021.
Bankier AA, Levine D, Halpern EF, et al. Consensus interpretation in imaging research: is there a better way? Radiology. 2010;257:14–17.
Bawa P. Retention in online courses. SAGE Open. 2016;6: 21777.