A pilot cost-analysis study comparing AI-based EyeArt® and ophthalmologist assessment of diabetic retinopathy in minority women in Oslo, Norway.
Artificial intelligence
Automated grading
Cost-analysis
Diabetic retinopathy
Manual grading
Minority women
Norway
Screening
Journal
International journal of retina and vitreous
ISSN: 2056-9920
Titre abrégé: Int J Retina Vitreous
Pays: England
ID NLM: 101677897
Informations de publication
Date de publication:
23 May 2024
23 May 2024
Historique:
received:
14
01
2024
accepted:
17
03
2024
medline:
24
5
2024
pubmed:
24
5
2024
entrez:
24
5
2024
Statut:
epublish
Résumé
Diabetic retinopathy (DR) is the leading cause of adult blindness in the working age population worldwide, which can be prevented by early detection. Regular eye examinations are recommended and crucial for detecting sight-threatening DR. Use of artificial intelligence (AI) to lessen the burden on the healthcare system is needed. To perform a pilot cost-analysis study for detecting DR in a cohort of minority women with DM in Oslo, Norway, that have the highest prevalence of diabetes mellitus (DM) in the country, using both manual (ophthalmologist) and autonomous (AI) grading. This is the first study in Norway, as far as we know, that uses AI in DR- grading of retinal images. On Minority Women's Day, November 1, 2017, in Oslo, Norway, 33 patients (66 eyes) over 18 years of age diagnosed with DM (T1D and T2D) were screened. The Eidon - True Color Confocal Scanner (CenterVue, United States) was used for retinal imaging and graded for DR after screening had been completed, by an ophthalmologist and automatically, using EyeArt Automated DR Detection System, version 2.1.0 (EyeArt, EyeNuk, CA, USA). The gradings were based on the International Clinical Diabetic Retinopathy (ICDR) severity scale [1] detecting the presence or absence of referable DR. Cost-minimization analyses were performed for both grading methods. 33 women (64 eyes) were eligible for the analysis. A very good inter-rater agreement was found: 0.98 (P < 0.01), between the human and AI-based EyeArt grading system for detecting DR. The prevalence of DR was 18.6% (95% CI: 11.4-25.8%), and the sensitivity and specificity were 100% (95% CI: 100-100% and 95% CI: 100-100%), respectively. The cost difference for AI screening compared to human screening was $143 lower per patient (cost-saving) in favour of AI. Our results indicate that The EyeArt AI system is both a reliable, cost-saving, and useful tool for DR grading in clinical practice.
Sections du résumé
BACKGROUND
BACKGROUND
Diabetic retinopathy (DR) is the leading cause of adult blindness in the working age population worldwide, which can be prevented by early detection. Regular eye examinations are recommended and crucial for detecting sight-threatening DR. Use of artificial intelligence (AI) to lessen the burden on the healthcare system is needed.
PURPOSE
OBJECTIVE
To perform a pilot cost-analysis study for detecting DR in a cohort of minority women with DM in Oslo, Norway, that have the highest prevalence of diabetes mellitus (DM) in the country, using both manual (ophthalmologist) and autonomous (AI) grading. This is the first study in Norway, as far as we know, that uses AI in DR- grading of retinal images.
METHODS
METHODS
On Minority Women's Day, November 1, 2017, in Oslo, Norway, 33 patients (66 eyes) over 18 years of age diagnosed with DM (T1D and T2D) were screened. The Eidon - True Color Confocal Scanner (CenterVue, United States) was used for retinal imaging and graded for DR after screening had been completed, by an ophthalmologist and automatically, using EyeArt Automated DR Detection System, version 2.1.0 (EyeArt, EyeNuk, CA, USA). The gradings were based on the International Clinical Diabetic Retinopathy (ICDR) severity scale [1] detecting the presence or absence of referable DR. Cost-minimization analyses were performed for both grading methods.
RESULTS
RESULTS
33 women (64 eyes) were eligible for the analysis. A very good inter-rater agreement was found: 0.98 (P < 0.01), between the human and AI-based EyeArt grading system for detecting DR. The prevalence of DR was 18.6% (95% CI: 11.4-25.8%), and the sensitivity and specificity were 100% (95% CI: 100-100% and 95% CI: 100-100%), respectively. The cost difference for AI screening compared to human screening was $143 lower per patient (cost-saving) in favour of AI.
CONCLUSION
CONCLUSIONS
Our results indicate that The EyeArt AI system is both a reliable, cost-saving, and useful tool for DR grading in clinical practice.
Identifiants
pubmed: 38783384
doi: 10.1186/s40942-024-00547-3
pii: 10.1186/s40942-024-00547-3
doi:
Types de publication
Journal Article
Langues
eng
Pagination
40Informations de copyright
© 2024. The Author(s).
Références
Cleland C. Comparing the International Clinical Diabetic Retinopathy (ICDR) severity scale. Community eye Health. 2023;36(119):10.
pubmed: 37600672
pmcid: 10436766
Danaei G, Finucane MM, Lu Y, Singh GM, Cowan MJ, Paciorek CJ, et al. National, regional, and global trends in fasting plasma glucose and diabetes prevalence since 1980: systematic analysis of health examination surveys and epidemiological studies with 370 country-years and 2.7 million participants. Lancet (British Edition). 2011;378(9785):31–40.
Saaddine JB, Honeycutt AA, Narayan KMV, Zhang X, Klein R, Boyle JP. Projection of Diabetic Retinopathy and other Major Eye diseases among people with diabetes Mellitus: United States, 2005–2050. Arch Ophthalmol. 2008;126(12):1740–7.
doi: 10.1001/archopht.126.12.1740
pubmed: 19064858
Stene LC, Ruiz PLD, Åsvold et al. How many people have diabetes in Norway in 2020? Tidsskr Nor Laegeforen. 2020.
Avogaro A, Fadini GP. Microvascular complications in diabetes: a growing concern for cardiologists. Int J Cardiol. 2019;291:29–35.
doi: 10.1016/j.ijcard.2019.02.030
pubmed: 30833106
Bourne RRA, Jonas JB, Bron AM, Cicinelli MV, Das A, Flaxman SR, et al. Prevalence and causes of vision loss in high-income countries and in Eastern and Central Europe in 2015: magnitude, temporal trends and projections. Br J Ophthalmol. 2018;102(5):575–85.
doi: 10.1136/bjophthalmol-2017-311258
pubmed: 29545417
Teo ZL, Tham Y-C, Yu M, Chee ML, Rim TH, Cheung N, et al. Global prevalence of Diabetic Retinopathy and Projection of Burden through 2045 systematic review and Meta-analysis. Ophthalmology. 2021;128(11):1580–91.
doi: 10.1016/j.ophtha.2021.04.027
pubmed: 33940045
Kilstad HN, Sjolie AK, Goransson L, Hapnes R, Henschien HJ, Alsbirk KE, et al. Prevalence of diabetic retinopathy in Norway: report from a screening study. Acta Ophthalmol. 2012;90(7):609–12.
doi: 10.1111/j.1755-3768.2011.02160.x
pubmed: 21955522
Aspelund T, Thornorisdottir O, Olafsdottir E, Gudmundsdottir A, Einarsdottir AB, Mehlsen J, et al. Individual risk assessment and information technology to optimise screening frequency for diabetic retinopathy. Diabetologia. 2011;54(10):2525–32.
doi: 10.1007/s00125-011-2257-7
pubmed: 21792613
Javitt JC, Aiello LP. Cost-effectiveness of detecting and treating diabetic retinopathy. Ann Intern Med. 1996;124(1):164–9.
doi: 10.7326/0003-4819-124-1_Part_2-199601011-00017
pubmed: 8554212
https://www. oslo-universitetssykehus.no: Oslo University Hospital; [Available from: https://www.oslo-universitetssykehus.no/behandlinger/diabetisk-retinopati-screening/ .
Heydon P, Egan C, Bolter L, Chambers R, Anderson J, Aldington S, et al. Prospective evaluation of an artificial intelligence-enabled algorithm for automated diabetic retinopathy screening of 30 000 patients. Br J Ophthalmol. 2021;105(5):723–8.
doi: 10.1136/bjophthalmol-2020-316594
pubmed: 32606081
Bellemo V, Lim G, Rim TH, Tan GSW, Cheung CY, Sadda S, et al. Artificial Intelligence Screening for Diabetic Retinopathy: the real-world emerging application. Curr Diab Rep. 2019;19(9):72–12.
doi: 10.1007/s11892-019-1189-3
pubmed: 31367962
Tufail A, Kapetanakis VV, Salas-Vega S, Egan C, Rudisill C, Owen CG, et al. An observational study to assess if automated diabetic retinopathy image assessment software can replace one or more steps of manual imaging grading and to determine their cost-effectiveness. Health Technol Assess. 2016;20(92):1–72.
doi: 10.3310/hta20920
pubmed: 27981917
pmcid: 5204130
Meredith S, Grinsven M, Engelberts J, Clarke D, Prior V, Vodrey J, et al. Performance of an artificial intelligence automated system for diabetic eye screening in a large English population. Diabet Med. 2023;40(6):e15055. -n/a.
doi: 10.1111/dme.15055
pubmed: 36719266
Tran AT, Berg TJ, Gjelsvik B, Mdala I, Thue G, Cooper JG, et al. Ethnic and gender differences in the management of type 2 diabetes: a cross-sectional study from Norwegian general practice. BMC Health Serv Res. 2019;19(1):904.
doi: 10.1186/s12913-019-4557-4
pubmed: 31779621
pmcid: 6883677
Tran AT, Straand J, Diep LM, Meyer HE, Birkeland KI, Jenum AK. Cardiovascular disease by diabetes status in five ethnic minority groups compared to ethnic norwegians. BMC Public Health. 2011;11(1):554.
doi: 10.1186/1471-2458-11-554
pubmed: 21752237
pmcid: 3199594
Jenum AK, Diep LM, Holmboe-Ottesen G, Holme IMK, Kumar BN, Birkeland KI. Diabetes susceptibility in ethnic minority groups from Turkey, Vietnam, Sri Lanka and Pakistan compared with norwegians - the association with adiposity is strongest for ethnic minority women. BMC Public Health. 2012;12(1):150.
doi: 10.1186/1471-2458-12-150
pubmed: 22380873
pmcid: 3315409
Tran AT, Diep LM, Cooper JG, Claudi T, Straand J, Birkeland K, et al. Quality of care for patients with type 2 diabetes in general practice according to patients’ ethnic background: a cross-sectional study from Oslo, Norway. BMC Health Serv Res. 2010;10(1):145.
doi: 10.1186/1472-6963-10-145
pubmed: 20507647
pmcid: 2887836
Government N. National minorities.
Group MR. Norway [Available from: https://minorityrights.org/country/norway/ .
Immigrants. and Norwegian-born to immigrant parents. www.ssb.no. Statistics Norway (SSB); 2023.
AMA releases. 2021 CPT code set [press release]. 2021.
Ophthalmology, ICo. Updated 2017 ICO guidelines for Diabetic Eye Care San Francisco2017 [.
Altman D. Benchmarking Inter-Rater Reliability Coefficients. 1991.
Abramoff MD, Roehrenbeck C, Trujillo S, Goldstein J, Graves AS, Repka MX, et al. A reimbursement framework for artificial intelligence in healthcare. NPJ Digit Med. 2022;5(1):72.
doi: 10.1038/s41746-022-00621-w
pubmed: 35681002
pmcid: 9184542
Langelaan M, de Boer MR, van Nispen RM, Wouters B, Moll AC, van Rens GH. Impact of visual impairment on quality of life: a comparison with quality of life in the general population and with other chronic conditions. Ophthalmic Epidemiol. 2007;14(3):119–26.
doi: 10.1080/09286580601139212
pubmed: 17613846
Jones S, Edwards RT. Diabetic retinopathy screening: a systematic review of the economic evidence. Diabet Medicine: J Br Diabet Association. 2010;27(3):249–56.
doi: 10.1111/j.1464-5491.2009.02870.x
Claudi T, Ingskog W, Cooper JG, Jenum AK, Hausken MF. Quality of diabetes care in Norwegian general practice. Tidsskr nor Laegeforen. 2008;128(22):2570–4.
pubmed: 19023353
Norsk Diabetesregister for. voksne [Available from: https://www.kvalitetsregistre.no/registers/364/resultater/466 .
Norsk diabetesregister for voksne. Årsrapport for 2014 med plan for forbedringstiltak [Available from: http://www.noklus.no/Portals/2/Diabetesregisteret/Arsrapport%20Norsk%20diabetesregister%20for%20voksne%202014.pdf .
Sauesund ES, Jørstad ØK, Brunborg C, Moe MC, Erke MG, Fosmark DS et al. A Pilot Study of Implementing Diabetic Retinopathy Screening in the Oslo Region, Norway: Baseline Results. Biomedicines. 2023;11(4):1222.
Julie M, Lauren ABH, Lyndell LL, Salmaan A-Q. Diabetic retinopathy in pregnancy: a review: Diabetic retinopathy in pregnancy. Clin Exp Ophthalmol. 2016;44:321–34.
doi: 10.1111/ceo.12760
Rajesh AE, Davidson OQ, Lee CS, Lee AY. Artificial Intelligence and Diabetic Retinopathy: AI Framework, prospective studies, head-to-head validation, and cost-effectiveness. Diabetes Care. 2023;46(10):1728–39.
doi: 10.2337/dci23-0032
pubmed: 37729502
Ethnicity is not. Biology: retinal pigment score to evaluate biological variability from ophthalmic imaging using machine learning. J Eng. 2023:411.
Tufail A, Rudisill C, Egan C, Kapetanakis VV, Salas-Vega S, Owen CG, et al. Automated Diabetic Retinopathy Image Assessment Software: diagnostic accuracy and cost-effectiveness compared with human graders. Ophthalmology. 2017;124(3):343–51.
doi: 10.1016/j.ophtha.2016.11.014
pubmed: 28024825
Heydon P, Egan C, Bolter L, Chambers R, Anderson J, Aldington S, et al. Prospective evaluation of an artificial intelligence-enabled algorithm for automated diabetic retinopathy screening of 30 000 patients. Br J Ophthalmol. 2021;105(5):723–8.
Lee AY, Lee CS, Hunt MS, Yanagihara RT, Blazes M, Boyko EJ, Multicenter. Head-to-Head, Real-World Validation Study of Seven Automated Artificial Intelligence Diabetic Retinopathy Screening Systems. Diabetes Care 2021;44:XXXX-XXXX. Diabetes Care. 2021;44(5):e108-e9.
Fuller SD, Hu J, Liu JC, Gibson E, Gregory M, Kuo J, et al. Five-year cost-effectiveness modeling of primary Care-Based, Nonmydriatic Automated Retinal Image Analysis Screening among low-income patients with diabetes. J Diabetes Sci Technol. 2022;16(2):415–27.
doi: 10.1177/1932296820967011
pubmed: 33124449
Srisubat A, Kittrongsiri K, Sangroongruangsri S, Khemvaranan C, Shreibati JB, Ching J, et al. Cost-utility analysis of deep learning and trained human graders for Diabetic Retinopathy Screening in a Nationwide Program. Ophthalmol Ther. 2023;12(2):1339–57.
doi: 10.1007/s40123-023-00688-y
pubmed: 36841895
pmcid: 10011252
Liu H, Li R, Zhang Y, Zhang K, Yusufu M, Liu Y, et al. Economic evaluation of combined population-based screening for multiple blindness-causing eye diseases in China: a cost-effectiveness analysis. Lancet Glob Health. 2023;11(3):e456–65.
doi: 10.1016/S2214-109X(22)00554-X
pubmed: 36702141
Lin S, Ma Y, Xu Y, Lu L, He J, Zhu J, et al. Artificial Intelligence in Community-Based Diabetic Retinopathy Telemedicine Screening in Urban China: cost-effectiveness and cost-utility analyses with Real-World Data. JMIR Public Health Surveill. 2023;9:e41624–e.
doi: 10.2196/41624
pubmed: 36821353
pmcid: 9999255
Rossi JG, Rojas-Perilla N, Krois J, Schwendicke F. Cost-effectiveness of Artificial Intelligence as a decision-support system Applied to the Detection and Grading of Melanoma, Dental Caries, and Diabetic Retinopathy. JAMA Netw Open. 2022;5(3):e220269–e.
doi: 10.1001/jamanetworkopen.2022.0269
Xie Y, Nguyen QD, Hamzah H, Lim G, Bellemo V, Gunasekeran DV, et al. Artificial intelligence for teleophthalmology-based diabetic retinopathy screening in a national programme: an economic analysis modelling study. Lancet Digit Health. 2020;2(5):E240–9.
doi: 10.1016/S2589-7500(20)30060-1
pubmed: 33328056
Ipp E, Liljenquist D, Bode B, Shah VN, Silverstein S, Regillo CD, et al. Pivotal evaluation of an Artificial Intelligence System for Autonomous Detection of Referrable and Vision-threatening Diabetic Retinopathy. JAMA Netw Open. 2021;4(11):e2134254–e.
doi: 10.1001/jamanetworkopen.2021.34254
pubmed: 34779843
pmcid: 8593763