A deep learning approach to hard exudates detection and disorganization of retinal inner layers identification on OCT images.
Journal
Scientific reports
ISSN: 2045-2322
Titre abrégé: Sci Rep
Pays: England
ID NLM: 101563288
Informations de publication
Date de publication:
19 Jul 2024
19 Jul 2024
Historique:
received:
11
01
2024
accepted:
03
06
2024
medline:
20
7
2024
pubmed:
20
7
2024
entrez:
19
7
2024
Statut:
epublish
Résumé
The purpose of the study was to detect Hard Exudates (HE) and classify Disorganization of Retinal Inner Layers (DRIL) implementing a Deep Learning (DL) system on optical coherence tomography (OCT) images of eyes with diabetic macular edema (DME). We collected a dataset composed of 442 OCT images on which we annotated 6847 HE and the presence of DRIL. A complex operational pipeline was defined to implement data cleaning and image transformations, and train two DL models. The state-of-the-art neural network architectures (Yolov7, ConvNeXt, RegNetX) and advanced techniques were exploited to aggregate the results (Ensemble learning, Edge detection) and obtain a final model. The DL approach reached good performance in detecting HE and classifying DRIL. Regarding HE detection the model got an AP@0.5 score equal to 34.4% with Precision of 48.7% and Recall of 43.1%; while for DRIL classification an Accuracy of 91.1% with Sensitivity and Specificity both of 91.1% and AUC and AUPR values equal to 91% were obtained. The P-value was lower than 0.05 and the Kappa coefficient was 0.82. The DL models proved to be able to identify HE and DRIL in eyes with DME with a very good accuracy and all the metrics calculated confirmed the system performance. Our DL approach demonstrated to be a good candidate as a supporting tool for ophthalmologists in OCT images analysis.
Identifiants
pubmed: 39030181
doi: 10.1038/s41598-024-63844-9
pii: 10.1038/s41598-024-63844-9
doi:
Types de publication
Journal Article
Langues
eng
Sous-ensembles de citation
IM
Pagination
16652Informations de copyright
© 2024. The Author(s).
Références
Yau, J. W. et al. Global prevalence and major risk factors of diabetic retinopathy. Diabetes Care 35(3), 556–564. https://doi.org/10.2337/dc11-1909 (2012).
doi: 10.2337/dc11-1909
pubmed: 22301125
pmcid: 3322721
Varma, R. et al. Prevalence of and risk factors for diabetic macular edema in the United States. JAMA Ophthalmol. 132(11), 1334–1340. https://doi.org/10.1001/jamaophthalmol.2014.2854 (2014).
doi: 10.1001/jamaophthalmol.2014.2854
pubmed: 25125075
pmcid: 4576994
Starace, V. et al. The role of inflammation and neurodegeneration in diabetic macular edema. Ther Adv Ophthalmol. 13, 25158414211055964. https://doi.org/10.1177/25158414211055963 (2021).
doi: 10.1177/25158414211055963
pubmed: 34901746
pmcid: 8652911
Capitão, M. & Soares, R. Angiogenesis and inflammation crosstalk in diabetic retinopathy. J Cell Biochem. 117(11), 2443–2453. https://doi.org/10.1002/jcb.25575 (2016).
doi: 10.1002/jcb.25575
pubmed: 27128219
Miyamoto, K. et al. Prevention of leukostasis and vascular leakage in streptozotocin-induced diabetic retinopathy via intercellular adhesion molecule-1 inhibition. Proc Natl Acad Sci U S A. 96(19), 10836–10841. https://doi.org/10.1073/pnas.96.19.10836 (1999).
doi: 10.1073/pnas.96.19.10836
pubmed: 10485912
pmcid: 17969
Lee, H., Jang, H., Choi, Y. A., Kim, H. C. & Chung, H. Association between soluble CD14 in the aqueous humor and hyperreflective foci on optical coherence tomography in patients with diabetic macular edema. Invest Ophthalmol Vis Sci. 59(2), 715–721. https://doi.org/10.1167/iovs.17-23042 (2018).
doi: 10.1167/iovs.17-23042
pubmed: 29392317
Virgili, G. et al. Optical coherence tomography (OCT) for detection of macular oedema in patients with diabetic retinopathy. Cochrane Database Syst Rev. 1, CD008081. https://doi.org/10.1002/14651858.CD008081.pub3 (2015).
doi: 10.1002/14651858.CD008081.pub3
pubmed: 25564068
Huang, H., Jansonius, N. M., Chen, H. & Los, L. I. Hyperreflective dots on OCT as a predictor of treatment outcome in diabetic macular edema: A systematic review. Ophthalmol Retina. 6(9), 814–827. https://doi.org/10.1016/j.oret.2022.03.020 (2022).
doi: 10.1016/j.oret.2022.03.020
pubmed: 35367382
Sorour, O. A. et al. Persistent diabetic macular edema: Definition, incidence, biomarkers, and treatment methods. Surv Ophthalmol. 68(2), 147–174. https://doi.org/10.1016/j.survophthal.2022.11.008 (2023).
doi: 10.1016/j.survophthal.2022.11.008
pubmed: 36436614
Zhang, J. et al. Diabetic macular edema: Current understanding, molecular mechanisms and therapeutic implications. Cells. 11(21), 3362. https://doi.org/10.3390/cells11213362 (2022).
doi: 10.3390/cells11213362
pubmed: 36359761
pmcid: 9655436
Kwan, C. C. & Fawzi, A. A. Imaging and biomarkers in diabetic macular edema and diabetic retinopathy. Curr Diab Rep. 19(10), 95. https://doi.org/10.1007/s11892-019-1226-2 (2019).
doi: 10.1007/s11892-019-1226-2
pubmed: 31473838
Chou, H. D. et al. Optical coherence tomography and imaging biomarkers as outcome predictors in diabetic macular edema treated with dexamethasone implant. Sci Rep. 12(1), 3872. https://doi.org/10.1038/s41598-022-07604-7 (2022).
doi: 10.1038/s41598-022-07604-7
pubmed: 35264625
pmcid: 8907275
Diabetic Retinopathy Clinical Research Network et al. Relationship between optical coherence tomography-measured central retinal thickness and visual acuity in diabetic macular edema. Ophthalmology. 2007;114(3):525–536. https://doi.org/10.1016/j.ophtha.2006.06.052
Fickweiler, W. et al. Predictive value of optical coherence tomographic features in the Bevacizumab and Ranibizumab in patients with diabetic macular edema (BRDME) study. Retina. 38(4), 812–819. https://doi.org/10.1097/IAE.0000000000001626 (2018).
doi: 10.1097/IAE.0000000000001626
pubmed: 28406860
Giocanti-Aurégan, A. et al. Functional and anatomical outcomes in patients with serous retinal detachment in diabetic macular edema treated with ranibizumab. Invest. Ophthalmol. Vis. Sci. 58(2), 797–800. https://doi.org/10.1167/iovs.16-20855 (2017).
doi: 10.1167/iovs.16-20855
pubmed: 28152140
Seo, K. H., Yu, S. Y., Kim, M. & Kwak, H. W. Visual and morphologic outcomes of intravitreal Ranibizumab for diabetic macular edema based on optical coherence tomography patterns. Retina. 36(3), 588–595. https://doi.org/10.1097/IAE.0000000000000770 (2016).
doi: 10.1097/IAE.0000000000000770
pubmed: 26398695
Pelosini, L. et al. Optical coherence tomography may be used to predict visual acuity in patients with macular edema. Invest Ophthalmol Vis Sci. 52(5), 2741–2748. https://doi.org/10.1167/iovs.09-4493 (2011).
doi: 10.1167/iovs.09-4493
pubmed: 20538987
Mateen, M., Wen, J., Nasrullah, N., Sun, S. & Hayat, S. Exudate detection for diabetic retinopathy using pretrained convolutional neural networks. Complexity. https://doi.org/10.1155/2020/5801870 (2020).
doi: 10.1155/2020/5801870
Alyoubi, W. L., Abulkhair, M. F. & Shalash, W. M. Diabetic retinopathy fundus image classification and lesions localization system using deep learning. Sensors. 21(11), 3704. https://doi.org/10.3390/s21113704 (2021).
doi: 10.3390/s21113704
pubmed: 34073541
pmcid: 8198489
Tsiknakis, N. et al. Deep learning for diabetic retinopathy detection and classifiction based on fundus images: A review. Comput Biol Med. 135, 104599. https://doi.org/10.1016/j.compbiomed.2021.104599 (2021).
doi: 10.1016/j.compbiomed.2021.104599
pubmed: 34247130
Santos, C., Aguiar, M., Welfer, D. & Belloni, B. A new approach for detecting fundus lesions using image processing and deep neural network architecture based on YOLO model. Sensors. 22(17), 6441. https://doi.org/10.3390/s22176441 (2022).
doi: 10.3390/s22176441
pubmed: 36080898
pmcid: 9460625
Samagaio, G. et al. Automatic macular edema identification and characterization using OCT images. Comput Methods Programs Biomed. 163, 47–63. https://doi.org/10.1016/j.cmpb.2018.05.033 (2018).
doi: 10.1016/j.cmpb.2018.05.033
pubmed: 30119857
Alsaih, K. et al. Machine learning techniques for diabetic macular edema (DME) classification on SD-OCT images. BioMed Eng OnLine 16(1), 68. https://doi.org/10.1186/s12938-017-0352-9 (2017).
doi: 10.1186/s12938-017-0352-9
pubmed: 28592309
pmcid: 5463338
Syed, A. M., Hassan, T., Akram, M. U., Naz, S. & Khalid, S. Automated diagnosis of macular edema and central serous retinopathy through robust reconstruction of 3D retinal surfaces. Comput Methods Programs Biomed. 137, 1–10. https://doi.org/10.1016/j.cmpb.2016.09.004 (2016).
doi: 10.1016/j.cmpb.2016.09.004
pubmed: 28110716
Li, F., Chen, H., Liu, Z., Zhang, X. & Wu, Z. Fully automated detection of retinal disorders by image-based deep learning. Graefes Arch Clin Exp Ophthalmol. 257(3), 495–505. https://doi.org/10.1007/s00417-018-04224-8 (2019).
doi: 10.1007/s00417-018-04224-8
pubmed: 30610422
Sandhu, H. S. et al. Automated diagnosis of diabetic retinopathy using clinical biomarkers, optical coherence tomography, and optical coherence tomography angiography. Am J Ophthalmol. 216, 201–206. https://doi.org/10.1016/j.ajo.2020.01.016 (2020).
doi: 10.1016/j.ajo.2020.01.016
pubmed: 31982407
Midena, E. et al. Validation of an automated artificial intelligence algorithm for the quantification of major OCT parameters in diabetic macular Edema. Clin Med. 12(6), 2134. https://doi.org/10.3390/jcm12062134 (2023).
doi: 10.3390/jcm12062134
Tejero, J. G. et al. Predicting OCT biological marker localization from weak annotations. Sci Rep. 13(1), 19667. https://doi.org/10.1038/s41598-023-47019-6 (2023).
doi: 10.1038/s41598-023-47019-6
pubmed: 37952011
pmcid: 10640596
Wang, X. et al. Automated evaluation of retinal hyperreflective foci changes in diabetic macular edema patients before and after intravitreal injection. Front Med (Lausanne). 6(10), 1280714. https://doi.org/10.3389/fmed.2023.1280714 (2023).
doi: 10.3389/fmed.2023.1280714
Suciu, C. I., Marginean, A., Suciu, V. I., Muntean, G. A. & Nicoară, S. D. Diabetic macular Edema optical coherence tomography biomarkers detected with EfficientNetV2B1 and ConvNeXt. Diagnostics (Basel). 14(1), 76. https://doi.org/10.3390/diagnostics14010076 (2023).
doi: 10.3390/diagnostics14010076
pubmed: 38201384
pmcid: 10795694
Singh, R. et al. Deep learning algorithm detects presence of disorganization of retinal inner layers (DRIL)—An early imaging biomarker in diabetic retinopathy. Transl Vis Sci Technol. 12(7), 6. https://doi.org/10.1167/tvst.12.7.6 (2023).
doi: 10.1167/tvst.12.7.6
pubmed: 37410472
pmcid: 10337787
Vujosevic, S. et al. Hyperreflective retinal spots in normal and diabetic eyes: B-Scan and En face spectral domain optical coherence tomography evaluation. Retina 37(6), 1092–1103. https://doi.org/10.1097/IAE.0000000000001304 (2017).
doi: 10.1097/IAE.0000000000001304
pubmed: 27668929
Fragiotta, S. et al. Significance of hyperreflective foci as an optical coherence tomography biomarker in retinal diseases: Characterization and clinical implications. J Ophthalmol Mol 17(2021), 6096017. https://doi.org/10.1155/2021/6096017 (2021).
doi: 10.1155/2021/6096017
Sun, J. K. et al. Disorganization of the retinal inner layers as a predictor of visual acuity in eyes with center-involved diabetic macular edema. JAMA Ophthalmol. 132(11), 1309–1316. https://doi.org/10.1001/jamaophthalmol.2014.2350 (2014).
doi: 10.1001/jamaophthalmol.2014.2350
pubmed: 25058813
Redmon, J., Divvala, S., Girshick, R., & Farhadi, A. You only look once: Unified, real-time object detection. (2015).
Wang, C.Y., Bochkovskiy, A., & Liao, H.Y.M. YOLOv7: Trainable bag-of-freebies sets new state-of-the-art for real-time object detectors. (2022); https://doi.org/10.48550/arXiv.2207.02696 .
Canny, J. A computational approach to edge detection. IEEE Trans. Pattern Anal. Mach. Intell. 8(6), 679–698. https://doi.org/10.1109/TPAMI.1986.4767851 (1986).
doi: 10.1109/TPAMI.1986.4767851
pubmed: 21869365
Stephens, R. S. Probabilistic approach to the Hough transform. Image and Vision Computing 9(1), 66–71. https://doi.org/10.1016/0262-8856(91)90051-P (1991).
doi: 10.1016/0262-8856(91)90051-P
He, K., Zhang, X., Ren, S., Sun, J. Deep residual learning for image recognition. In CVPR 770–778 (2016).
Liu, Z. et al. A ConvNet for the 2020s. In CVPR, 11966–11976 (2022) https://doi.org/10.1109/CVPR52688.2022.01167 .
Radosavovic, I., Kosaraju, R.P., Girshick, R., He, K., Dollár, P. Designing network design spaces. In CVPR. 10425–10433 (2020). https://doi.org/10.1109/CVPR42600.2020.01044 .
Bekkar, M., Djemaa, K. H. & Alitouche, T. A. Evaluation measures for models assessment over imbalanced data sets. J. Inf. Eng. Appl. 3(10), 2013 (2013).
Fleiss, J. L. Measuring nominal scale agreement among many raters. Psychol. Bull. 76(5), 378–382 (1971).
doi: 10.1037/h0031619
Landis, J. R. & Koch, G. G. The measurement of observer agreement for categorical data. Biometrics. 33(1), 159–174 (1977).
doi: 10.2307/2529310
pubmed: 843571
Ashraf, M., Souka, A. & Adelman, R. Predicting outcomes to anti-vascular endothelial growth factor (VEGF) therapy in diabetic macular oedema: A review of the literature. Br J Ophthalmol. 100(12), 1596–1604. https://doi.org/10.1136/bjophthalmol-2016-308388 (2016).
doi: 10.1136/bjophthalmol-2016-308388
pubmed: 27231313
Parravano, M., Costanzo, E. & Querques, G. Profile of non-responder and late responder patients treated for diabetic macular Edema: Systemic and ocular factors. Acta Diabetol. 57(8), 911–921. https://doi.org/10.1007/s00592-020-01496-7 (2020).
doi: 10.1007/s00592-020-01496-7
pubmed: 32114642
Cusick, M. et al. Histopathology and regression of retinal hard exudates in diabetic retinopathy after reduction of elevated serum lipid levels. Ophthalmology. 110(11), 2126–2133. https://doi.org/10.1016/j.ophtha.2003.01.001 (2003).
doi: 10.1016/j.ophtha.2003.01.001
pubmed: 14597519
Møller, F. & Bek, T. The relation between visual acuity, fixation stability, and the size and location of foveal hard exudates after photocoagulation for diabetic maculopathy: A 1-year follow-up study. Graefes Arch Clin Exp Ophthalmol. 241(6), 458–462. https://doi.org/10.1007/s00417-003-0661-5 (2003).
doi: 10.1007/s00417-003-0661-5
pubmed: 12756576
Mehta, H. et al. Efficacy of dexamethasone versus bevacizumab on regression of hard exudates in diabetic maculopathy: Data from the BEVORDEX randomised clinical trial. Br J Ophthalmol. 100(7), 1000–1004. https://doi.org/10.1136/bjophthalmol-2015-307797 (2016).
doi: 10.1136/bjophthalmol-2015-307797
pubmed: 26537156
Singuri, S. Clinical utility of spectral-domain optical coherence tomography marker disorganization of retinal inner layers in diabetic retinopathy. Ophthalmic Surg Lasers Imaging Retina. 54(12), 692–700. https://doi.org/10.3928/23258160-20231031-02 (2023).
doi: 10.3928/23258160-20231031-02
pubmed: 38113360
Das, R., Spence, G., Hogg, R. E., Stevenson, M. & Chakravarthy, U. Disorganization of inner retina and outer retinal morphology in diabetic macular Edema. JAMA Ophthalmol. 136(2), 202–208. https://doi.org/10.1001/jamaophthalmol.2017.6256 (2018).
doi: 10.1001/jamaophthalmol.2017.6256
pubmed: 29327033
pmcid: 5838716
Marupally, A. G. et al. Semi-automated quantification of hard exudates in colour fundus photographs diagnosed with diabetic retinopathy. BMC Ophthalmo. https://doi.org/10.1186/s12886-017-0563-7 (2017).
doi: 10.1186/s12886-017-0563-7
Sasaki, M. et al. Quantitative measurement of hard exudates in patients with diabetes and their associations with serum lipid levels. Invest Ophthalmol Vis Sci. 54(8), 5544–5550. https://doi.org/10.1167/iovs.13-11849 (2013).
doi: 10.1167/iovs.13-11849
pubmed: 23847305
Kessler, L. J., Bagautdinov, D., Łabuz, G., Auffarth, G. U. & Khoramnia, R. Semi-automated quantification of retinal and choroidal biomarkers in retinal vascular diseases: Agreement of spectral-domain optical coherence tomography with and without enhanced depth imaging mode. Diagnostics (Basel) 12(2), 333. https://doi.org/10.3390/diagnostics12020333 (2022).
doi: 10.3390/diagnostics12020333
pubmed: 35204422
Tao, L. et al. Diagnostic assessment of deep learning algorithms for diabetic retinopathy screening. Inf. Sci. 2019(501C), 511–522. https://doi.org/10.1016/j.ins.2019.06.011 (2019).
doi: 10.1016/j.ins.2019.06.011
Porwal, P. et al. IDRiD: Diabetic retinopathy—segmentation and grading challenge. Med Image Anal. 59, 101561. https://doi.org/10.1016/j.media.2019.101561 (2020).
doi: 10.1016/j.media.2019.101561
pubmed: 31671320
Vahadane, A., Joshi, A., Madan, K., & Dastidar, T.R. Detection of diabetic macular edema in optical coherence tomography scans using patch based deep learning. In 2018 IEEE 15th International Symposium on Biomedical Imaging (ISBI 2018), 1427–1430 (Washington, DC, USA, 2018) https://doi.org/10.1109/ISBI.2018.8363840 .