A comparative evaluation of deep learning approaches for ophthalmology.
Artificial intelligence
Deep learning
Diabetic retinopathy
Fundus imaging
Ophthalmoscopy
Journal
Scientific reports
ISSN: 2045-2322
Titre abrégé: Sci Rep
Pays: England
ID NLM: 101563288
Informations de publication
Date de publication:
18 Sep 2024
18 Sep 2024
Historique:
received:
25
07
2023
accepted:
09
09
2024
medline:
19
9
2024
pubmed:
19
9
2024
entrez:
18
9
2024
Statut:
epublish
Résumé
There is a growing number of publicly available ophthalmic imaging datasets and open-source code for Machine Learning algorithms. This allows ophthalmic researchers and practitioners to independently perform various deep-learning tasks. With the advancement in artificial intelligence (AI) and in the field of imaging, the choice of the most appropriate AI architecture for different tasks will vary greatly. The best-performing AI-dataset combination will depend on the specific problem that needs to be solved and the type of data available. The article discusses different machine learning models and deep learning architectures currently used for various ophthalmic imaging modalities and for different machine learning tasks. It also proposes the most appropriate models based on accuracy and other important factors such as training time, the ability to deploy the model on clinical devices/smartphones, heatmaps that enhance the self-explanatory nature of classification decisions, and the ability to train/adapt on small image datasets to determine if further data collection is worthwhile. The article extensively reviews the existing state-of-the-art AI methods focused on useful machine-learning applications for ophthalmology. It estimates their performance and viability through training and evaluating architectures with different public and private image datasets of different modalities, such as full-color retinal images, OCT images, and 3D OCT scans. The article is expected to benefit the readers by enriching their knowledge of artificial intelligence applied to ophthalmology.
Identifiants
pubmed: 39294275
doi: 10.1038/s41598-024-72752-x
pii: 10.1038/s41598-024-72752-x
doi:
Types de publication
Journal Article
Review
Comparative Study
Langues
eng
Sous-ensembles de citation
IM
Pagination
21829Informations de copyright
© 2024. The Author(s).
Références
Adam, B., & Kaveh, M. The rise of artificial intelligence in healthcare applications. In: Artificial Intelligence in healthcare 25–60 (Elsevier, 2020).
Saria, S. Not all ai is created equal: Strategies for safe and effective adoption. NEJM Catal. Innov. Care Deliv.3(2), https://catalyst.nejm.org/doi/full/10.1056/CAT.22.0075 (2022).
Decide-ai: New reporting guidelines to bridge the development-to-implementation gap in clinical artificial intelligence. Nat. Med.27(2), 186–187 (2021).
Liu, X. et al. Reporting guidelines for clinical trial reports for interventions involving artificial intelligence: The consort-ai extension. Lancet Digit. Health2(10), e537–e548 (2020).
doi: 10.1016/S2589-7500(20)30218-1
Food and Drug Administration and others, Proposed regulatory framework for modifications to artificial intelligence/machine learning (ai/ml)-based software as a medical device (samd) (2019).
Korot, E. et al. Code-free deep learning for multi-modality medical image classification. Nat. Mach. Intelligen.3(4), 288–298. https://www.nature.com/articles/s42256-021-00305-2 (2021).
Yang, G., Ye, Q. & Xia, J. Unbox the black-box for the medical explainable ai via multi-modal and multi-centre data fusion: A mini-review, two showcases and beyond. Inf. Fusion77, 29–52 (2022).
doi: 10.1016/j.inffus.2021.07.016
Papers with code (2022). https://paperswithcode.com
Varun, G. et al. Development and validation of a deep learning algorithm for detection of diabetic retinopathy in retinal fundus photographs. jama316(22), 2402–2410. https://jamanetwork.com/journals/jama/fullarticle/2588763/ (2016).
Lee, C., Baughman, D. & Lee, A. Deep learning is effective for the classification of OCT images of normal versus age-related macular degeneration (2017).
Barros, D. et al. Machine learning applied to retinal image processing for glaucoma detection: Review and perspective. Biomed. Eng. Online19(1), 1–21 (2020).
doi: 10.1186/s12938-020-00767-2
Singh, L. K., Khanna, M., Thawkar, S. & Singh, R. A novel hybridized feature selection strategy for the effective prediction of glaucoma in retinal fundus images. Multimed. Tools Appl.83(15), 46087–46159 (2024).
doi: 10.1007/s11042-023-17081-3
Jiang, P., Dou, Q. & Shi, L. Ophthalmologist-level classification of fundus disease with deep neural networks. Transl. Vis. Sci. Technol.9(2), 39–39 (2020).
doi: 10.1167/tvst.9.2.39
Yijin, H., Lina, L., Pujin, C., Junyan, L. & Xiaoying, T. Identifying the key components in resnet-50 for diabetic retinopathy grading from fundus images: A systematic investigation. Diagnostics13(10), 1664. https://www.mdpi.com/2075-4418/13/10/1664 (2021).
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, Ł. & Polosukhin, I. Attention is all you need Adv. Neural Inf. Process. Syst.30(1), 261–272. https://user.phil.hhu.de/~cwurm/wpcontent/uploads/2020/01/7181-attention-is-all-you-need.pdf (2017).
Imagenet rank (2022). https://paperswithcode.com/sota/image-classification-on-imagenet
He, K., Gan, C., Li, Z., Rekik, I., Yin, Z., Ji, W., Gao, Y., Wang, Q., Zhang, J. & Shen, D. Transformers in medical image analysis: A review Intell. Med.3(1), 59–78. https://www.sciencedirect.com/science/article/pii/S2667102622000717 (2022).
Korngiebel, D. M. & Mooney, S. D. Considering the possibilities and pitfalls of generative pre-trained transformer 3 (gpt-3) in healthcare delivery. NPJ Digit. Med.4(1), 93 (2021).
doi: 10.1038/s41746-021-00464-x
Eyepacs (2022). https://www.kaggle.com/c/diabetic-retinopathy-detection/data
Messidor (2022). https://www.adcis.net/en/third-party/messidor/
Messidor-2 (2022). https://www.adcis.net/en/third-party/messidor2/
Acrima (2022). https://www.kaggle.com/sshikamaru/glaucoma-detection
Papers with code on imagenet (2022). https://paperswithcode.com/sota/image-classification-on-imagenet
Imagenet (2022). https://www.image-net.org/
Boesch, G. Vision transformers (vit) in image recognition–2022 guide, viso. ai (2022). https://viso.ai/deep-learning/vision-transformer-vit/
Dosovitskiy, A. et al. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11929 (2020).
vit-keras (2022). https://github.com/faustomorales/vit-keras
Yuan, L., Hou, Q., Jiang, Z., Feng, J. & Yan, S. Volo: Vision outlooker for visual recognition. IEEE Trans. Pattern Anal. Mach. Intell.45(5), 6575–6586 (2022).
d Garse, L. Keras cv attention models (2022). URL: hhttps://github.com/leondgarse/keras_cv_attention_models
Bao, H., Dong, L., Piao, S. & Wei, F. Beit: Bert pre-training of image transformers. arXiv preprint arXiv:2106.08254 (2021).
Ding, M., Xiao, B., Codella, N., Luo, P., Wang, J. & Yuan, L. Davit: Dual attention vision transformers. In European Conference on Computer Vision 74–92 (Springer, 2022).
Li, Y., Yao, T., Pan, Y. & Mei, T. Contextual transformer networks for visual recognition. IEEE Trans. Pattern Anal. Mach. Intell.45(2), 1489–1500 (2022).
doi: 10.1109/TPAMI.2022.3164083
Dai, Z., Liu, H., Le, Q. V. & Tan, M. Coatnet: Marrying convolution and attention for all data sizes. Adv. Neural Inf. Process. Syst.34, 3965–3977 (2021).
Zhang, H., Wu, C., Zhang, Z., Zhu, Y., Lin, H., Zhang, Z., Sun, Y., He, T., Mueller, J., Manmatha, R. et al. Resnest: Split-attention networks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition 2736–2746 (2022).
Tolstikhin, I. O. et al. Mlp-mixer: An all-mlp architecture for vision. Adv. Neural Inf. Process. Syst.34, 24261–24272 (2021).
Xu, J., Pan, Y., Pan, X., Hoi, S., Yi, Z. & Xu, Z. Regnet: Self-regulated network for image classification. IEEE Trans. Neural Netw. Learn. Syst.34(11), 9562–9567. https://ieeexplore.ieee.org/abstract/document/9743274/ (2022).
Brock, A., De, S., Smith, S. L. & Simonyan, K. High-performance large-scale image recognition without normalization. In International Conference on Machine Learning 1059–1071 (PMLR, 2021).
Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J. & Wojna, Z. Rethinking the inception architecture for computer vision. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition 2818–2826 (2016).
Inceptionv3 code (2022). https://github.com/tensorflow/tensorflow/tree/r1.4/tensorflow/examples/image_retraining
Quantization (2022). https://intellabs.github.io/distiller/quantization.html
Tensorflow android camera demo (2017). https://github.com/tensorflow/tensorflow/tree/r1.4/tensorflow/examples/android
Tensorflow ios examples (2017). https://github.com/tensorflow/tensorflow/tree/r1.4/tensorflow/examples/ios
Selvaraju, R. R., Cogswell, M., Das, A., Vedantam, R., Parikh, D. & Batra, D. Grad-cam: Visual explanations from deep networks via gradient-based localization. In Proceedings of the IEEE International Conference on Computer Vision 618–626 (2017).
Chollet, F. Grad-cam class activation visualization (2021). https://keras.io/examples/vision/grad_cam/
keras cv attention models visualizing (2022). https://github.com/leondgarse/keras_cv_attention_models/tree/main/keras_cv_attention_models/visualizing
Guided back prop (2020). https://github.com/hummat/saliency/blob/master/guided_backprop.py
Chen, Z., Xie, L., Niu, J., Liu, X., Wei, L. & Tian, Q. Visformer: The vision-friendly transformer. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) 589–598 (2021).
François Chollet, J. A. Image classification on small datasets with keras, Posit AI Blog (2017). https://blogs.rstudio.com/ai/posts/2017-12-14-image-classification-on-small-datasets/
Dijkinga, F. J. Methods to avoid overfitting in artificial neural networks, Medium (2023). https://medium.com/@fernando.dijkinga/methods-to-avoid-overfitting-in-artificial-neural-networks-7564518bf65d
Zhu, H., Chen, B. & Yang, C. Understanding why vit trains badly on small datasets: An intuitive perspective. arXiv preprint arXiv:2302.03751 (2023).
Zhang, G. et al. Diabetic retinopathy grading by deep graph correlation network on retinal images without manual annotations. Front. Med.9, 872214 (2022).
doi: 10.3389/fmed.2022.872214
Maddury, S. & Desai, K. Deepad: A deep learning application for predicting amyloid standardized uptake value ratio through pet for alzheimer’s prognosis. Front. Artif. Intell.6, 1091506 (2023).
doi: 10.3389/frai.2023.1091506
Tsuji, T. et al. Classification of optical coherence tomography images using a capsule network. BMC Ophthalmol.20(1), 1–9 (2020).
doi: 10.1186/s12886-020-01382-4
OCT2017 (2017). https://www.kaggle.com/paultimothymooney/kermany2018#OCT2017.zip
OCTID (2018). https://dataverse.scholarsportal.info/dataverse/OCTID
Retinal oct disease classification on oct2017 (2022). https://paperswithcode.com/sota/retinal-oct-disease-classification-on-oct2017
Midena, E. et al. Optical coherence tomography and color fundus photography in the screening of age-related macular degeneration: A comparative, population-based study. Plos One15(8), e0237352 (2020).
doi: 10.1371/journal.pone.0237352
Ahmed, E., Saint, A., El Rahman Shabayek, A., Cherenkova, K., Das, R., Gusev, G., Aouada, D. & Ottersten, B. A survey on deep learning advances on different 3d data representations. arXiv e-prints (2018) arXiv–1808.
Cnn 3d images using tensorflow (2019). https://github.com/jibikbam/CNN-3D-images-Tensorflow
Keras io 3d image classification (2021). https://github.com/keras-team/keras-io/blob/master/examples/vision/3D_image_classification.py
3d-cnn-keras (2016). https://github.com/Ectsang/3D-CNN-Keras
Jaegle, A., Gimeno, F., Brock, A., Vinyals, O., Zisserman, A. & Carreira, J. Perceiver: General perception with iterative attention. In International Conference on Machine Learning 4651–4664 (PMLR, 2021).
Perceiver image classification (2019). https://github.com/keras-team/keras-io/blob/master/examples/vision/perceiver_image_classification.py
Mehanna, N. Visualizing convolutional neural networks outputs (2018). https://naifmehanna.com/2018-09-14-visualizing-convolutional-neural-networks-outputs-part-1/
Covid-19 imaging datasets (2021). https://www.eibir.org/covid-19-imaging-datasets/
Lee, H. The rise of chatgpt: Exploring its potential in medical education. Anat. Sci. Educ. (2023).
Artificial intelligence and human rights (2023). https://www.judiciary.senate.gov/committee-activity/hearings/artificial-intelligence-and-human-rights
Chia, M. A. et al. Validation of a deep learning system for the detection of diabetic retinopathy in indigenous Australians. Br. J. Ophthalmol.108(2), 268–273 (2024).
doi: 10.1136/bjo-2022-322237
Obermeyer, Z., Powers, B., Vogeli, C. & Mullainathan, S. Dissecting racial bias in an algorithm used to manage the health of populations. Science366(6464), 447–453 (2019).
doi: 10.1126/science.aax2342
Wu, J.-H. & Liu, T. Y. A. Application of deep learning to retinal-image-based oculomics for evaluation of systemic health: A review. J. Clin. Med.12(1), 152 (2022).
doi: 10.3390/jcm12010152
Balaskas, K. Oculomics: The eye as a window to systemic disease. Acta Ophthalmol. 100(S275). https://doi.org/10.1111/j.1755-3768.2022.15399 (2022).
MunishKhanna, Singh, L. K. & Garg, H. A novel approach for human diseases prediction using nature inspired computing & machine learning approach. Multimed. Tools Appl.83(6), 17773–17809 (2024).