Histopathological Image Deep Feature Representation for CBIR in Smart PACS.
Computational pathology
Content-based image retrieval
Deep learning
PACS
Journal
Journal of digital imaging
ISSN: 1618-727X
Titre abrégé: J Digit Imaging
Pays: United States
ID NLM: 9100529
Informations de publication
Date de publication:
10 2023
10 2023
Historique:
received:
16
03
2023
accepted:
12
04
2023
revised:
16
03
2023
medline:
18
9
2023
pubmed:
10
6
2023
entrez:
9
6
2023
Statut:
ppublish
Résumé
Pathological Anatomy is moving toward computerizing processes mainly due to the extensive digitization of histology slides that resulted in the availability of many Whole Slide Images (WSIs). Their use is essential, especially in cancer diagnosis and research, and raises the pressing need for increasingly influential information archiving and retrieval systems. Picture Archiving and Communication Systems (PACSs) represent an actual possibility to archive and organize this growing amount of data. The design and implementation of a robust and accurate methodology for querying them in the pathology domain using a novel approach are mandatory. In particular, the Content-Based Image Retrieval (CBIR) methodology can be involved in the PACSs using a query-by-example task. In this context, one of many crucial points of CBIR concerns the representation of images as feature vectors, and the accuracy of retrieval mainly depends on feature extraction. Thus, our study explored different representations of WSI patches by features extracted from pre-trained Convolution Neural Networks (CNNs). In order to perform a helpful comparison, we evaluated features extracted from different layers of state-of-the-art CNNs using different dimensionality reduction techniques. Furthermore, we provided a qualitative analysis of obtained results. The evaluation showed encouraging results for our proposed framework.
Identifiants
pubmed: 37296349
doi: 10.1007/s10278-023-00832-x
pii: 10.1007/s10278-023-00832-x
pmc: PMC10501985
doi:
Types de publication
Journal Article
Langues
eng
Sous-ensembles de citation
IM
Pagination
2194-2209Informations de copyright
© 2023. The Author(s).
Références
Fraggetta, F. (2019), ‘Clinical-grade Computational Pathology: Alea Iacta Est’, Journal of Pathology Informatics 10, 38.
doi: 10.4103/jpi.jpi_54_19
pubmed: 31921486
pmcid: 6939341
Kumar, N., Gupta, R. & Gupta, S. (2020), ‘Whole Slide Imaging (WSI) in Pathology: Current Perspectives and Future Directions’, Journal of Digital Imaging 33(4), 1034–1040.
doi: 10.1007/s10278-020-00351-z
pubmed: 32468487
pmcid: 7522141
Choplin, R. H., Boehme 2nd, J. & Maynard, C. (1992), ‘Picture archiving and communication systems: an overview.’, Radiographics 12(1), 127–129.
Strickland, N. H. (2000), ‘Pacs (picture archiving and communication systems): filmless radiology’, Archives of disease in childhood 83(1), 82–86.
doi: 10.1136/adc.83.1.82
pubmed: 10869010
pmcid: 1718393
Cui, M. & Zhang, D. Y. (2021), ‘Artificial intelligence and computational pathology’, Laboratory Investigation 101(4), 412–422.
doi: 10.1038/s41374-020-00514-0
pubmed: 33454724
Rinaldi, A. M., Russo, C. & Tommasino, C. (2021), Visual query posing in multimedia web document retrieval, in ‘2021 IEEE 15th International Conference on Semantic Computing (ICSC)’, IEEE, pp. 415–420.
Jeyakumar, V. & Kanagaraj, B. (2019), A medical image retrieval system in pacs environment for clinical decision making, in ‘Intelligent Data Analysis for Biomedical Applications’, Elsevier, pp. 121–146.
Valente, F., Costa, C. & Silva, A. (2013), ‘Dicoogle, a pacs featuring profiled content based image retrieval’, PloS one 8(5), e61888.
doi: 10.1371/journal.pone.0061888
pubmed: 23671578
pmcid: 3646026
Rinaldi, A. M. & Russo, C. (2020), A content based image retrieval approach based on multiple multimedia features descriptors in e-health environment, in ‘2020 IEEE International Symposium on Medical Measurements and Applications (MeMeA)’, IEEE, pp. 1–6.
Ma, J., Jiang, X., Fan, A., Jiang, J. & Yan, J. (2021), ‘Image matching from handcrafted to deep features: A survey’, International Journal of Computer Vision 129(1), 23–79.
doi: 10.1007/s11263-020-01359-2
Donahue, J., Jia, Y., Vinyals, O., Hoffman, J., Zhang, N., Tzeng, E. & Darrell, T. (2014), Decaf: A deep convolutional activation feature for generic visual recognition, in ‘International conference on machine learning’, PMLR, pp. 647–655.
Choe, J., Hwang, H. J., Seo, J. B., Lee, S. M., Yun, J., Kim, M.-J., Jeong, J., Lee, Y., Jin, K., Park, R. et al. (2022), ‘Content-based image retrieval by using deep learning for interstitial lung disease diagnosis with chest ct’, Radiology 302(1), 187–197.
doi: 10.1148/radiol.2021204164
pubmed: 34636634
Kumar, S. et al. (2021), A comprehensive study on deep learning approach for cbir, in ‘2021 10th IEEE International Conference on Communication Systems and Network Technologies (CSNT)’, IEEE, pp. 560–564.
Maji, S. & Bose, S. (2021), ‘Cbir using features derived by deep learning’, ACM/IMS Transactions on Data Science (TDS) 2(3), 1–24.
doi: 10.1145/3470568
Rinaldi, A. M., Russo, C. & Tommasino, C. (2020), ‘A knowledge-driven multimedia retrieval system based on semantics and deep features’, Future Internet 12(11), 183.
doi: 10.3390/fi12110183
Saritha, R. R., Paul, V. & Kumar, P. G. (2019), ‘Content based image retrieval using deep learning process’, Cluster Computing 22(2), 4187–4200.
doi: 10.1007/s10586-018-1731-0
Van der Laak, J., Litjens, G. & Ciompi, F. (2021), ‘Deep learning in histopathology: the path to the clinic’, Nature medicine 27(5), 775–784.
doi: 10.1038/s41591-021-01343-4
pubmed: 33990804
Banerji, S. & Mitra, S. (2022), ‘Deep learning in histopathology: A review’, Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery 12(1), e1439.
Kimeswenger, S., Tschandl, P., Noack, P., Hofmarcher, M., Rumetshofer, E., Kindermann, H., Silye, R., Hochreiter, S., Kaltenbrunner, M., Guenova, E. et al. (2021), ‘Artificial neural networks and pathologists recognize basal cell carcinomas based on different histological patterns’, Modern Pathology 34(5), 895–903.
doi: 10.1038/s41379-020-00712-7
pubmed: 33184470
Alturkistani, H. A., Tashkandi, F. M. & Mohammedsaleh, Z. M. (2015), ‘Histological Stains: A Literature Review and Case Study’, Global Journal of Health Science 8(3), 72–79.
doi: 10.5539/gjhs.v8n3p72
pubmed: 26493433
pmcid: 4804027
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K. & Fei-Fei, L. (n.d.), Imagenet: A large-scale hierarchical image database.
Caicedo, J. C., Gonzalez, F. A. & Romero, E. (2008), A semantic content-based retrieval method for histopathology images, in ‘Asia Information Retrieval Symposium’, Springer, pp. 51–60.
Caicedo, J. C., Vanegas, J. A., Páez, F. & González, F. A. (2014), ‘Histology image search using multimodal fusion’, Journal of Biomedical Informatics 51, 114–128.
doi: 10.1016/j.jbi.2014.04.016
pubmed: 24820052
Zhang, X., Liu, W., Dundar, M., Badve, S. & Zhang, S. (2014), ‘Towards large-scale histopathological image analysis: Hashing-based image retrieval’, IEEE Transactions on Medical Imaging 34(2), 496–506.
doi: 10.1109/TMI.2014.2361481
pubmed: 25314696
Qi, X., Wang, D., Rodero, I., Diaz-Montes, J., Gensure, R. H., Xing, F., Zhong, H., Goodell, L., Parashar, M., Foran, D. J. et al. (2014), ‘Content-based histopathology image retrieval using cometcloud’, BMC bioinformatics 15(1), 1–17.
doi: 10.1186/1471-2105-15-287
Jimenez-del Toro, O., Otálora, S., Atzori, M. & Müller, H. (2017), Deep multimodal case–based retrieval for large histopathology datasets, in ‘International Workshop on Patch-based Techniques in Medical Imaging’, Springer, pp. 149–157.
Zheng, Y., Jiang, Z., Ma, Y., Zhang, H., Xie, F., Shi, H. & Zhao, Y. (2017), Content-based histopathological image retrieval for whole slide image database using binary codes, in ‘Medical Imaging 2017: Digital Pathology’, Vol. 10140, SPIE, pp. 266–271.
Kumar, M. D., Babaie, M. & Tizhoosh, H. R. (2018), Deep barcodes for fast retrieval of histopathology scans, in ‘2018 International Joint Conference on Neural Networks (IJCNN)’, IEEE, pp. 1–8.
Hegde, N., Hipp, J. D., Liu, Y., Emmert-Buck, M., Reif, E., Smilkov, D., Terry, M., Cai, C. J., Amin, M. B., Mermel, C. H. et al. (2019), ‘Similar image search for histopathology: Smily’, NPJ digital medicine 2(1), 1–9.
doi: 10.1038/s41746-019-0131-z
Schaer, R., Otálora, S., Jimenez-del Toro, O., Atzori, M. & Müller, H. (2019), ‘Deep learning-based retrieval system for gigapixel histopathology cases and the open access literature’, Journal of pathology informatics 10.
Yazdi, M. & Erfankhah, H. (2020), ‘Multiclass histology image retrieval, classification using riesz transform and local binary pattern features’, Computer Methods in Biomechanics and Biomedical Engineering: Imaging & Visualization 8(6), 595–607.
Yang, P., Zhai, Y., Li, L., Lv, H., Wang, J., Zhu, C. & Jiang, R. (2020), ‘A deep metric learning approach for histopathological image retrieval’, Methods 179, 14–25.
doi: 10.1016/j.ymeth.2020.05.015
pubmed: 32439386
Kalra, S., Tizhoosh, H. R., Choi, C., Shah, S., Diamandis, P., Campbell, C. J. & Pantanowitz, L. (2020), ‘Yottixel–an image search engine for large archives of histopathology whole slide images’, Medical Image Analysis 65, 101757.
doi: 10.1016/j.media.2020.101757
pubmed: 32623275
Ricardo Baeza, Y. & Berthier Ribeiro, N. (2011), ‘Modern information retrieval’.
Keras (2022), ‘Keras applications’. https://keras.io/api/applications/
Simonyan, K. & Zisserman, A. (2014) , ‘Very deep convolutional networks for large-scale image recognition’, arXiv preprint arXiv:1409.1556 .
Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J. & Wojna, Z. (2015), Rethinking theinception architecture for computer vision, in ‘Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition’, pp. 2818–2826.
Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J. & Wojna, Z. (2016), Rethinking the inception architecture for computer vision, in ‘Proceedings of the IEEE conference on computer vision and pattern recognition’, pp. 2818–2826.
He, K., Zhang, X., Ren, S. & Sun, J. (2016a), Deep residual learning for image recognition, in ‘Proceedings of the IEEE conference on computer vision and pattern recognition’, pp. 770–778.
He, K., Zhang, X., Ren, S. & Sun, J. (2016b), Identity mappings in deep residual networks, in ‘European conference on computer vision’, Springer, pp. 630–645.
Szegedy, C., Ioffe, S., Vanhoucke, V. & Alemi, A. A. (2017), Inception-v4, inception-resnet and the impact of residual connections on learning, in ‘Thirty-first AAAI conference on artificial intelligence’.
Chollet, F. (2016), ‘Xception: deep learning with depthwise separable convolutions. corr abs/1610.02357 (2016)’, arXiv preprint arXiv:1610.02357 .
Huang, G., Liu, Z., Van Der Maaten, L. & Weinberger, K. Q. (2017), Densely connected convolutional networks, in ‘Proceedings of the IEEE conference on computer vision and pattern recognition’, pp. 4700–4708.
Zoph, B., Vasudevan, V., Shlens, J. & Le, Q. V. (2017), ‘Learning transferable architectures for scalable image recognition.(2017)’, arXiv preprint arXiv:1707.07012 2(6).
Howard, A. G., Zhu, M., Chen, B., Kalenichenko, D., Wang, W., Weyand, T., Andreetto, M. & Adam, H. (2017), ‘Mobilenets: Efficient convolutional neural networks for mobile vision applications’, arXiv preprint arXiv:1704.04861 .
Sandler, M., Howard, A., Zhu, M., Zhmoginov, A. & Chen, L.-C. (2018), Mobilenetv2: Inverted residuals and linear bottlenecks, in ‘Proceedings of the IEEE conference on computer vision and pattern recognition’, pp. 4510–4520.
Tan, M. & Le, Q. (2019), Efficientnet: Rethinking model scaling for convolutional neural networks, in ‘International conference on machine learning’, PMLR, pp. 6105–6114.
Chen, Z., Zhou, F., Trimponias, G. & Li, Z. (2020), ‘Multi-objective neural architecture search via non-stationary policy gradient’, arXiv preprint arXiv:2001.08437 .
Tan, M. & Le, Q. (2021), Efficientnetv2: Smaller models and faster training, in ‘International Conference on Machine Learning’, PMLR, pp. 10096–10106.
Xiong, Y., Liu, H., Gupta, S., Akin, B., Bender, G., Wang, Y., Kindermans, P.-J., Tan, M., Singh, V. & Chen, B. (2021), Mobiledets: Searching for object detection architectures for mobile accelerators, in ‘Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition’, pp. 3825–3834.
Aresta, G., Araújo, T., Kwok, S., Chennamsetty, S. S., Safwan, M., Alex, V., Marami, B., Prastawa, M., Chan, M., Donovan, M. et al. (2019), ‘Bach: Grand challenge on breast cancer histology images’, Medical image analysis 56, 122–139.
doi: 10.1016/j.media.2019.05.010
pubmed: 31226662