A pathologist-AI collaboration framework for enhancing diagnostic accuracies and efficiencies.
Journal
Nature biomedical engineering
ISSN: 2157-846X
Titre abrégé: Nat Biomed Eng
Pays: England
ID NLM: 101696896
Informations de publication
Date de publication:
19 Jun 2024
19 Jun 2024
Historique:
received:
09
06
2023
accepted:
03
05
2024
medline:
20
6
2024
pubmed:
20
6
2024
entrez:
19
6
2024
Statut:
aheadofprint
Résumé
In pathology, the deployment of artificial intelligence (AI) in clinical settings is constrained by limitations in data collection and in model transparency and interpretability. Here we describe a digital pathology framework, nuclei.io, that incorporates active learning and human-in-the-loop real-time feedback for the rapid creation of diverse datasets and models. We validate the effectiveness of the framework via two crossover user studies that leveraged collaboration between the AI and the pathologist, including the identification of plasma cells in endometrial biopsies and the detection of colorectal cancer metastasis in lymph nodes. In both studies, nuclei.io yielded considerable diagnostic performance improvements. Collaboration between clinicians and AI will aid digital pathology by enhancing accuracies and efficiencies.
Identifiants
pubmed: 38898173
doi: 10.1038/s41551-024-01223-5
pii: 10.1038/s41551-024-01223-5
doi:
Types de publication
Journal Article
Langues
eng
Sous-ensembles de citation
IM
Informations de copyright
© 2024. The Author(s), under exclusive licence to Springer Nature Limited.
Références
Kirillov, A. et al. Segment anything. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) 4015–4026 (IEEE, 2023).
Gamper, J., Alemi Koohbanani, N., Benet, K., Khuram, A. & Rajpoot, N. PanNuke: An open pan-cancer histology dataset for nuclei instance segmentation and classification. In Digital Pathology. ECDP 2019. Lecture Notes in Computer Science Vol. 11435 (eds Reyes-Aldasoro, C. C. et al.) 11–19 (Springer, 2019).
Kather, J. N. et al. Predicting survival from colorectal cancer histology slides using deep learning: a retrospective multicenter study. PLoS Med. 16, e1002730 (2019).
pubmed: 30677016
pmcid: 6345440
doi: 10.1371/journal.pmed.1002730
Huang, Z., Bianchi, F., Yuksekgonul, M., Montine, T. J. & Zou, J. A visual–language foundation model for pathology image analysis using medical Twitter. Nat. Med. 29, 2307–2316 (2023).
pubmed: 37592105
doi: 10.1038/s41591-023-02504-3
Lu, M. Y. et al. A visual-language foundation model for computational pathology. Nat. Med. 30, 863–874 (2024).
pubmed: 38504017
doi: 10.1038/s41591-024-02856-4
Chen, R. J. et al. Towards a general-purpose foundation model for computational pathology. Nat. Med. 30, 850–862 (2024).
pubmed: 38504018
doi: 10.1038/s41591-024-02857-3
Amgad, M. et al. A population-level digital histologic biomarker for enhanced prognosis of invasive breast cancer. Nat. Med. 30, 85–97 (2024).
pubmed: 38012314
doi: 10.1038/s41591-023-02643-7
Jiang, X. et al. End-to-end prognostication in colorectal cancer by deep learning: a retrospective, multicentre study. Lancet Digit. Health 6, e33–e43 (2024).
pubmed: 38123254
doi: 10.1016/S2589-7500(23)00208-X
Liu, Y. et al. Artificial intelligence-based breast cancer nodal metastasis detection: insights into the black box for pathologists. Arch. Pathol. Lab. Med. 143, 859–868 (2019).
pubmed: 30295070
doi: 10.5858/arpa.2018-0147-OA
Krogue, J. D. et al. Predicting lymph node metastasis from primary tumor histology and clinicopathologic factors in colorectal cancer using deep learning. Commun. Med. 3, 59 (2023).
pubmed: 37095223
pmcid: 10125969
doi: 10.1038/s43856-023-00282-0
Huang, Z. et al. Artificial intelligence reveals features associated with breast cancer neoadjuvant chemotherapy responses from multi-stain histopathologic images. npj Precis. Oncol. 7, 14 (2023).
pubmed: 36707660
pmcid: 9883475
doi: 10.1038/s41698-023-00352-5
Yamashita, R. et al. Deep learning model for the prediction of microsatellite instability in colorectal cancer: a diagnostic study. Lancet Oncol. 22, 132–141 (2021).
pubmed: 33387492
doi: 10.1016/S1470-2045(20)30535-0
He, J. et al. The practical implementation of artificial intelligence technologies in medicine. Nat. Med. 25, 30–36 (2019).
pubmed: 30617336
pmcid: 6995276
doi: 10.1038/s41591-018-0307-0
Price, W. N. II, Gerke, S. & Cohen, I. G. Potential liability for physicians using artificial intelligence. JAMA 322, 1765–1766 (2019).
Acs, B., Rantalainen, M. & Hartman, J. Artificial intelligence as the next step towards precision pathology. J. Intern. Med. 288, 62–81 (2020).
pubmed: 32128929
doi: 10.1111/joim.13030
Steiner, D. F. et al. Impact of deep learning assistance on the histopathologic review of lymph nodes for metastatic breast cancer. Am. J. Surg. Pathol. 42, 1636–1646 (2018).
pubmed: 30312179
pmcid: 6257102
doi: 10.1097/PAS.0000000000001151
Kiani, A. et al. Impact of a deep learning assistant on the histopathologic classification of liver cancer. npj Digit. Med. 3, 23 (2020).
pubmed: 32140566
pmcid: 7044422
doi: 10.1038/s41746-020-0232-8
Challa, B. et al. Artificial intelligence-aided diagnosis of breast cancer lymph node metastasis on histologic slides in a digital workflow. Mod. Pathol. 36, 100216 (2023).
pubmed: 37178923
doi: 10.1016/j.modpat.2023.100216
Bankhead, P., Loughrey, M. B. & Fernández, J. A. QuPath: open source software for digital pathology image analysis. Sci. Rep. 7, 16878 (2017).
pubmed: 29203879
pmcid: 5715110
doi: 10.1038/s41598-017-17204-5
Schneider, C. A., Rasband, W. S. & Eliceiri, K. W. NIH Image to ImageJ: 25 years of image analysis. Nat. Methods 9, 671–675 (2012).
pubmed: 22930834
pmcid: 5554542
doi: 10.1038/nmeth.2089
Chiu, C. & Clack, N. Napari: a Python multi-dimensional image viewer platform for the research community. Microsc. Microanal. 28(S1), 1576–1577 (2022).
doi: 10.1017/S1431927622006328
Aubreville, M., Bertram, C., Klopfleisch, R. & Maier, A. SlideRunner—a tool for massive cell annotations in whole slide images. in Bildverarbeitung für die Medizin 2018 (eds Maier, A. et al.) 309–314 (Springer, 2018).
Pocock, J. et al. TIAToolbox as an end-to-end library for advanced tissue image analytics. Commun. Med. 2, 120 (2022).
pubmed: 36168445
pmcid: 9509319
doi: 10.1038/s43856-022-00186-5
Schindelin, J. et al. Fiji: an open-source platform for biological-image analysis. Nat. Methods 9, 676–682 (2012).
pubmed: 22743772
doi: 10.1038/nmeth.2019
MONAI model zoo. GitHub https://github.com/Project-MONAI/model-zoo (2022).
Amgad, M. et al. HistomicsTK. GitHub https://digitalslidearchive.github.io/HistomicsTK/ (2016).
Dietvorst, B. J., Simmons, J. P. & Massey, C. Overcoming algorithm aversion: people will use imperfect algorithms if they can (even slightly) modify them. Manage. Sci. 64, 1155–1170 (2018).
doi: 10.1287/mnsc.2016.2643
Longoni, C., Bonezzi, A. & Morewedge, C. K. Resistance to medical artificial intelligence. J. Consum. Res. 46, 629–650 (2019).
doi: 10.1093/jcr/ucz013
Medela, A. et al. Few shot learning in histopathological images: reducing the need of labeled data on biological datasets. In 2019 IEEE 16th International Symposium on Biomedical Imaging (ISBI 2019) 1860–1864 (IEEE, 2019).
van Rijthoven, M. et al. Few-shot weakly supervised detection and retrieval in histopathology whole-slide images. In Medical Imaging 2021: Digital Pathology Vol. 11603, 137–143 (SPIE, 2021).
Chen, J., Jiao, J., He, S., Han, G. & Qin, J. Few-shot breast cancer metastases classification via unsupervised cell ranking. IEEE/ACM Trans. Comput. Biol. Bioinform. 18, 1914–1923 (2021).
pubmed: 31841420
doi: 10.1109/TCBB.2019.2960019
Zhu, Z. et al. EasierPath: an open-source tool for human-in-the-loop deep learning of renal pathology. In Interpretable and Annotation-Efficient Learning for Medical Image Computing. IMIMIC 2020, MIL3ID 2020, LABELS 2020 Vol. 12446 (eds Cardoso, J., et al.) 214–222 (Springer, 2020).
Singh, H. & Graber, M. L. Improving diagnosis in health care–the next imperative for patient safety. N. Engl. J. Med. 373, 2493–2495 (2015).
pubmed: 26559457
doi: 10.1056/NEJMp1512241
Erickson, L. A., Mete, O., Juhlin, C. C., Perren, A. & Gill, A. J. Overview of the 2022 WHO classification of parathyroid tumors. Endocr. Pathol. 33, 64–89 (2022).
pubmed: 35175514
doi: 10.1007/s12022-022-09709-1
Budd, S., Robinson, E. C. & Kainz, B. A survey on active learning and human-in-the-loop deep learning for medical image analysis. Med. Image Anal. 71, 102062 (2021).
pubmed: 33901992
doi: 10.1016/j.media.2021.102062
van der Wal, D. et al. Biological data annotation via a human-augmenting AI-based labeling system. npj Digit. Med. 4, 145 (2021).
pubmed: 34620993
pmcid: 8497580
doi: 10.1038/s41746-021-00520-6
Settles, B. Active Learning Literature Survey (University of Wisconsin-Madison Department of Computer Sciences, 2009); https://digital.library.wisc.edu/1793/60660
Go, H. Digital pathology and artificial intelligence applications in pathology. Brain Tumor Res. Treat. 10, 76–82 (2022).
pubmed: 35545826
pmcid: 9098984
doi: 10.14791/btrt.2021.0032
Wen, S. et al. Comparison of different classifiers with active learning to support quality control in nucleus segmentation in pathology images. AMIA Jt. Summits Transl. Sci. Proc. 2017, 227–236 (2018).
pubmed: 29888078
Hamilton, P. W. et al. Digital pathology and image analysis in tissue biomarker research. Methods 70, 59–73 (2014).
pubmed: 25034370
doi: 10.1016/j.ymeth.2014.06.015
Cheng, J. et al. Integrative analysis of histopathological images and genomic data predicts clear cell renal cell carcinoma prognosis. Cancer Res. 77, e91–e100 (2017).
pubmed: 29092949
pmcid: 7262576
doi: 10.1158/0008-5472.CAN-17-0313
McQueen, D. B., Perfetto, C. O., Hazard, F. K. & Lathi, R. B. Pregnancy outcomes in women with chronic endometritis and recurrent pregnancy loss. Fertil. Steril. 104, 927–931 (2015).
pubmed: 26207958
doi: 10.1016/j.fertnstert.2015.06.044
Ryan, E. et al. The menstrual cycle phase impacts the detection of plasma cells and the diagnosis of chronic endometritis in endometrial biopsy specimens. Fertil. Steril. 118, 787–794 (2022).
pubmed: 36182264
doi: 10.1016/j.fertnstert.2022.07.011
Kim, H. J. & Choi, G.-S. Clinical Implications of lymph node metastasis in colorectal cancer: current status and future perspectives. Ann. Coloproctol. 35, 109–117 (2019).
pubmed: 31288500
pmcid: 6625771
doi: 10.3393/ac.2019.06.12
Kiehl, L. et al. Deep learning can predict lymph node status directly from histology in colorectal cancer. Eur. J. Cancer 157, 464–473 (2021).
pubmed: 34649117
doi: 10.1016/j.ejca.2021.08.039
Khan, A. et al. Computer-assisted diagnosis of lymph node metastases in colorectal cancers using transfer learning with an ensemble model. Mod. Pathol. 36, 100118 (2023).
pubmed: 36805793
doi: 10.1016/j.modpat.2023.100118
Mescoli, C. et al. Isolated tumor cells in regional lymph nodes as relapse predictors in stage I and II colorectal cancer. J. Clin. Oncol. 30, 965–971 (2012).
pubmed: 22355061
doi: 10.1200/JCO.2011.35.9539
Tizhoosh, H. R. & Pantanowitz, L. Artificial intelligence and digital pathology: challenges and opportunities. J. Pathol. Inform. 9, 38 (2018).
pubmed: 30607305
pmcid: 6289004
doi: 10.4103/jpi.jpi_53_18
Baxi, V. et al. Association of artificial intelligence-powered and manual quantification of programmed death-ligand 1 (PD-L1) expression with outcomes in patients treated with nivolumab ± ipilimumab. Mod. Pathol. 35, 1529–1539 (2022).
pubmed: 35840720
pmcid: 9596372
doi: 10.1038/s41379-022-01119-2
Graham, S. et al. Screening of normal endoscopic large bowel biopsies with interpretable graph learning: a retrospective study. Gut 72, 1709–1721 (2023).
pubmed: 37173125
doi: 10.1136/gutjnl-2023-329512
Alemi Koohbanani, N., Jahanifar, M., Zamani Tajadin, N. & Rajpoot, N. NuClick: a deep learning framework for interactive segmentation of microscopic images. Med. Image Anal. 65, 101771 (2020).
pubmed: 32769053
doi: 10.1016/j.media.2020.101771
Schemmer, M., Kühl, N., Benz, C. & Satzger, G. On the influence of explainable AI on automation bias. Preprint at https://arxiv.org/abs/2204.08859 (2022).
Bond, R. R. et al. Automation bias in medicine: the influence of automated diagnoses on interpreter accuracy and uncertainty when reading electrocardiograms. J. Electrocardiol. 51, S6–S11 (2018).
pubmed: 30122457
doi: 10.1016/j.jelectrocard.2018.08.007
Parikh, R. B., Teeple, S. & Navathe, A. S. Addressing bias in artificial intelligence in health care. JAMA 322, 2377–2378 (2019).
pubmed: 31755905
doi: 10.1001/jama.2019.18058
Alon-Barkat, S. & Busuioc, M. Human–AI interactions in public sector decision making: ‘automation bias’ and ‘selective adherence’ to algorithmic advice. J. Public Adm. Res. Theory 33, 153–169 (2022).
doi: 10.1093/jopart/muac007
Schmidt, U., Weigert, M., Broaddus, C. & Myers, G. Cell detection with star-convex polygons. In Medical Image Computing and Computer Assisted Intervention—MICCAI 2018. Lecture Notes in Computer Science Vol. 11071 (eds Frangi, A. et al.) 265–273 (Springer, 2018).
Haralick, R. M., Shanmugam, K. & Dinstein, H. I. Textural features for image classification. IEEE Trans. Syst. Man Cybern. SMC-3, 610–621 (1973).
doi: 10.1109/TSMC.1973.4309314
Liu, Z. et al. A ConvNet for the 2020s. In 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) 11966–11976 (IEEE, 2022).
Chen, T. & Guestrin, C. XGBoost: a scalable tree boosting system. In Proc. 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining 785–794 (Association for Computing Machinery, 2016).
Mosqueira-Rey, E., Hernández-Pereira, E., Alonso-Ríos, D., Bobes-Bascarán, J. & Fernández-Leal, Á. Human-in-the-loop machine learning: a state of the art. Artif. Intell. Rev. 56, 3005–3054 (2023).
doi: 10.1007/s10462-022-10246-w
He, K., Zhang, X., Ren, S. & Sun, J. Deep residual learning for image recognition. In 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 770–778 (IEEE, 2016).
Deng, J. et al. ImageNet: a large-scale hierarchical image database. In 2009 IEEE Conference on Computer Vision and Pattern Recognition 248–255 (IEEE, 2009).
Li, W., Zhu, X. & Gong, S. Harmonious attention network for person re-identification. In Proc. IEEE Conference on Computer Vision and Pattern Recognition 2285–2294 (IEEE, 2018).
McHugh, M. L. Interrater reliability: the kappa statistic. Biochem. Med. 22, 276–282 (2012).
doi: 10.11613/BM.2012.031
Zou, K. H., Fielding, J. R., Silverman, S. G. & Tempany, C. M. C. Hypothesis testing I: proportions. Radiology 226, 609–613 (2003).
pubmed: 12601204
doi: 10.1148/radiol.2263011500