A deep-learning framework to predict cancer treatment response from histopathology images through imputed transcriptomics.

Journal

Nature cancer

ISSN: 2662-1347

Titre abrégé: Nat Cancer

Pays: England

ID NLM: 101761119

Informations de publication

Date de publication:
03 Jul 2024

Historique:

received: 10 08 2023

accepted: 06 06 2024

medline: 4 7 2024

pubmed: 4 7 2024

entrez: 3 7 2024

Statut: aheadofprint

Résumé

Advances in artificial intelligence have paved the way for leveraging hematoxylin and eosin-stained tumor slides for precision oncology. We present ENLIGHT-DeepPT, an indirect two-step approach consisting of (1) DeepPT, a deep-learning framework that predicts genome-wide tumor mRNA expression from slides, and (2) ENLIGHT, which predicts response to targeted and immune therapies from the inferred expression values. We show that DeepPT successfully predicts transcriptomics in all 16 The Cancer Genome Atlas cohorts tested and generalizes well to two independent datasets. ENLIGHT-DeepPT successfully predicts true responders in five independent patient cohorts involving four different treatments spanning six cancer types, with an overall odds ratio of 2.28 and a 39.5% increased response rate among predicted responders versus the baseline rate. Notably, its prediction accuracy, obtained without any training on the treatment data, is comparable to that achieved by directly predicting the response from the images, which requires specific training on the treatment evaluation cohorts.

Identifiants

DOI: 10.1038/s43018-024-00793-2 PMID: 38961276

pubmed: 38961276

doi: 10.1038/s43018-024-00793-2

pii: 10.1038/s43018-024-00793-2

doi:

Types de publication

Journal Article

Langues

eng

Sous-ensembles de citation

Subventions

Organisme : Centre of Excellence for Electromaterials Science, Australian Research Council (ARC Centre of Excellence for Electromaterials Science)

ID : DP190103402

Organisme : Centre of Excellence for Electromaterials Science, Australian Research Council (ARC Centre of Excellence for Electromaterials Science)

ID : DP190103402

Informations de copyright

Références

Golub, T. R. et al. Molecular classification of cancer: class discovery and class prediction by gene expression monitoring. Science 286, 531–537 (1999).

pubmed: 10521349 doi: 10.1126/science.286.5439.531

Curtis, C. et al. The genomic and transcriptomic architecture of 2,000 breast tumours reveals novel subgroups. Nature 486, 346–352 (2012).

pubmed: 22522925 pmcid: 3440846 doi: 10.1038/nature10983

Doroshow, D. B. & Doroshow, J. H. Genomics and the history of precision oncology. Surg. Oncol. Clin. N. Am. 29, 35–49 (2020).

pubmed: 31757312 doi: 10.1016/j.soc.2019.08.003

Rosenthal, J. et al. Building tools for machine learning and artificial intelligence in cancer research: best practices and a case study with the PathML toolkit for computational pathology. Mol. Cancer Res. 20, 202–206 (2022).

pubmed: 34880124 doi: 10.1158/1541-7786.MCR-21-0665

Ström, P. et al. Artificial intelligence for diagnosis and grading of prostate cancer in biopsies: a population-based, diagnostic study. Lancet Oncol. 21, 222–232 (2020).

pubmed: 31926806 doi: 10.1016/S1470-2045(19)30738-7

Fu, Y. et al. Pan-cancer computational histopathology reveals mutations, tumor composition and prognosis. Nat. Cancer 1, 800–810 (2020).

pubmed: 35122049 doi: 10.1038/s43018-020-0085-8

Noorbakhsh, J. et al. Deep learning-based cross-classifications reveal conserved spatial behaviors within tumor histological images. Nat. Commun. 11, 6367 (2020).

pubmed: 33311458 pmcid: 7733499 doi: 10.1038/s41467-020-20030-5

Kim, J. et al. Unsupervised discovery of tissue architecture in multiplexed imaging. Nat. Methods 19, 1653–1661 (2022).

pubmed: 36316562 pmcid: 11102857 doi: 10.1038/s41592-022-01657-2

Coudray, N. et al. Classification and mutation prediction from non-small cell lung cancer histopathology images using deep learning. Nat. Med. 24, 1559–1567 (2018).

pubmed: 30224757 pmcid: 9847512 doi: 10.1038/s41591-018-0177-5

Swiderska-Chadaj, Z. et al. Learning to detect lymphocytes in immunohistochemistry with deep learning. Med. Image Anal. 58, 101547 (2019).

pubmed: 31476576 doi: 10.1016/j.media.2019.101547

Yu, K.-H. et al. Classifying non-small cell lung cancer types and transcriptomic subtypes using convolutional neural networks. J. Am. Med. Inform. Assoc. 27, 757–769 (2020).

pubmed: 32364237 pmcid: 7309263 doi: 10.1093/jamia/ocz230

Couture, H. D. et al. Image analysis with deep learning to predict breast cancer grade, ER status, histologic subtype, and intrinsic subtype. NPJ Breast Cancer 4, 30 (2018).

pubmed: 30182055 pmcid: 6120869 doi: 10.1038/s41523-018-0079-1

Campanella, G. et al. Clinical-grade computational pathology using weakly supervised deep learning on whole slide images. Nat. Med. 25, 1301–1309 (2019).

pubmed: 31308507 pmcid: 7418463 doi: 10.1038/s41591-019-0508-1

Xu, H. et al. Spatial heterogeneity and organization of tumor mutation burden with immune infiltrates within tumors based on whole slide images correlated with patient survival in bladder cancer. J. Pathol. Inform. 13, 100105 (2022).

pubmed: 36268064 pmcid: 9577053 doi: 10.1016/j.jpi.2022.100105

Kather, J. N. et al. Pan-cancer image-based detection of clinically actionable genetic alterations. Nat. Cancer 1, 789–799 (2020).

pubmed: 33763651 pmcid: 7610412 doi: 10.1038/s43018-020-0087-6

Qu, H. et al. Genetic mutation and biological pathway prediction based on whole slide images in breast carcinoma using deep learning. NPJ Precis. Oncol. 5, 87 (2021).

pubmed: 34556802 pmcid: 8460699 doi: 10.1038/s41698-021-00225-9

Schaumberg, A. J., Rubin, M. A. & Fuchs, T. J. H&E-stained whole slide image deep learning predicts SPOP mutation state in prostate cancer. Preprint at bioRxiv https://doi.org/10.1101/064279 (2018).

Tsou, P. & Wu, C.-J. Mapping driver mutations to histopathological subtypes in papillary thyroid carcinoma: applying a deep convolutional neural network. J. Clin. Med. 8, 1675 (2019).

pubmed: 31614962 pmcid: 6832421 doi: 10.3390/jcm8101675

Chang, P. et al. Deep-learning convolutional neural networks accurately classify genetic mutations in gliomas. AJNR Am. J. Neuroradiol. 39, 1201–1207 (2018).

pubmed: 29748206 pmcid: 6880932 doi: 10.3174/ajnr.A5667

Kather, J. N. et al. Deep learning can predict microsatellite instability directly from histology in gastrointestinal cancer. Nat. Med. 25, 1054–1056 (2019).

pubmed: 31160815 pmcid: 7423299 doi: 10.1038/s41591-019-0462-y

Kim, R. H. et al. A deep learning approach for rapid mutational screening in melanoma. Preprint at bioRxiv https://doi.org/10.1101/610311 (2019).

Chen, M. et al. Classification and mutation prediction based on histopathology H&E images in liver cancer using deep learning. NPJ Precis. Oncol. 4, 14 (2020).

pubmed: 32550270 pmcid: 7280520 doi: 10.1038/s41698-020-0120-3

Ghaffari Laleh, N. et al. Benchmarking weakly-supervised deep learning pipelines for whole slide classification in computational pathology. Med. Image Anal. 79, 102474 (2022).

pubmed: 35588568 doi: 10.1016/j.media.2022.102474

Mobadersany, P. et al. Predicting cancer outcomes from histology and genomics using convolutional networks. Proc. Natl Acad. Sci. USA 115, E2970–E2979 (2018).

pubmed: 29531073 pmcid: 5879673 doi: 10.1073/pnas.1717139115

Cheng, J. et al. Integrative analysis of histopathological images and genomic data predicts clear cell renal cell carcinoma prognosis. Cancer Res. 77, e91–e100 (2017).

pubmed: 29092949 pmcid: 7262576 doi: 10.1158/0008-5472.CAN-17-0313

Beck, A. H. et al. Systematic analysis of breast cancer morphology uncovers stromal features associated with survival. Sci. Transl. Med. 3, 108ra113 (2011).

pubmed: 22072638 doi: 10.1126/scitranslmed.3002564

Bulten, W. et al. Automated deep-learning system for Gleason grading of prostate cancer using biopsies: a diagnostic study. Lancet Oncol. 21, 233–241 (2020).

pubmed: 31926805 doi: 10.1016/S1470-2045(19)30739-9

Courtiol, P. et al. Deep learning-based classification of mesothelioma improves prediction of patient outcome. Nat. Med. 25, 1519–1525 (2019).

pubmed: 31591589 doi: 10.1038/s41591-019-0583-3

Boehm, K. M. et al. Multimodal data integration using machine learning improves risk stratification of high-grade serous ovarian cancer. Nat. Cancer 3, 723–733 (2022).

pubmed: 35764743 pmcid: 9239907 doi: 10.1038/s43018-022-00388-9

Chen, Y. et al. Computational pathology improves risk stratification of a multi-gene assay for early stage ER+ breast cancer. NPJ Breast Cancer 9, 40 (2023).

pubmed: 37198173 pmcid: 10192429 doi: 10.1038/s41523-023-00545-y

Zheng, H., Momeni, A., Cedoz, P.-L., Vogel, H. & Gevaert, O. Whole slide images reflect DNA methylation patterns of human tumors. NPJ Genom. Med. 5, 11 (2020).

pubmed: 32194984 pmcid: 7064513 doi: 10.1038/s41525-020-0120-9

Wang, H. et al. Mitosis detection in breast cancer pathology images by combining handcrafted and convolutional neural network features. J. Med. Imaging (Bellingham) 1, 034003 (2014).

pubmed: 26158062 doi: 10.1117/1.JMI.1.3.034003

Turkki, R., Linder, N., Kovanen, P. E., Pellinen, T. & Lundin, J. Antibody-supervised deep learning for quantification of tumor-infiltrating immune cells in hematoxylin and eosin stained breast cancer samples. J. Pathol. Inform. 7, 38 (2016).

pubmed: 27688929 pmcid: 5027738 doi: 10.4103/2153-3539.189703

Saltz, J. et al. Spatial organization and molecular correlation of tumor-infiltrating lymphocytes using deep learning on pathology images. Cell Rep. 23, 181–193 (2018).

pubmed: 29617659 pmcid: 5943714 doi: 10.1016/j.celrep.2018.03.086

Page, D. B. et al. Spatial analyses of immune cell infiltration in cancer: current methods and future directions: a report of the International Immuno-Oncology Biomarker Working Group on Breast Cancer. J. Pathol. 260, 514–532 (2023).

pubmed: 37608771 doi: 10.1002/path.6165

Wang, X. et al. Spatial interplay patterns of cancer nuclei and tumor-infiltrating lymphocytes (TILs) predict clinical benefit for immune checkpoint inhibitors. Sci. Adv. 8, eabn3966 (2022).

pubmed: 35648850 pmcid: 9159577 doi: 10.1126/sciadv.abn3966

Hu, J. et al. Using deep learning to predict anti-PD-1 response in melanoma and lung cancer patients from histopathology images. Transl. Oncol. 14, 100921 (2021).

pubmed: 33129113 doi: 10.1016/j.tranon.2020.100921

Johannet, P. et al. Using machine learning algorithms to predict immunotherapy response in patients with advanced melanoma. Clin. Cancer Res. 27, 131–140 (2021).

pubmed: 33208341 doi: 10.1158/1078-0432.CCR-20-2415

Zhang, F. et al. Predicting treatment response to neoadjuvant chemoradiotherapy in local advanced rectal cancer by biopsy digital pathology image features. Clin. Transl. Med. 10, e110 (2020).

pubmed: 32594660 pmcid: 7403709 doi: 10.1002/ctm2.110

Honomichl, N. HER2 and trastuzumab treatment response H&E slides with tumor ROI annotations (HER2 tumor ROIs). The Cancer Imaging Archive https://doi.org/10.7937/E65C-AM96 (2022).

He, B. et al. Integrating spatial gene expression and breast tumour morphology via deep learning. Nat. Biomed. Eng. 4, 827–834 (2020).

pubmed: 32572199 doi: 10.1038/s41551-020-0578-x

Wang, Y. et al. Predicting molecular phenotypes from histopathology images: a transcriptome-wide expression–morphology analysis in breast cancer. Cancer Res. 81, 5115–5126 (2021).

pubmed: 34341074 pmcid: 9397635 doi: 10.1158/0008-5472.CAN-21-0482

Monjo, T., Koido, M., Nagasawa, S., Suzuki, Y. & Kamatani, Y. Efficient prediction of a spatial transcriptomics profile better characterizes breast cancer tissue sections without costly experimentation. Sci. Rep. 12, 4133 (2022).

pubmed: 35260632 pmcid: 8904587 doi: 10.1038/s41598-022-07685-4

Schmauch, B. et al. A deep learning model to predict RNA-Seq expression of tumours from whole slide images. Nat. Commun. 11, 3877 (2020).

pubmed: 32747659 pmcid: 7400514 doi: 10.1038/s41467-020-17678-4

Levy-Jurgenson, A., Tekpli, X., Kristensen, V. N. & Yakhini, Z. Spatial transcriptomics inferred from pathology whole-slide images links tumor heterogeneity to survival in breast and lung cancer. Sci. Rep. 10, 18802 (2020).

pubmed: 33139755 pmcid: 7606448 doi: 10.1038/s41598-020-75708-z

Alsaafin, A., Safarpoor, A., Sikaroudi, M., Hipp, J. D. & Tizhoosh, H. R. Learning to predict RNA sequence expressions from whole slide images with applications for search and classification. Commun. Biol. 6, 304 (2023).

pubmed: 36949169 pmcid: 10033650 doi: 10.1038/s42003-023-04583-x

Dinstag, G. et al. Clinically oriented prediction of patient response to targeted and immunotherapies from the tumor transcriptome. Med 4, 15–30 (2023).

pubmed: 36513065 doi: 10.1016/j.medj.2022.11.001

Sammut, S.-J. et al. Multi-omic machine learning predictor of breast cancer therapy response. Nature 601, 623–629 (2022).

pubmed: 34875674 doi: 10.1038/s41586-021-04278-5

Zheng, Y. et al. Digital profiling of cancer transcriptomes from histology images with grouped vision attention. Preprint at bioRxiv https://doi.org/10.1101/2023.09.28.560068 (2023).

Hanahan, D. & Weinberg, R. A. Hallmarks of cancer: the next generation. Cell 144, 646–674 (2011).

pubmed: 21376230 doi: 10.1016/j.cell.2011.02.013

Iorio, F. et al. Pathway-based dissection of the genomic heterogeneity of cancer hallmarks’ acquisition with SLAPenrich. Sci. Rep. 8, 6713 (2018).

pubmed: 29713020 pmcid: 5928049 doi: 10.1038/s41598-018-25076-6

Subramanian, A. et al. Gene set enrichment analysis: a knowledge-based approach for interpreting genome-wide expression profiles. Proc. Natl Acad. Sci. USA 102, 15545–15550 (2005).

pubmed: 16199517 pmcid: 1239896 doi: 10.1073/pnas.0506580102

Richardsen, E. et al. Evaluation of the proliferation marker Ki-67 in a large prostatectomy cohort. PLoS ONE 12, e0186852 (2017).

pubmed: 29141018 pmcid: 5687762 doi: 10.1371/journal.pone.0186852

Whitfield, M. L., George, L. K., Grant, G. D. & Perou, C. M. Common markers of proliferation. Nat. Rev. Cancer 6, 99–106 (2006).

pubmed: 16491069 doi: 10.1038/nrc1802

Liberzon, A. et al. The Molecular Signatures Database (MSigDB) hallmark gene set collection. Cell Syst. 1, 417–425 (2015).

pubmed: 26771021 pmcid: 4707969 doi: 10.1016/j.cels.2015.12.004

Farahmand, S. et al. Deep learning trained on hematoxylin and eosin tumor region of interest predicts HER2 status and trastuzumab treatment response in HER2+ breast cancer. Mod. Pathol. 35, 44–51 (2022).

pubmed: 34493825 doi: 10.1038/s41379-021-00911-w

Strauss, J. et al. Bintrafusp alfa, a bifunctional fusion protein targeting TGF-β and PD-L1, in patients with human papillomavirus-associated malignancies. J. Immunother. Cancer 8, e001395 (2020).

pubmed: 33323462 pmcid: 7745517 doi: 10.1136/jitc-2020-001395

Kanopoulos, N., Vasanthavada, N. & Baker, R. L. Design of an image edge detection filter using the Sobel operator. IEEE J. Solid-State Circuits 23, 358–367 (1988).

doi: 10.1109/4.996

Macenko, M. et al. A method for normalizing histology slides for quantitative analysis. in 2009 IEEE International Symposium on Biomedical Imaging: From Nano to Macro 1107–1110 (IEEE, 2009); https://doi.org/10.1109/ISBI.2009.5193250

He, K., Zhang, X., Ren, S. & Sun, J. Deep residual learning for image recognition. in 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 770–778 (IEEE, 2016); https://doi.org/10.1109/cvpr.2016.90

Hoang, D.-T. et al. DeepPT: a deep learning model for predicting transcriptomics from histopathology images. Zenodo https://doi.org/10.5281/zenodo.11125591 (2024).