Thibaud Coroller

Abstract Purpose: Tumors are continuously evolving biological systems, and medical imaging is uniquely positioned to monitor changes throughout treatment. Although qualitatively tracking lesions over space and time may be trivial, the development of clinically relevant, automated radiomics methods that incorporate serial imaging data is far more challenging. In this study, we evaluated deep learning networks for predicting clinical outcomes through analyzing time series CT images of patients with locally advanced non–small cell lung cancer (NSCLC). Experimental Design: Dataset A consists of 179 patients with stage III NSCLC treated with definitive chemoradiation, with pretreatment and posttreatment CT images at 1, 3, and 6 months follow-up (581 scans). Models were developed using transfer learning of convolutional neural networks (CNN) with recurrent neural networks (RNN), using single seed-point tumor localization. Pathologic response validation was performed on dataset B, comprising 89 patients with NSCLC treated with chemoradiation and surgery (178 scans). Results: Deep learning models using time series scans were significantly predictive of survival and cancer-specific outcomes (progression, distant metastases, and local-regional recurrence). Model performance was enhanced with each additional follow-up scan into the CNN model (e.g., 2-year overall survival: AUC = 0.74, P < 0.05). The models stratified patients into low and high mortality risk groups, which were significantly associated with overall survival [HR = 6.16; 95% confidence interval (CI), 2.17–17.44; P < 0.001]. The model also significantly predicted pathologic response in dataset B (P = 0.016). Conclusions: We demonstrate that deep learning can integrate imaging scans at multiple timepoints to improve clinical outcome predictions. AI-based noninvasive radiomics biomarkers can have a significant impact in the clinic given their low cost and minimal requirements for human input.

BACKGROUND: Non-small-cell lung cancer (NSCLC) patients often demonstrate varying clinical courses and outcomes, even within the same tumor stage. This study explores deep learning applications in medical imaging allowing for the automated quantification of radiographic characteristics and potentially improving patient stratification. METHODS AND FINDINGS: We performed an integrative analysis on 7 independent datasets across 5 institutions totaling 1,194 NSCLC patients (age median = 68.3 years [range 32.5-93.3], survival median = 1.7 years [range 0.0-11.7]). Using external validation in computed tomography (CT) data, we identified prognostic signatures using a 3D convolutional neural network (CNN) for patients treated with radiotherapy (n = 771, age median = 68.0 years [range 32.5-93.3], survival median = 1.3 years [range 0.0-11.7]). We then employed a transfer learning approach to achieve the same for surgery patients (n = 391, age median = 69.1 years [range 37.2-88.0], survival median = 3.1 years [range 0.0-8.8]). We found that the CNN predictions were significantly associated with 2-year overall survival from the start of respective treatment for radiotherapy (area under the receiver operating characteristic curve [AUC] = 0.70 [95% CI 0.63-0.78], p < 0.001) and surgery (AUC = 0.71 [95% CI 0.60-0.82], p < 0.001) patients. The CNN was also able to significantly stratify patients into low and high mortality risk groups in both the radiotherapy (p < 0.001) and surgery (p = 0.03) datasets. Additionally, the CNN was found to significantly outperform random forest models built on clinical parameters-including age, sex, and tumor node metastasis stage-as well as demonstrate high robustness against test-retest (intraclass correlation coefficient = 0.91) and inter-reader (Spearman's rank-order correlation = 0.88) variations. To gain a better understanding of the characteristics captured by the CNN, we identified regions with the most contribution towards predictions and highlighted the importance of tumor-surrounding tissue in patient stratification. We also present preliminary findings on the biological basis of the captured phenotypes as being linked to cell cycle and transcriptional processes. Limitations include the retrospective nature of this study as well as the opaque black box nature of deep learning networks. CONCLUSIONS: Our results provide evidence that deep learning networks may be used for mortality risk stratification based on standard-of-care CT images from NSCLC patients. This evidence motivates future research into better deciphering the clinical and biological basis of deep learning networks as well as validation in prospective data.

Abstract Tumors are characterized by somatic mutations that drive biological processes ultimately reflected in tumor phenotype. With regard to radiographic phenotypes, generally unconnected through present understanding to the presence of specific mutations, artificial intelligence methods can automatically quantify phenotypic characters by using predefined, engineered algorithms or automatic deep-learning methods, a process also known as radiomics. Here we demonstrate how imaging phenotypes can be connected to somatic mutations through an integrated analysis of independent datasets of 763 lung adenocarcinoma patients with somatic mutation testing and engineered CT image analytics. We developed radiomic signatures capable of distinguishing between tumor genotypes in a discovery cohort (n = 353) and verified them in an independent validation cohort (n = 352). All radiomic signatures significantly outperformed conventional radiographic predictors (tumor volume and maximum diameter). We found a radiomic signature related to radiographic heterogeneity that successfully discriminated between EGFR+ and EGFR− cases (AUC = 0.69). Combining this signature with a clinical model of EGFR status (AUC = 0.70) significantly improved prediction accuracy (AUC = 0.75). The highest performing signature was capable of distinguishing between EGFR+ and KRAS+ tumors (AUC = 0.80) and, when combined with a clinical model (AUC = 0.81), substantially improved its performance (AUC = 0.86). A KRAS+/KRAS− radiomic signature also showed significant albeit lower performance (AUC = 0.63) and did not improve the accuracy of a clinical predictor of KRAS status. Our results argue that somatic mutations drive distinct radiographic phenotypes that can be predicted by radiomics. This work has implications for the use of imaging-based biomarkers in the clinic, as applied noninvasively, repeatedly, and at low cost. Cancer Res; 77(14); 3922–30. ©2017 AACR.