Jianhua Yao

Remarkable progress has been made in image recognition, primarily due to the availability of large-scale annotated datasets and deep convolutional neural networks (CNNs). CNNs enable learning data-driven, highly representative, hierarchical image features from sufficient training data. However, obtaining datasets as comprehensively annotated as ImageNet in the medical imaging domain remains a challenge. There are currently three major techniques that successfully employ CNNs to medical image classification: training the CNN from scratch, using off-the-shelf pre-trained CNN features, and conducting unsupervised CNN pre-training with supervised fine-tuning. Another effective method is transfer learning, i.e., fine-tuning CNN models pre-trained from natural image dataset to medical image tasks. In this paper, we exploit three important, but previously understudied factors of employing deep convolutional neural networks to computer-aided detection problems. We first explore and evaluate different CNN architectures. The studied models contain 5 thousand to 160 million parameters, and vary in numbers of layers. We then evaluate the influence of dataset scale and spatial image context on performance. Finally, we examine when and why transfer learning from pre-trained ImageNet (via fine-tuning) can be useful. We study two specific computer-aided detection (CADe) problems, namely thoraco-abdominal lymph node (LN) detection and interstitial lung disease (ILD) classification. We achieve the state-of-the-art performance on the mediastinal LN detection, and report the first five-fold cross-validation classification results on predicting axial CT slices with ILD categories. Our extensive empirical evaluation, CNN model analysis and valuable insights can be extended to the design of high performance CAD systems for other medical imaging tasks.

Automated computer-aided detection (CADe) has been an important tool in clinical practice and research. State-of-the-art methods often show high sensitivities at the cost of high false-positives (FP) per patient rates. We design a two-tiered coarse-to-fine cascade framework that first operates a candidate generation system at sensitivities ∼ 100% of but at high FP levels. By leveraging existing CADe systems, coordinates of regions or volumes of interest (ROI or VOI) are generated and function as input for a second tier, which is our focus in this study. In this second stage, we generate 2D (two-dimensional) or 2.5D views via sampling through scale transformations, random translations and rotations. These random views are used to train deep convolutional neural network (ConvNet) classifiers. In testing, the ConvNets assign class (e.g., lesion, pathology) probabilities for a new set of random views that are then averaged to compute a final per-candidate classification probability. This second tier behaves as a highly selective process to reject difficult false positives while preserving high sensitivities. The methods are evaluated on three data sets: 59 patients for sclerotic metastasis detection, 176 patients for lymph node detection, and 1,186 patients for colonic polyp detection. Experimental results show the ability of ConvNets to generalize well to different medical imaging CADe applications and scale elegantly to various data sets. Our proposed methods improve performance markedly in all cases. Sensitivities improved from 57% to 70%, 43% to 77%, and 58% to 75% at 3 FPs per patient for sclerotic metastases, lymph nodes and colonic polyps, respectively.

BACKGROUND: Genetic defects in telomere maintenance and repair cause bone marrow failure, liver cirrhosis, and pulmonary fibrosis, and they increase susceptibility to cancer. Historically, androgens have been useful as treatment for marrow failure syndromes. In tissue culture and animal models, sex hormones regulate expression of the telomerase gene. METHODS: In a phase 1-2 prospective study involving patients with telomere diseases, we administered the synthetic sex hormone danazol orally at a dose of 800 mg per day for a total of 24 months. The goal of treatment was the attenuation of accelerated telomere attrition, and the primary efficacy end point was a 20% reduction in the annual rate of telomere attrition measured at 24 months. The occurrence of toxic effects of treatment was the primary safety end point. Hematologic response to treatment at various time points was the secondary efficacy end point. RESULTS: After 27 patients were enrolled, the study was halted early, because telomere attrition was reduced in all 12 patients who could be evaluated for the primary end point; in the intention-to-treat analysis, 12 of 27 patients (44%; 95% confidence interval [CI], 26 to 64) met the primary efficacy end point. Unexpectedly, almost all the patients (11 of 12, 92%) had a gain in telomere length at 24 months as compared with baseline (mean increase, 386 bp [95% CI, 178 to 593]); in exploratory analyses, similar increases were observed at 6 months (16 of 21 patients; mean increase, 175 bp [95% CI, 79 to 271]) and 12 months (16 of 18 patients; mean increase, 360 bp [95% CI, 209 to 512]). Hematologic responses occurred in 19 of 24 patients (79%) who could be evaluated at 3 months and in 10 of 12 patients (83%) who could be evaluated at 24 months. Known adverse effects of danazol--elevated liver-enzyme levels and muscle cramps--of grade 2 or less occurred in 41% and 33% of the patients, respectively. CONCLUSIONS: In our study, treatment with danazol led to telomere elongation in patients with telomere diseases. (Funded by the National Institutes of Health; ClinicalTrials.gov number, NCT01441037.).

Despite the recent advances in automatically describing image contents, their applications have been mostly limited to image caption datasets containing natural images (e.g., Flickr 30k, MSCOCO). In this paper, we present a deep learning model to efficiently detect a disease from an image and annotate its contexts (e.g., location, severity and the affected organs). We employ a publicly available radiology dataset of chest x-rays and their reports, and use its image annotations to mine disease names to train convolutional neural networks (CNNs). In doing so, we adopt various regularization techniques to circumvent the large normalvs-diseased cases bias. Recurrent neural networks (RNNs) are then trained to describe the contexts of a detected disease, based on the deep CNN features. Moreover, we introduce a novel approach to use the weights of the already trained pair of CNN/RNN on the domain-specific image/text dataset, to infer the joint image/text contexts for composite image labeling. Significantly improved image annotation results are demonstrated using the recurrent neural cascade model by taking the joint image/text contexts into account.