John R. Murphy

Contextual word embedding models such as ELMo and BERT have dramatically improved performance for many natural language processing (NLP) tasks in recent months. However, these models have been minimally explored on specialty corpora, such as clinical text; moreover, in the clinical domain, no publicly-available pre-trained BERT models yet exist. In this work, we address this need by exploring and releasing BERT models for clinical text: one for generic clinical text and another for discharge summaries specifically. We demonstrate that using a domain-specific model yields performance improvements on 3/5 clinical NLP tasks, establishing a new state-of-the-art on the MedNLI dataset. We find that these domain-specific models are not as performant on 2 clinical de-identification tasks, and argue that this is a natural consequence of the differences between de-identified source text and synthetically non de-identified task text.

Contextual word embedding models such as ELMo (Peters et al., 2018) and BERT (Devlin et al., 2018) have dramatically improved performance for many natural language processing (NLP) tasks in recent months. However, these models have been minimally explored on specialty corpora, such as clinical text; moreover, in the clinical domain, no publicly-available pre-trained BERT models yet exist. In this work, we address this need by exploring and releasing BERT models for clinical text: one for generic clinical text and another for discharge summaries specifically. We demonstrate that using a domain-specific model yields performance improvements on three common clinical NLP tasks as compared to nonspecific embeddings. These domain-specific models are not as performant on two clinical de-identification tasks, and argue that this is a natural consequence of the differences between de-identified source text and synthetically non de-identified task text.

Abstract Previous studies have demonstrated that erythrocytes can be separated on the basis of specific gravity by centrifugation in various media. Horizontal centrifugation with swinging bucket rotors at 0 ° to 5 °C. was invariably used in these earlier studies. The use of discontinuous gradients with centrifugation has been more effective in separating cells by specific gravity, but is limited in the volume of cells that can be separated and is time consuming to prepare. The method described in this study utilizes the internal circulation in a tube centrifuged in an angle-head rotor at 30 ° C. at which the erythrocyte is more deformable as compared to 0 to 5 °C. Angle-head centrifugation at 30 °C. ± 2 ° for 1 hour resulted in separation of normal cells to the extent that the bottom 5 per cent of the cells had a mean corpuscular hemoglobin concentration (MCHC) of 124 per cent that of the top 5 per cent of the cells. In sickle cell anemia the MCHC of the bottom cells was 160 per cent that of the top cells. This improved method for separating erythrocytes on the basis of specific gravity, which has previously been shown to be related to cell age, will provide a means for more definitive observations of cell aging.

We have genetically replaced the diphtheria toxin receptor binding domain with a synthetic gene encoding interleukin-2 (IL-2) and a translational stop signal. The diphtheria toxin-related T-cell growth factor fusion gene encodes a 70 586-d polypeptide, pro-IL-2-toxin. The mature form of IL-2-toxin has a deduced mol. wt of 68,086 and is shown to be exported to the periplasmic compartment of Escherichia coli (pABI508), and contain immunologic determinants intrinsic to both its diphtheria toxin and IL-2 components. IL-2-toxin has been purified from periplasmic extracts of recombinant strains of E. coli (pABI508) by immunoaffinity chromatography using immobilized anti-IL-2. The purified chimeric toxin is shown to selectively inhibit protein synthesis in IL-2 receptor bearing targeted cells, whereas cell lines which do not express the IL-2 receptor are resistant to IL-2-toxin action.

The IL-2 toxin-mediated inhibition of protein synthesis in high affinity IL-2-R-positive murine and human T cell lines has been examined. Both excess free IL-2 and mAb to the Tac epitope of the p55 subunit of IL-2-R are shown to block the action of IL-2 toxin; whereas, agents that interact with other receptors or antigens on the T cell surface have no effect. We show that IL-2 toxin, like diphtheria toxin, must pass through an acidic vesicle in order to intoxicate target T cells. Finally, we demonstrate that the IL-2 toxin-mediated inhibition of protein synthesis in both human and murine T cells that bear the high affinity IL-2-R is due to the classic diphtheria toxin fragment A-catalyzed ADP ribosylation of elongation factor 2.