Ross Taylor

In this work, we develop and release Llama 2, a collection of pretrained and fine-tuned large language models (LLMs) ranging in scale from 7 billion to 70 billion parameters. Our fine-tuned LLMs, called Llama 2-Chat, are optimized for dialogue use cases. Our models outperform open-source chat models on most benchmarks we tested, and based on our human evaluations for helpfulness and safety, may be a suitable substitute for closed-source models. We provide a detailed description of our approach to fine-tuning and safety improvements of Llama 2-Chat in order to enable the community to build on our work and contribute to the responsible development of LLMs.

Information overload is a major obstacle to scientific progress. The explosive growth in scientific literature and data has made it ever harder to discover useful insights in a large mass of information. Today scientific knowledge is accessed through search engines, but they are unable to organize scientific knowledge alone. In this paper we introduce Galactica: a large language model that can store, combine and reason about scientific knowledge. We train on a large scientific corpus of papers, reference material, knowledge bases and many other sources. We outperform existing models on a range of scientific tasks. On technical knowledge probes such as LaTeX equations, Galactica outperforms the latest GPT-3 by 68.2% versus 49.0%. Galactica also performs well on reasoning, outperforming Chinchilla on mathematical MMLU by 41.3% to 35.7%, and PaLM 540B on MATH with a score of 20.4% versus 8.8%. It also sets a new state-of-the-art on downstream tasks such as PubMedQA and MedMCQA dev of 77.6% and 52.9%. And despite not being trained on a general corpus, Galactica outperforms BLOOM and OPT-175B on BIG-bench. We believe these results demonstrate the potential for language models as a new interface for science. We open source the model for the benefit of the scientific community.

Acromegaly was diagnosed in 14 middle-aged to old cats of mixed breeding. Thirteen (93%) of the cats were male and one was female. The earliest clinical signs in the 14 cats included polyuria, polydipsia, polyphagia, all of which were associated with untreated diabetes mellitus. All developed severe insulin resistance within a few months; peak insulin dosages required to control severe hyperglycemia ranged from 20 to 130 U per day. Other clinical findings weeks to months after diagnosis included enlargement of one or more organs (e.g., liver, heart, kidneys, and tongue) (n = 14), cardiomyopathy (n = 13), increase in body size and weight gain (n = 8), nephropathy associated with azotemia and clinical signs of renal failure (n = 7), degenerative arthropathy (n = 6), and central nervous system signs (i.e., circling and seizures) caused by enlargement of the pituitary tumor (n = 2). The diagnosis of acromegaly was confirmed by demonstration of extremely high basal serum growth hormone concentrations (22 to 131 micrograms/l) in all cats. Computerized tomography disclosed a mass in the region of the pituitary gland and hypothalamus in five of the six cats in which it was performed. Two cats were treated by cobalt radiotherapy followed by administration of a somatostatin analogue (octreotide), whereas two cats were treated with octreotide alone. Treatment had little to no effect in decreasing serum GH concentrations in any of the cats. Eleven of the 14 cats were euthanized or died four to 42 months (median survival time, 20.5 months) after the onset of acromegaly because of renal failure (n = 2), congestive heart failure (n = 1), concomitant renal failure and congestive heart failure (n = 3), progressive neurologic signs (n = 2), persistent anorexia and lethargy of unknown cause (n = 1), the owner's unwillingness to treat the diabetes mellitus (n = 1), or unknown causes (n = 1). Results of necropsy examination in ten cats revealed a large pituitary acidophil adenoma (n = 10), marked left ventricular and septal hypertrophy (n = 7), dilated cardiomyopathy (n = 1), arthropathy affecting the shoulder, elbow, or stifle (n = 5), and glomerulopathy characterized by expansion of the mesangial matrix and variable periglomerular fibrosis (n = 10).

The btuB gene product from Escherichia coli is a 66.5-kDa integral outer membrane protein required for high-affinity uptake of cyanocobalamin and the translocation of E group colicins and colicin A. Efficient purification of overexpressed BtuB containing stoichiometric levels of bound lipopolysaccharide has been achieved through the extraction of the outer membrane with nonionic detergent followed by ion-exchange chromatography. Analysis of far UV circular dichroism spectra indicates a predominantly beta-sheet secondary structure (76 +/- 4%) with a low alpha-helical content (15 +/- 3%), providing the first direct evidence for secondary structure models derived from sequence and hydropathy analysis. Characterization of the octylglucoside-solubilized receptor by sedimentation equilibrium and sedimentation velocity analysis reveals a monodisperse protein-detergent complex of approximately 89 kDa with a sedimentation coefficient of 4.7 S which, after correction for bound detergent, indicates that BtuB is purified as a monomer. BtuB binds vitamin B12 with a stoichiometry of approximately 1:1, as observed by a shift in the sedimentation profile of the vitamin to the much faster velocity observed for the protein-detergent complex. The preincubation of colicin E3 with stoichiometric levels of BtuB protects susceptible strains from the lethal effects of the colicin and results in a complex with a sedimentation coefficient appropriate for a BtuB-detergent-colicin E3 complex, demonstrating that monomeric BtuB retains high affinity for this particular ligand after isolation.