Critical Assessment of Metagenome Interpretation: the second round of challenges
Fernando Meyer(Helmholtz Centre for Infection Research), Adrian Fritz(German Center for Infection Research), Zhi-Luo Deng(Medizinische Hochschule Hannover), David Koslicki(Pennsylvania State University), Till Robin Lesker(German Center for Infection Research), Alexey Gurevich(St Petersburg University), Gary Robertson(Helmholtz Centre for Infection Research), Mohammed Alser(ETH Zurich), Dmitry Antipov(St Petersburg University), Francesco Beghini(University of Trento), Denis Bertrand(Genome Institute of Singapore), Jaqueline Brito(University of Southern California), C. Titus Brown(University of California, Davis), Jan P. Buchmann(Heinrich Heine University Düsseldorf), Aydın Buluç(Lawrence Berkeley National Laboratory), Bo Chen(Lawrence Berkeley National Laboratory), Rayan Chikhi(Institut Pasteur), Philip T. L. C. Clausen(Technical University of Denmark), Alexandru Cristian(Google (United States)), Piotr Wojciech Dąbrowski(HTW Berlin - University of Applied Sciences), Aaron E. Darling(University of Technology Sydney), Rob Egan(Lawrence Berkeley National Laboratory), Eleazar Eskin(University of California, Los Angeles), Evangelos Georganas(Intel (United States)), Eugene Goltsman(Lawrence Berkeley National Laboratory), Melissa A. Gray(Drexel University), Lars Hestbjerg Hansen(University of Copenhagen), Steven Hofmeyr(Lawrence Berkeley National Laboratory), Pingqin Huang(Fudan University), Luiz Irber(University of California, Davis), Huijue Jia(BGI Group (China)), Tue Sparholt Jørgensen(Aarhus University), Silas Kieser(University of Geneva), Terje Klemetsen(UiT The Arctic University of Norway), Axel Kola(Charité - Universitätsmedizin Berlin), Mikhail Kolmogorov(University of California San Diego), Anton Korobeynikov(St Petersburg University), Jason C. Kwan(University of Wisconsin–Madison), Nathan LaPierre(University of California, Los Angeles), Claire Lemaitre(Centre National de la Recherche Scientifique), Chenhao Li(Genome Institute of Singapore), Antoine Limasset(Centre National de la Recherche Scientifique), Fábio Malcher Miranda(Hasso Plattner Institute), Serghei Mangul(University of Southern California), Vanessa R. Marcelino(The University of Sydney), Camille Marchet(Centre National de la Recherche Scientifique), Pierre Marijon(Centre National de la Recherche Scientifique), Dmitry Meleshko(St Petersburg University), Daniel R. Mende(Amsterdam University Medical Centers), Alessio Milanese(SIB Swiss Institute of Bioinformatics), Niranjan Nagarajan(Agency for Science, Technology and Research), Jakob Nybo Nissen, Sergey Nurk(National Institutes of Health), Leonid Oliker(Lawrence Berkeley National Laboratory), Lucas Paoli(SIB Swiss Institute of Bioinformatics), Pierre Peterlongo(Centre National de la Recherche Scientifique), Vitor C. Piro(Hasso Plattner Institute), Jacob S. Porter(University of Virginia), Simon Rasmussen(University of Copenhagen), Evan Rees(University of Wisconsin–Madison), Knut Reinert(Freie Universität Berlin), Bernhard Y. Renard(Hasso Plattner Institute), Espen Mikal Robertsen(UiT The Arctic University of Norway), Gail Rosen(Center for Discovery), Hans‐Joachim Ruscheweyh(SIB Swiss Institute of Bioinformatics), Varuni Sarwal(University of California, Los Angeles), Nicola Segata(University of Trento), Enrico Seiler(Freie Universität Berlin), Lizhen Shi(Florida Polytechnic University), Fengzhu Sun(University of Southern California), Shinichi Sunagawa(SIB Swiss Institute of Bioinformatics), Søren J. Sørensen(University of Copenhagen), Ashleigh Thomas(University of British Columbia), Chengxuan Tong(Genome Institute of Singapore), Mirko Trajkovski(University of Geneva), Julien Tremblay(National Research Council Canada), Gherman Uritskiy(ID Genomics (United States)), Riccardo Vicedomini(Institut Pasteur), Zhengyang Wang(Lawrence Berkeley National Laboratory), Ziye Wang(Lawrence Berkeley National Laboratory), Zhong Wang(Lawrence Berkeley National Laboratory), Andrew Warren(University of Virginia), Nils Peder Willassen(UiT The Arctic University of Norway), Katherine Yelick(Lawrence Berkeley National Laboratory), Ronghui You(Fudan University), Georg Zeller, Zhengqiao Zhao(Drexel University), Shanfeng Zhu(Fudan University), Jie Zhu(BGI Group (China)), Rubén Garrido‐Oter(Max Planck Institute for Plant Breeding Research), Petra Gastmeier(Charité - Universitätsmedizin Berlin), Stéphane Hacquard(Max Planck Institute for Plant Breeding Research), Susanne Häußler(Helmholtz Centre for Infection Research), Ariane Khaledi(Helmholtz Centre for Infection Research), Friederike Maechler(Charité - Universitätsmedizin Berlin), Fantin Mesny(Max Planck Institute for Plant Breeding Research), Simona Radutoiu(Aarhus University), Paul Schulze‐Lefert(Max Planck Institute for Plant Breeding Research), Nathiana Smit(Helmholtz Centre for Infection Research), Till Strowig(Helmholtz Centre for Infection Research), Andreas Bremges(German Center for Infection Research), Alexander Sczyrba(Bielefeld University), Alice C. McHardy(Medizinische Hochschule Hannover)
Cited by 390Open Access
Abstract
Evaluating metagenomic software is key for optimizing metagenome interpretation and focus of the Initiative for the Critical Assessment of Metagenome Interpretation (CAMI). The CAMI II challenge engaged the community to assess methods on realistic and complex datasets with long- and short-read sequences, created computationally from around 1,700 new and known genomes, as well as 600 new plasmids and viruses. Here we analyze 5,002 results by 76 program versions. Substantial improvements were seen in assembly, some due to long-read data. Related strains still were challenging for assembly and genome recovery through binning, as was assembly quality for the latter. Profilers markedly matured, with taxon profilers and binners excelling at higher bacterial ranks, but underperforming for viruses and Archaea. Clinical pathogen detection results revealed a need to improve reproducibility. Runtime and memory usage analyses identified efficient programs, including top performers with other metrics. The results identify challenges and guide researchers in selecting methods for analyses.
Related Papers
No related papers found
Powered by citation graph analysis