Paul Whaley

[First paragraph] A clearly-framed question creates the structure and delineates the approach to defining research objectives, conducting systematic reviews and developing health guidance (Guyatt et al., 2011; Armstrong et al., 2007). To assess the association between exposures and outcomes, including in the field of nutrition, environmental and occupational health, the concept of defining the Population (including animal species), Exposure, Comparator, and Outcomes (PECO) as pillars of the question is increasingly accepted (Morgan et al., 2016; Morgan et al., n.d.). Thus, the PECO defines the objectives of the review or guideline. Furthermore, the PECO informs the study design or inclusion and exclusion criteria for a review, as well as facilitating the interpretation of the directness of the findings based on how well the actual research findings represent the original question.

Reliable synthesis of the various rapidly expanding bodies of evidence is vital for the process of evidence-informed decision-making in environmental policy, practice and research. With the rise of evidence-base medicine and increasing numbers of published systematic reviews, criteria for assessing the quality of reporting have been developed. First QUOROM (Lancet 354:1896–1900, 1999) and then PRISMA (Ann Intern Med 151:264, 2009) were developed as reporting guidelines and standards to ensure medical meta-analyses and systematic reviews are reported to a high level of detail. PRISMA is now widely used by a range of journals as a pre-submission checklist. However, due to its development for systematic reviews in healthcare, PRISMA has limited applicability for reviews in conservation and environmental management. We highlight 12 key problems with the application of PRISMA to this field, including an overemphasis on meta-analysis and no consideration for other synthesis methods. We introduce ROSES (RepOrting standards for Systematic Evidence Syntheses), a pro forma and flow diagram designed specifically for systematic reviews and systematic maps in the field of conservation and environmental management. We describe how ROSES solves the problems with PRISMA. We outline the key benefits of our approach to designing ROSES, in particular the level of detail and inclusion of rich guidance statements. We also introduce the extraction of meta-data that describe key aspects of the conduct of the review. Collated together, this summary record can help to facilitate rapid review and appraisal of the conduct of a systematic review or map, potentially speeding up the peer-review process. We present the results of initial road testing of ROSES with systematic review experts, and propose a plan for future development of ROSES.

The objective of this paper is to explain how to apply, interpret, and present the results of a new instrument to assess the risk of bias (RoB) in non-randomized studies (NRS) dealing with effects of environmental exposures on health outcomes. This instrument is modeled on the Risk Of Bias In Non-randomized Studies of Interventions (ROBINS-I) instrument. The RoB instrument for NRS of exposures assesses RoB along a standardized comparison to a randomized target experiment, instead of the study-design directed RoB approach. We provide specific guidance for the integral steps of developing a research question and target experiment, distinguishing issues of indirectness from RoB, making individual-study judgments, and performing and interpreting sensitivity analyses for RoB judgments across a body of evidence. Also, we present an approach for integrating the RoB assessments within the Grading of Recommendations Assessment, Development, and Evaluation (GRADE) framework to assess the certainty of the evidence in the systematic review. Finally, we guide the reader through an overall assessment to support the rating of all domains that determine the certainty of a body of evidence using the GRADE approach.

BACKGROUND: While systematic review (SR) methods are gaining traction as a method for providing a reliable summary of existing evidence for health risks posed by exposure to chemical substances, it is becoming clear that their value is restricted to a specific range of risk management scenarios - in particular, those which can be addressed with tightly focused questions and can accommodate the time and resource requirements of a systematic evidence synthesis. METHODS: The concept of a systematic evidence map (SEM) is defined and contrasted to the function and limitations of systematic review (SR) in the context of risk management decision-making. The potential for SEMs to facilitate evidence-based decision-making are explored using a hypothetical example in risk management priority-setting. The potential role of SEMs in reference to broader risk management workflows is characterised. RESULTS: SEMs are databases of systematically gathered research which characterise broad features of the evidence base. Although not intended to substitute for the evidence synthesis element of systematic reviews, SEMs provide a comprehensive, queryable summary of a large body of policy relevant research. They provide an evidence-based approach to characterising the extent of available evidence and support forward looking predictions or trendspotting in the chemical risk sciences. In particular, SEMs facilitate the identification of related bodies of decision critical chemical risk information which could be further analysed using SR methods, and highlight gaps in the evidence which could be addressed with additional primary studies to reduce uncertainties in decision-making. CONCLUSIONS: SEMs have strong and growing potential as a high value tool in resource efficient use of existing research in chemical risk management. They can be used as a critical precursor to efficient deployment of high quality SR methods for characterising chemical health risks. Furthermore, SEMs have potential, at a large scale, to support the sort of evidence summarisation and surveillance methods which would greatly increase the resource efficiency, transparency and effectiveness of regulatory initiatives such as EU REACH and US TSCA.

BACKGROUND: The issue of endocrine disrupting chemicals (EDCs) is receiving wide attention from both the scientific and regulatory communities. Recent analyses of the EDC literature have been criticized for failing to use transparent and objective approaches to draw conclusions about the strength of evidence linking EDC exposures to adverse health or environmental outcomes. Systematic review methodologies are ideal for addressing this issue as they provide transparent and consistent approaches to study selection and evaluation. Objective methods are needed for integrating the multiple streams of evidence (epidemiology, wildlife, laboratory animal, in vitro, and in silico data) that are relevant in assessing EDCs. METHODS: We have developed a framework for the systematic review and integrated assessment (SYRINA) of EDC studies. The framework was designed for use with the International Program on Chemical Safety (IPCS) and World Health Organization (WHO) definition of an EDC, which requires appraisal of evidence regarding 1) association between exposure and an adverse effect, 2) association between exposure and endocrine disrupting activity, and 3) a plausible link between the adverse effect and the endocrine disrupting activity. RESULTS: Building from existing methodologies for evaluating and synthesizing evidence, the SYRINA framework includes seven steps: 1) Formulate the problem; 2) Develop the review protocol; 3) Identify relevant evidence; 4) Evaluate evidence from individual studies; 5) Summarize and evaluate each stream of evidence; 6) Integrate evidence across all streams; 7) Draw conclusions, make recommendations, and evaluate uncertainties. The proposed method is tailored to the IPCS/WHO definition of an EDC but offers flexibility for use in the context of other definitions of EDCs. CONCLUSIONS: When using the SYRINA framework, the overall objective is to provide the evidence base needed to support decision making, including any action to avoid/minimise potential adverse effects of exposures. This framework allows for the evaluation and synthesis of evidence from multiple evidence streams. Finally, a decision regarding regulatory action is not only dependent on the strength of evidence, but also the consequences of action/inaction, e.g. limited or weak evidence may be sufficient to justify action if consequences are serious or irreversible.