Roberto D’Amico

Michel Foucault. Discipline and Punish: The Birth of the Prison. New York: Pantheon, 1977. $10.95. 333 pages.

OBJECTIVES: To consider methods and related evidence for evaluating bias in non-randomised intervention studies. DATA SOURCES: Systematic reviews and methodological papers were identified from a search of electronic databases; handsearches of key medical journals and contact with experts working in the field. New empirical studies were conducted using data from two large randomised clinical trials. METHODS: Three systematic reviews and new empirical investigations were conducted. The reviews considered, in regard to non-randomised studies, (1) the existing evidence of bias, (2) the content of quality assessment tools, (3) the ways that study quality has been assessed and addressed. (4) The empirical investigations were conducted generating non-randomised studies from two large, multicentre randomised controlled trials (RCTs) and selectively resampling trial participants according to allocated treatment, centre and period. RESULTS: In the systematic reviews, eight studies compared results of randomised and non-randomised studies across multiple interventions using meta-epidemiological techniques. A total of 194 tools were identified that could be or had been used to assess non-randomised studies. Sixty tools covered at least five of six pre-specified internal validity domains. Fourteen tools covered three of four core items of particular importance for non-randomised studies. Six tools were thought suitable for use in systematic reviews. Of 511 systematic reviews that included non-randomised studies, only 169 (33%) assessed study quality. Sixty-nine reviews investigated the impact of quality on study results in a quantitative manner. The new empirical studies estimated the bias associated with non-random allocation and found that the bias could lead to consistent over- or underestimations of treatment effects, also the bias increased variation in results for both historical and concurrent controls, owing to haphazard differences in case-mix between groups. The biases were large enough to lead studies falsely to conclude significant findings of benefit or harm. Four strategies for case-mix adjustment were evaluated: none adequately adjusted for bias in historically and concurrently controlled studies. Logistic regression on average increased bias. Propensity score methods performed better, but were not satisfactory in most situations. Detailed investigation revealed that adequate adjustment can only be achieved in the unrealistic situation when selection depends on a single factor. CONCLUSIONS: Results of non-randomised studies sometimes, but not always, differ from results of randomised studies of the same intervention. Non-randomised studies may still give seriously misleading results when treated and control groups appear similar in key prognostic factors. Standard methods of case-mix adjustment do not guarantee removal of bias. Residual confounding may be high even when good prognostic data are available, and in some situations adjusted results may appear more biased than unadjusted results. Although many quality assessment tools exist and have been used for appraising non-randomised studies, most omit key quality domains. Healthcare policies based upon non-randomised studies or systematic reviews of non-randomised studies may need re-evaluation if the uncertainty in the true evidence base was not fully appreciated when policies were made. The inability of case-mix adjustment methods to compensate for selection bias and our inability to identify non-randomised studies that are free of selection bias indicate that non-randomised studies should only be undertaken when RCTs are infeasible or unethical. Recommendations for further research include: applying the resampling methodology in other clinical areas to ascertain whether the biases described are typical; developing or refining existing quality assessment tools for non-randomised studies; investigating how quality assessments of non-randomised studies can be incorporated into reviews and the implications of individual quality features for interpretation of a review's results; examination of the reasons for the apparent failure of case-mix adjustment methods; and further evaluation of the role of the propensity score.

OBJECTIVES: To survey the frequency of use of indirect comparisons in systematic reviews and evaluate the methods used in their analysis and interpretation. Also to identify alternative statistical approaches for the analysis of indirect comparisons, to assess the properties of different statistical methods used for performing indirect comparisons and to compare direct and indirect estimates of the same effects within reviews. DATA SOURCES: Electronic databases. REVIEW METHODS: The Database of Abstracts of Reviews of Effects (DARE) was searched for systematic reviews involving meta-analysis of randomised controlled trials (RCTs) that reported both direct and indirect comparisons, or indirect comparisons alone. A systematic review of MEDLINE and other databases was carried out to identify published methods for analysing indirect comparisons. Study designs were created using data from the International Stroke Trial. Random samples of patients receiving aspirin, heparin or placebo in 16 centres were used to create meta-analyses, with half of the trials comparing aspirin and placebo and half heparin and placebo. Methods for indirect comparisons were used to estimate the contrast between aspirin and heparin. The whole process was repeated 1000 times and the results were compared with direct comparisons and also theoretical results. Further detailed case studies comparing the results from both direct and indirect comparisons of the same effects were undertaken. RESULTS: Of the reviews identified through DARE, 31/327 (9.5%) included indirect comparisons. A further five reviews including indirect comparisons were identified through electronic searching. Few reviews carried out a formal analysis and some based analysis on the naive addition of data from the treatment arms of interest. Few methodological papers were identified. Some valid approaches for aggregate data that could be applied using standard software were found: the adjusted indirect comparison, meta-regression and, for binary data only, multiple logistic regression (fixed effect models only). Simulation studies showed that the naive method is liable to bias and also produces over-precise answers. Several methods provide correct answers if strong but unverifiable assumptions are fulfilled. Four times as many similarly sized trials are needed for the indirect approach to have the same power as directly randomised comparisons. Detailed case studies comparing direct and indirect comparisons of the same effect show considerable statistical discrepancies, but the direction of such discrepancy is unpredictable. CONCLUSIONS: Direct evidence from good-quality RCTs should be used wherever possible. Without this evidence, it may be necessary to look for indirect comparisons from RCTs. However, the results may be susceptible to bias. When making indirect comparisons within a systematic review, an adjusted indirect comparison method should ideally be used employing the random effects model. If both direct and indirect comparisons are possible within a review, it is recommended that these be done separately before considering whether to pool data. There is a need to evaluate methods for the analysis of indirect comparisons for continuous data and for empirical research into how different methods of indirect comparison perform in cases where there is a large treatment effect. Further study is needed into when it is appropriate to look at indirect comparisons and when to combine both direct and indirect comparisons. Research into how evidence from indirect comparisons compares to that from non-randomised studies may also be warranted. Investigations using individual patient data from a meta-analysis of several RCTs using different protocols and an evaluation of the impact of choosing different binary effect measures for the inverse variance method would also be useful.

BACKGROUND: This review is the third update of the Cochrane review "Selenium for preventing cancer". Selenium is a naturally occurring element with both nutritional and toxicological properties. Higher selenium exposure and selenium supplements have been suggested to protect against several types of cancer. OBJECTIVES: To gather and present evidence needed to address two research questions:1. What is the aetiological relationship between selenium exposure and cancer risk in humans?2. Describe the efficacy of selenium supplementation for cancer prevention in humans. SEARCH METHODS: We updated electronic searches of the Cochrane Central Register of Controlled Trials (CENTRAL; 2017, Issue 2), MEDLINE (Ovid, 2013 to January 2017, week 4), and Embase (2013 to 2017, week 6), as well as searches of clinical trial registries. SELECTION CRITERIA: We included randomised controlled trials (RCTs) and longitudinal observational studies that enrolled adult participants. DATA COLLECTION AND ANALYSIS: We performed random-effects (RE) meta-analyses when two or more RCTs were available for a specific outcome. We conducted RE meta-analyses when five or more observational studies were available for a specific outcome. We assessed risk of bias in RCTs and in observational studies using Cochrane's risk assessment tool and the Newcastle-Ottawa Scale, respectively. We considered in the primary analysis data pooled from RCTs with low risk of bias. We assessed the certainty of evidence by using the GRADE approach. MAIN RESULTS: We included 83 studies in this updated review: two additional RCTs (10 in total) and a few additional trial reports for previously included studies. RCTs involved 27,232 participants allocated to either selenium supplements or placebo. For analyses of RCTs with low risk of bias, the summary risk ratio (RR) for any cancer incidence was 1.01 (95% confidence interval (CI) 0.93 to 1.10; 3 studies, 19,475 participants; high-certainty evidence). The RR for estimated cancer mortality was 1.02 (95% CI 0.80 to 1.30; 1 study, 17,444 participants). For the most frequently investigated site-specific cancers, investigators provided little evidence of any effect of selenium supplementation. Two RCTs with 19,009 participants indicated that colorectal cancer was unaffected by selenium administration (RR 0.99, 95% CI 0.69 to 1.43), as were non-melanoma skin cancer (RR 1.16, 95% CI 0.30 to 4.42; 2 studies, 2027 participants), lung cancer (RR 1.16, 95% CI 0.89 to 1.50; 2 studies, 19,009 participants), breast cancer (RR 2.04, 95% CI 0.44 to 9.55; 1 study, 802 participants), bladder cancer (RR 1.07, 95% CI 0.76 to 1.52; 2 studies, 19,009 participants), and prostate cancer (RR 1.01, 95% CI 0.90 to 1.14; 4 studies, 18,942 participants). Certainty of the evidence was high for all of these cancer sites, except for breast cancer, which was of moderate certainty owing to imprecision, and non-melanoma skin cancer, which we judged as moderate certainty owing to high heterogeneity. RCTs with low risk of bias suggested increased melanoma risk.Results for most outcomes were similar when we included all RCTs in the meta-analysis, regardless of risk of bias. Selenium supplementation did not reduce overall cancer incidence (RR 0.99, 95% CI 0.86 to 1.14; 5 studies, 21,860 participants) nor mortality (RR 0.81, 95% CI 0.49 to 1.32; 2 studies, 18,698 participants). Summary RRs for site-specific cancers showed limited changes compared with estimates from high-quality studies alone, except for liver cancer, for which results were reversed.In the largest trial, the Selenium and Vitamin E Cancer Trial, selenium supplementation increased risks of alopecia and dermatitis, and for participants with highest background selenium status, supplementation also increased risk of high-grade prostate cancer. RCTs showed a slightly increased risk of type 2 diabetes associated with supplementation. A hypothesis generated by the Nutritional Prevention of Cancer Trial - that individuals with low blood selenium levels could reduce their risk of cancer (particularly prostate cancer) by increasing selenium intake - has not been confirmed. As RCT participants have been overwhelmingly male (88%), we could not assess the potential influence of sex or gender.We included 15 additional observational cohort studies (70 in total; over 2,360,000 participants). We found that lower cancer incidence (summary odds ratio (OR) 0.72, 95% CI 0.55 to 0.93; 7 studies, 76,239 participants) and lower cancer mortality (OR 0.76, 95% CI 0.59 to 0.97; 7 studies, 183,863 participants) were associated with the highest category of selenium exposure compared with the lowest. Cancer incidence was lower in men (OR 0.72, 95% CI 0.46 to 1.14, 4 studies, 29,365 men) than in women (OR 0.90, 95% CI 0.45 to 1.77, 2 studies, 18,244 women). Data show a decrease in risk of site-specific cancers for stomach, colorectal, lung, breast, bladder, and prostate cancers. However, these studies have major weaknesses due to study design, exposure misclassification, and potential unmeasured confounding due to lifestyle or nutritional factors covarying with selenium exposure beyond those taken into account in multi-variable analyses. In addition, no evidence of a dose-response relation between selenium status and cancer risk emerged. Certainty of evidence was very low for each outcome. Some studies suggested that genetic factors might modify the relation between selenium and cancer risk - an issue that merits further investigation. AUTHORS' CONCLUSIONS: Well-designed and well-conducted RCTs have shown no beneficial effect of selenium supplements in reducing cancer risk (high certainty of evidence). Some RCTs have raised concerns by reporting a higher incidence of high-grade prostate cancer and type 2 diabetes in participants with selenium supplementation. No clear evidence of an influence of baseline participant selenium status on outcomes has emerged in these studies.Observational longitudinal studies have shown an inverse association between selenium exposure and risk of some cancer types, but null and direct relations have also been reported, and no systematic pattern suggesting dose-response relations has emerged. These studies suffer from limitations inherent to the observational design, including exposure misclassification and unmeasured confounding.Overall, there is no evidence to suggest that increasing selenium intake through diet or supplementation prevents cancer in humans. However, more research is needed to assess whether selenium may modify the risk of cancer in individuals with a specific genetic background or nutritional status, and to investigate possible differential effects of various forms of selenium.