John Concato

BACKGROUND: In the hierarchy of research designs, the results of randomized, controlled trials are considered to be evidence of the highest grade, whereas observational studies are viewed as having less validity because they reportedly overestimate treatment effects. We used published meta-analyses to identify randomized clinical trials and observational studies that examined the same clinical topics. We then compared the results of the original reports according to the type of research design. METHODS: A search of the Medline data base for articles published in five major medical journals from 1991 to 1995 identified meta-analyses of randomized, controlled trials and meta-analyses of either cohort or case-control studies that assessed the same intervention. For each of five topics, summary estimates and 95 percent confidence intervals were calculated on the basis of data from the individual randomized, controlled trials and the individual observational studies. RESULTS: For the five clinical topics and 99 reports evaluated, the average results of the observational studies were remarkably similar to those of the randomized, controlled trials. For example, analysis of 13 randomized, controlled trials of the effectiveness of bacille Calmette-Guérin vaccine in preventing active tuberculosis yielded a relative risk of 0.49 (95 percent confidence interval, 0.34 to 0.70) among vaccinated patients, as compared with an odds ratio of 0.50 (95 percent confidence interval, 0.39 to 0.65) from 10 case-control studies. In addition, the range of the point estimates for the effect of vaccination was wider for the randomized, controlled trials (0.20 to 1.56) than for the observational studies (0.17 to 0.84). CONCLUSIONS: The results of well-designed observational studies (with either a cohort or a case-control design) do not systematically overestimate the magnitude of the effects of treatment as compared with those in randomized, controlled trials on the same topic.

PURPOSE: To review the principles of multivariable analysis and to examine the application of multivariable statistical methods in general medical literature. DATA SOURCES: A computer-assisted search of articles in The Lancet and The New England Journal of Medicine identified 451 publications containing multivariable methods from 1985 through 1989. A random sample of 60 articles that used the two most common methods--logistic regression or proportional hazards analysis--was selected for more intensive review. DATA EXTRACTION: During review of the 60 randomly selected articles, the focus was on generally accepted methodologic guidelines that can prevent problems affecting the accuracy and interpretation of multivariable analytic results. RESULTS: From 1985 to 1989, the relative frequency of multivariable statistical methods increased annually from about 10% to 18% among all articles in the two journals. In 44 (73%) of 60 articles using logistic or proportional hazards regression, risk estimates were quantified for individual variables ("risk factors"). Violations and omissions of methodologic guidelines in these 44 articles included overfitting of data; no test of conformity of variables to a linear gradient; no mention of pertinent checks for proportional hazards; no report of testing for interactions between independent variables; and unspecified coding or selection of independent variables. These problems would make the reported results potentially inaccurate, misleading, or difficult to interpret. CONCLUSIONS: The findings suggest a need for improvement in the reporting and perhaps conducting of multivariable analyses in medical research.

Multivariable methods of analysis can yield problematic results if methodological guidelines and mathematical assumptions are ignored. A problem arising from a too-small ratio of events per variable (EPV) can affect the accuracy and precision of regression coefficients and their tests of statistical significance. The problem occurs when a proportional hazards analysis contains too few "failure" events (e.g., deaths) in relation to the number of included independent variables. In the current research, the impact of EPV was assessed for results of proportional hazards analysis done with Monte Carlo simulations in an empirical data set of 673 subjects enrolled in a multicenter trial of coronary artery bypass surgery. The research is presented in two parts: Part I describes the data set and strategy used for the analyses, including the Monte Carlo simulation studies done to determine and compare the impact of various values of EPV in proportional hazards analytical results. Part II compares the output of regression models obtained from the simulations, and discusses the implication of the findings.

BACKGROUND: Weight gain is used by disease-management programs as a marker of heart failure decompensation, but little information is available to quantify the relationship between weight change in patients with heart failure and the risk for imminent hospitalization. METHODS AND RESULTS: We conducted a nested case-control study among patients with heart failure referred to a home monitoring system by managed care organizations. We matched 134 case patients with heart failure hospitalization to 134 control patients without heart failure hospitalization on the basis of age, sex, duration of home monitoring, heart failure severity, and baseline body weight. Compared with control patients, case patients experienced gradual weight gain beginning approximately 30 days before hospitalization; changes in daily weight between case and control patients were statistically significant (P<0.001). Within the week before hospitalization, when weight patterns in case and control patients began to diverge more substantially, mean increases of more than 2 and up to 5 pounds, more than 5 and up to 10 pounds, and more than 10 pounds (relative to time of enrollment in the monitoring system) were associated with matched adjusted odds ratios for heart failure hospitalization of 2.77 (95% confidence interval 1.13 to 6.80), 4.46 (95% confidence interval 1.45 to 13.75), and 7.65 (95% confidence interval 2.22 to 26.39), respectively, compared with mean increases of 2 pounds or less. CONCLUSIONS: Increases in body weight are associated with hospitalization for heart failure and begin at least 1 week before admission. Daily information about patients' body weight identifies a high-risk period during which interventions to avert decompensated heart failure that necessitates hospitalization may be beneficial.