James B. Kirby

This article is an empirical evaluation of the choice of fixed cutoff points in assessing the root mean square error of approximation (RMSEA) test statistic as a measure of goodness-of-fit in Structural Equation Models. Using simulation data, the authors first examine whether there is any empirical evidence for the use of a universal cutoff, and then compare the practice of using the point estimate of the RMSEA alone versus that of using it jointly with its related confidence interval. The results of the study demonstrate that there is little empirical support for the use of .05 or any other value as universal cutoff values to determine adequate model fit, regardless of whether the point estimate is used alone or jointly with the confidence interval. The authors' analyses suggest that to achieve a certain level of power or Type I error rate, the choice of cutoff values depends on model specifications, degrees of freedom, and sample size.

In this article, the authors examine the most common type of improper solutions: zero or negative error variances. They address the causes of, consequences of, and strategies to handle these issues. Several hypotheses are evaluated using Monte Carlo simulation models, including two structural equation models with several misspecifications of each model. Results suggested several unique findings. First, increasing numbers of omitted paths in the measurement model were associated with decreasing numbers of improper solutions. Second, bias in the parameter estimates was higher in samples with improper solutions than in samples including only proper solutions. Third, investigations of the consequences of using constrained estimates in the presence of improper solutions indicated that inequality constraints helped some samples achieve convergence. Finally, the use of confidence intervals as well as four other proposed tests yielded similar results when testing whether the error variance was greater than or equal to zero.

OBJECTIVES: To review how heterogeneity has been examined in systematic reviews of diagnostic test accuracy studies. DATA SOURCES: Centre for Reviews and Dissemination's Database of Abstracts of Reviews of Effects (DARE). REVIEW METHODS: Systematic reviews that evaluated a diagnostic or screening test by including studies that compared a test with a reference test were identified from DARE. Reviews for which structured abstracts had been written up to December 2002 were screened for inclusion. Data extraction was undertaken using standardised data extraction forms. RESULTS: A total of 189 systematic reviews met the inclusion criteria. The median number of studies included was 18. Meta-analyses have a higher number with a median of 22 studies compared with 11 for narrative reviews. Graphical plots to demonstrate the spread in study results were provided in 56% of meta-analyses; in 79% these were plots of sensitivity and specificity in the receiver operating characteristic (ROC) space. Statistical tests to identify heterogeneity were used in 32% of reviews: 41% of meta-analyses and 9% of reviews using narrative syntheses. The chi-squared test and Fisher's exact test to assess heterogeneity in individual aspects of test performance were the most common. In contrast, only 16% of meta-analyses used correlation coefficients to test for a threshold effect. A narrative synthesis was used in 30% of reviews. Of the meta-analyses, 52% carried out statistical pooling alone, 18% conducted only summary receiver operator characteristic (SROC) analyses and 30% used both methods of statistical synthesis. For those undertaking SROC analyses, the main differences between the models used were the weights chosen for the regression models, although in 42% of cases the use of, or choice of, weight was not provided. The proportion of reviews using statistical pooling alone has declined from 67% in 1995 to 42% in 2001, with a corresponding increase in the use of SROC methods, from 33% to 58%. However, two-thirds of those using SROC methods also carried out statistical pooling rather than presenting only SROC models. Reviews using SROC analyses also tended to present their results as some combination of sensitivity and specificity rather than using alternative, perhaps less clinically meaningful, means of data presentation such as diagnostic odds ratios. Three-quarters of meta-analyses attempted to investigate statistically possible sources of variation, using subgroup analysis or regression analysis. The impact of clinical or socio-demographic variables was investigated in 74% of these reviews and test- or threshold-related variables in 79%. At least one quality-related variable was investigated in 63% of reviews. Within this subset, the most commonly considered variables were the use of blinding, sample size, the reference test used and the avoidance of verification bias. CONCLUSIONS: The emphasis on pooling individual aspects of diagnostic test performance and the under-use of statistical tests and graphical approaches to identify heterogeneity perhaps reflect the uncertainty in the most appropriate methods to use and also greater familiarity with more traditional indices of test accuracy. This indicates the difficulty and complexity of carrying out such reviews. In these cases it is strongly suggested that meta-analyses are carried out with the involvement of a statistician familiar with the field. Further methodological work on the statistical methods available for combining diagnostic test accuracy studies is needed, as are sufficiently large, prospectively designed primary studies of diagnostic test accuracy comparing two or more tests for the same target disorder. Use of individual patient data meta-analysis in diagnostic test accuracy reviews should be explored to allow heterogeneity to be considered in more detail.

Most research on access to health care focuses on individual-level determinants such as income and insurance coverage. The role of community-level factors in helping or hindering individuals in obtaining needed care, however, has not received much attention. We address this gap in the literature by examining how neighborhood socioeconomic disadvantage is associated with access to health care. We find that living in disadvantaged neighborhoods reduces the likelihood of having a usual source of care and of obtaining recommended preventive services, while it increases the likelihood of having unmet medical need. These associations are not explained by the supply of health care providers. Furthermore, though controlling for individual-level characteristics reduces the association between neighborhood disadvantage and access to health care, a significant association remains. This suggests that when individuals who are disadvantaged are concentrated into specific areas, disadvantage becomes an "emergent characteristic " of those areas that predicts the ability of residents to obtain health care.

OBJECTIVES: The substantial racial and ethnic disparities in access to and use of health services are well documented. A number of studies highlight factors such as health insurance coverage and socioeconomic differences that explain some of the differences between groups, but much remains unexplained. We build on this previous research by incorporating additional factors such as attitudes about health care and neighborhood characteristics, as well as separately analyzing different Hispanic subgroups. METHODS: We use the Oaxaca-Blinder regression-based method to decompose differences among racial and ethnic groups in 3 measures related to access, quantifying the portion explained by each of a number of underlying characteristics and the differences that remain unexplained. We use data from the 2000 and 2001 Medical Expenditure Panel Survey (MEPS), a nationally representative survey of the noninstitutionalized U.S. population. We link these data to detailed neighborhood characteristics from the Census Bureau and local provider supply data from the Health Services Resource Administration (HRSA). RESULTS: Consistent with earlier studies, we find insurance status and socioeconomic differences explain a significant part of the disparities. Additionally, neighborhood racial and ethnic composition account for a large portion of disparities in access, and language differences help explain observed disparities in the use-based access measure. However, much of the differences between racial and ethnic groups remain unexplained. We also found substantial variation in the level of disparities among different groups of Hispanics. CONCLUSIONS: Researchers and policymakers may need to broaden the scope of factors they consider as barriers to access if the goal of eliminating disparities in health care is to be achieved.