NCJ Number
214553
Journal
The Annals Volume: 587 Dated: May 2003 Pages: 69-81
Date Published
May 2003
Length
13 pages
Annotation
This article examines the difference in effect sizes associated with randomized compared to nonrandomized evaluation designs in a large meta-analytic database of delinquency intervention evaluations.
Abstract
The central theme of the article is that interesting moderator variables (variables that represent differences among studies in terms of methods, samples, and interventions) that might explain different treatment effects in meta-analysis cannot be assumed to be statistically independent. They tend to be unevenly distributed across studies, and correlations and differences make it difficult to determine which relationships are significant in comparing findings. Meta-analysis, however, has the potential to identify the variety of interactive patterns between treatment methods and subject characteristics, which in turn are related to the intervention effects. This is the information that is most important for improving practice and policy. It is difficult to determine and clarify the relationships between moderators and identify those that have significant independent relationships with effect sizes. In meta-analysis, multivariate techniques can address this problem; however, this requires that the number of available studies be sufficiently large to produce reliable findings. Even then, multivariate analysis must include all the important moderator variables. If an unmeasured moderator variable is mixed with a measured one and accounts for the measured variable's relationship to effect size, it will remain undetected. Meta-analysis is the most systematic and credible procedure currently available for providing a broad picture of the results of intervention studies, so as to guide practitioners, policymakers, and research. The claims of a meta-analysis are credible only when alternative explanations associated with moderator variables have been explored as fully as possible and framed as hypotheses for further testing in primary studies. 4 tables and 12 references