Commentary on the Pitfalls and Pratfalls of Evaluation Research With Intervention and Prevention Programs

• NCJ Number: 215613
• Journal: New Directions for Evaluation Issue: 110 Dated: Summer 2006 Pages: 87-96
• Author(s): Karen L. Bierman
• Date Published: 2006
• Length: 10 pages
• Publication Format: Article
• Publication Type: Program/Project Evaluation
• Language: English
• Country: United States of America
• Note: For other articles in this journal, see NCJ-215607-12.

Annotation

This article summarizes the findings of previous articles regarding issues that should be considered in devising evaluation methods and interpreting evaluation findings for intervention and prevention programs, specifically youth violence prevention programs.

Abstract

These articles focus on lessons learned from youth violence prevention programs that failed to show significant results in community settings. The articles indicate that one reason an intervention may fail is that the theoretical model underlying the intervention is flawed. The solution must be to rework the basic foundation of the intervention design and rethink the intervention focus. In other cases, the theoretical model underlying the intervention may be accurate, but the intervention may fail because it does not modify factors that tend to prevent changes in targeted attitudes and behaviors. Remedial action does not require changing the program's target goals, but rather a strengthening of the means used to induce change in attitudes and behaviors. A third cause of failed programs is lack of compliance with planned program implementation procedures and methods. Intervention delivery and data collection are crucial in determining whether the outcome is a valid measure of a program's ability to achieve its goals. In addition to the aforementioned issue, the articles in this journal also identify the challenges that prevention evaluation methodology faces in the field. Participant recruitment and engagement continue to be major challenges in most areas of prevention programming. Measurement and the assessment of significant change are another key challenge for prevention research. The articles also emphasize the importance of a thorough analysis of evaluation findings that do not show the level of program impact desired but which show promise for use with individuals having particular characteristics and under particular conditions. 20 references