BOOBYTRAPS AND PITFALLS IN THE EVALUATION OF SOCIAL ACTION PROGRAMS (FROM READINGS IN EVALUATION RESEARCH, 2ND ED., 1977, BY FRANCIS G CARO - SEE NCJ-50468)

• NCJ Number: 50477
• Author(s): P H ROSSI
• Date Published: 1977
• Length: 10 pages
• Corporate Author: Russell Sage Foundation
• Publication Format: Document
• Language: English
• Country: United States of America
• Note: REPRINTED FROM THE 1966 SOCIAL STATISTICS SECTION, PROCEEDINGS OF THE AMERICAN STATISTICAL ASSOCIATION

Annotation

THE POLITICS OF EVALUATION, THE USE OF EVALUATION RESULTS, THE PROBLEM OF USING CONTROL GROUPS, AND A STRATEGY FOR EVALUATION RESEARCH ARE DISCUSSED.

Abstract

EVALUATION HAS BECOME A POPULAR ENTERPRISE. EVALUATIONS WHOSE METHODOLOGIES RENDER UNEQUIVOCAL RESULTS, HOWEVER, ARE RARE. THIS SITUATION PREVAILS BECAUSE INNOVATORS AND ADMINISTRATORS OF SOCIAL ACTION PROGRAMS OFTEN HAVE CAREER AND ECONOMIC SELF-INTERESTS THAT CAN BE THREATENED BY A THOROUGHLY SCIENTIFIC EVALUATION THAT REVEALS THE INEFFECTIVENESS OF A PROGRAM IN WHICH THEY ARE INVOLVED. THIS SAME SELF-INTEREST ALSO CAN LEAD TO IGNORING THE RESULTS OF A VALID EVALUATON WHEN THEY DO NOT FAVOR THE UNCHANGED CONTINUATION OF A PROGRAM. A WELL-CONCEIVED AND EFFECTIVELY IMPLEMENTED EVALUATION IS VIEWED AS HAVING MEANING ONLY WHEN THE ADMINISTRATORS AND FUNDING AUTHORITIES SERIOUSLY INTEND TO INCORPORATE EVALUATION FINDINGS INTO THEIR DECISIONMAKING. ONE METHOD OF ASSURING POSITIVE RESULTS IN A SOCIAL ACTION PROGRAM IS TO USE THE MOST SOPHISTICATED SERVICE AND TREATMENT APPROACHES AVAILABLE WITHOUT REGARD FOR COST. THIS APPROACH, HOWEVER, IS NOT FEASIBLE FOR MOST PROGRAMS BECAUSE OF THE HIGH COSTS INVOLVED. EVALUATIONS, THEREFORE, MUST TAKE THESE ASPECTS INTO ACCOUNT WHEN DEALING WITH DEMONSTRATION PROJECTS. THE MOST EFFECTIVE EVALUATION DESIGN EMPLOYS EXPERIMENTAL AND CONTROL GROUPS, BUT MOST PUBLIC AGENCIES PROHIBIT THE WILLFUL EXCLUSION OF ANY ELIGIBLE PERSON FROM THE THE SERVICE MAKING THIS DESIGN INAPPROPRIATE. AS A WAY OF CIRCUMVENTING THIS POLICY, IT IS SUGGESTED THAT A PLACEBO TREATMENT BE DEVELOPED FOR COMPARISON WITH THE EXPERIMENTAL GROUP THAT DOES NOT SLIGHT ANY INDIVIDUALS INVOLVED. IN THE LIGHT OF THE PROBLEMS ASSOCIATED WITH THE USE OF CONTROL GROUPS, THE FOLLOWING STRATEGY FOR EVALUATION IS SUGGESTED: (1) THE USE OF A 'RECONNAISSANCE PHASE' EMPLOYING SOFT CORRECTIONAL EVALUATION DESIGNS TO SCREEN PROGRAMS THAT SHOULD BE INVESTIGATED FURTHER, AND (2) THE USE OF AN 'EXPERIMENTAL PHASE' IN WHICH CONTROLLED EXPERIMENTAL DESIGNS ARE USED WITH PROJECTS FOR WHICH A FAVORABLE RESULT WAS OBTAINED IN THE 'RECONNAISSANCE PHASE.' (RCB)