PROPOSED GUIDELINES FOR EVALUATION RESEARCH

• NCJ Number: 47219
• Journal: Policy Studies Journal Volume: 3 Issue: 1 Dated: (AUTUMN 1974) Pages: 107-111
• Author(s): N P ROOS
• Date Published: 1974
• Length: 5 pages
• Corporate Author: Policy Studies Organization
• Publication Format: Article
• Publication Type: Program/Project Evaluation
• Language: English
• Country: United States of America

Annotation

GUIDELINES FOR EVALUATION RESEARCH ARE PROPOSED; PROCESS AND EXPERIMENTAL RESEARCH DESIGNS ARE PRESENTED; AND THEIR ADVANTAGES ARE DISCUSSED.

Abstract

FOR THE EFFECTIVE CONDUCT OF EVALUATIVE RESEARCH, SUCH RESEARCH MUST ADDRESS THE POLICYMAKER'S OBJECTIVES, MUST EXAMINE OPERATIONALLY RELEVANT ALTERNATIVES, AND MUST BE BASED ON A SUFFICIENTLY VALID METHODOLOGY. IN ADDITION, RESEARCH RESULTS MUST ALLOW FOR TIMELY, INTELLIGENT, AND NONTHREATENING PRESENTATION TO THE POLICYMAKER. THE RESEARCH DESIGN WILL DEPEND ON THE NATURE OF THE PROGRAM, THE EXTENT TO WHICH IT HAS WELL-DEFINED OBJECTIVES, AND THE AVAILABILITY OF DATA AND DATA COLLECTION MECHANISMS. MAJOR RESEARCH DESIGNS IN ORDER OF INCREASING EMPIRICAL VALIDITY ARE: (1) DESCRIPTIVE MONITORING -- AN INFORMED COMMON-SENSE APPROACH TO EVALUATION WHICH IS APPROPRIATE WHEN PROGRAM EFFECTS ARE OF AN AMBIGUOUS OR EXTREMELY BROAD NATURE; (2) AUDITING THE EFFICIENCY WITH WHICH THE PROGRAM IS CONDUCTED, RATHER THAN THE EFFECTIVENESS OF THE PROGRAM IN ACHIEVING OBJECTIVES; (3) EX POST FACTO DESIGNS -- USE OF STATISTICAL CONTROLS RATHER THAN PREINTERVENTION DATA TO ASSESS PROGRAM IMPACT; (4) ONE-GROUP PRETEST/POSTTEST DESIGN -- ASSESSMENT OF COMPARABLE DATA ON PROGRAM OBJECTIVES FOR THE PERIOD BEFORE AND AFTER THE INTERVENTION; (5) PRETEST/POSTTEST CONTROL GROUP DESIGNS -- UTILIZING BEFORE AND AFTER INTERVENTION DATA FOR A COMPARABLE GROUP NOT RECEIVING INTERVENTION; AND (6) PRETEST/POSTTEST CONTROL GROUP DESIGN -- USING RANDOM ASSIGNMENT OF SUBJECTS TO TEST AND CONTROL GROUPS AND COMPARING TREATMENT/NONTREATMENT CHANGE. DESIGNS 4, 5, AND 6 SHOULD BE USED WHEN POSSIBLE, AND PROGRAM EVALUATION SHOULD BE INCORPORATED IN THE INITIAL DESIGN OF AN EXPERIMENTAL PROGRAM. IN GENERAL THE WEAKER DESIGNS (1, 2, AND 3) ARE USED AS A RESULT OF MISUNDERSTANDING ON THE PART OF POLICYMAKERS OF THE ROLE OF EVALUATIVE RESEARCH OR AS A RESULT OF THE UNANTICIPATED NATURE OF POLICY RESEARCH NEEDS. THE WEAKER DESIGNS ARE LIKELY TO PRODUCE UNINTERPRETABLE OR MISLEADING RESULTS AND SHOULD BE AVOIDED WHENEVER POSSIBLE. REFERENCES ARE PROVIDED. (JAP)