EMPIRICAL STUDY OF METHODS USED IN CRIMINAL JUSTICE EVALUATIONS - FOURTH QUARTERLY PROGRESS REPORT OCTOBER 1, 1978-DECEMBER 31, 1978

• NCJ Number: 55104
• Author(s): R C LARSON
• Date Published: 1978
• Length: 30 pages
• Grant Number(s): 78N1-AX-0007
• Sponsoring Agency: US Dept of Justice
• Corporate Author: Massachusetts Institute of Technology
• Sale Source: National Institute of Justice/
• Publication Type: Report (Study/Research)
• Language: English
• Country: United States of America

Annotation

AN EXECUTIVE SUMMARY IS PRESENTED OF FINDINGS FROM AN EXAMINATION OF 200 REPORTS OF EVALUATIONS OF CRIMINAL JUSTICE PROGRAMS.

Abstract

ALTHOUGH THE AIM OF AN EFFECTIVE EVALUATION IS TO ASSESS THE VALUE OF A PARTICULAR PROGRAM, IF DONE PROPERLY, IT MUST ALSO BE USEFUL EITHER TO PROGRAM MANAGERS, PROGRAM CLIENTS, POLICYMAKERS, OR OTHER SOCIAL SCIENTISTS. THE PROJECT STAFF DEVELOPED SIX CRITERIA BY WHICH THE USABILITY OF A GIVEN EVALUATION MAY BE ASSESSED VIS-A-VIS ITS AUDIENCE: (1) DEGREE OF COMPREHENSIVENESS, (2) APPROPRIATENESS AND APPLICATION OF METHODOLOGY, (3) FLEXIBILITY OF DESIGN AND IMPLEMENTATION, (4) AMOUNT AND QUALITY OF COMMUNICATION BETWEEN PROGRAM STAFF AND EVALUATORS, (5) ACCESSIBILITY OF THE FINAL REPORT, AND (6) DEMONSTRATED AWARENESS OF INSTITUTIONAL AND ENVIRONMENTAL ISSUES AFFECTING THE PROGRAM AND THE EVALUATION. THESE CRITERIA WERE APPLIED TO THE 200 EVALUATIONS STUDIED. KNOWLEDGE OF INPUTS (TYPE OF FUNDINGS, STAFFING PATTERNS TIMING, ETC.) TO AN EVALUATION IS NECESSARY FOR A FAIR EVALUATION OF THAT EVALUATION. FEW OF THE EVALUATIONS CONSIDERED PROVIDED THIS KIND OF INFORMATION, WHICH COULD BE HELPFUL TO ANYONE CONSIDERING THE EVALUATION REPORT. THE QUESTION OF WHETHER THE PROGRAM BEING EVALUATED WAS DIRECTED AT THE APPROPRIATE TARGET POPULATION WAS NOT CONSIDERED IN NEARLY ONE-THIRD OF THE EVALUATIONS IN THE SAMPLE. THE PRIMARY REASON FOR THIS WAS THAT NOT ENOUGH INFORMATION WAS PROVIDED IN THE EVALUATION TO BE ABLE TO DETERMINE THIS. IT IS IMPORTANT TO FIND OUT WHY SO FEW STUDIES EXPLICITLY CONSIDERED THE CLIENT POPULATION ISSUE, PARTICULARLY IN THE NONLOGISTICAL PROGRAMS. THE EVALUATIONS WERE APPARENTLY WEAKEST IN THE AREA OF METHODOLOGY, FOLLOWED BY COMPREHENSIVENESS AND FLEXIBILITY OF DESIGN AND IMPLEMENTATION. SOME OF THE PROBLEMS MANIFESTED IN THESE EVALUATIONS COULD BE RECTIFIED AT THE PROPOSAL STAGE IF SPONSORING AGENCIES DETERMINE THE OBJECTIVES OF AN EVALUATION AND REVIEW PROPOSALS CAREFULLY. IT IS EXPECTED THAT ADDITIONAL INSIGHTS WILL BE GAINED INTO THE CAUSES OF EVALUATION WEAKNESSES THROUGH RESPONSES TO QUESTIONNAIRES THAT WILL BE SENT TO THE AUTHORS AND USERS OF THE EVALUATIONS EXAMINED. PUBLICATIONS RECEIVED FOR THE PROJECT LIBRARY ARE LISTED, AND THE COMPUTER SYSTEM USED FOR THE PROJECT IS BRIEFLY DISCUSSED. FOR THE FIRST THREE QUARTERLY REPORTS, SEE NCJ 52597-52599. (RCB)