DEVELOPING USEFUL EVALUATION CAPABILITY - LESSONS FROM THE MODEL EVALUATION PROGRAM

• NCJ Number: 55682
• Author(s): J D WALLER; J W SCANLON; D M KEMP; P G NALLEY
• Date Published: 1979
• Length: 156 pages
• Grant Number(s): 77-NI-99-0022
• Sponsoring Agency: US Dept of Justice
• Corporate Author: The Urban Institute
• Sale Source: Superintendent of Documents, GPO; National Institute of Justice/
• Publication Type: Report (Study/Research)
• Language: English
• Country: United States of America

Annotation

THIS REPORT PRESENTS FINDINGS FROM AN ASSESSMENT OF EIGHT MODEL EVALUATION PROGRAMS (MEP) FUNDED BY NILECJ AND CONDUCTED BY STATE AND REGIONAL PLANNING AGENCIES TO DEVELOP AND IMPLEMENT EVALUATION APPROACHES.

Abstract

PROGRAM OBJECTIVES WERE TO ENCOURAGE STATE AND LOCAL PLANNING AGENCIES TO GENERATE AND USE EVALUATION INFORMATION AND TEST WAYS IN WHICH EFFECTIVE USE OF EVALUATION INFORMATION CAN HELP THESE AGENCIES TO ACHIEVE THEIR OBJECTIVES. THIS PAPER REPORTS ON LESSONS LEARNED FROM THE MEP'S, WAYS TO MEASURE MEP SUCCESS, METHODS FOR BUILDING AN EVALUATION CAPABILITY, AND PROBLEMS TO BE EXPECTED IN IMPLEMENTING AND OPERATING EVALUATION SYSTEMS. IT WAS DETERMINED THAT EVALUATION SYSTEMS (1) CANNOT BE EXPECTED TO HAVE A MEASURABLE IMPACT ON THE PERFORMANCE OF STATE PLANNING AGENCIES AND REGIONAL PLANNING UNITS, SINCE THESE AGENCIES DO NOT HAVE PRIMARY OBJECTIVES THAT ARE BOTH VERIFIABLE AND REALISTIC, (2) ARE NOT LIKELY TO BE USEFUL TO THE PLANNING AGENCY STAFF UNLESS THE AGENCY IS BEING DIRECTED TOWARD SOME MISSION-RELATED OBJECTIVE, AND (3) CAN INCREASE THEIR CHANCES OF PROVIDING USEFUL INFORMATION BY SERVING USERS' EXISTING INFORMATION NEEDS AND ALLOWING USERS TO CONTROL THE STUDY. EVALUATION SYSTEMS OF VALUE TO POTENTIAL USERS CAN BE SET UP ONLY IF THE USER IS INVOLVED IN EVALUATION ACTIVITY. GIVEN THE OPPORTUNITY, PLANNING AGENCIES WILL SET UP EVALUATION SYSTEMS WHICH DIFFER IN PRODUCT, COST, AND OPERATIONS, AND THESE SYSTEMS WILL NOT BE SUCCESSFUL IN LOCAL CRIMINAL JUSTICE AGENCIES. PERSONS SETTING UP EVALUATION SYSTEMS CAN EXPECT A STAFF TURNOVER CAUSING DELAYS AND DISRUPTIONS, RELUCTANCE FROM PROJECT PERSONNEL TO RELINQUISH DATA, AND PROBLEMS WITH CIVIL SERVICE RULES, COMPUTER PROCESSING, AND CONTRACTING. SUCCESS OF THE MEP'S WHEN MEASURED BY ACHIEVEMENT OF AGENCY OBJECTIVES, USE, AND CONTINUATION, WAS SLIGHT, BUT DEMONSTRABLE SUCCESS IN HALF OF THE MEP SITES WAS NOTICEABLE AFTER MINIMIZING SUCCESS CRITERIA TO 'PROVIDING USEFUL INFORMATION TO AN IDENTIFIED MARKET.' MEP GRANTEES USED VARIOUS EVALUATION CAPABILITIES, SUCH AS OPERATING SYSTEMS ON A FULL-SCALE BASIS, ON A PILOT BASIS, ON A ONE-TIME BASIS, OR AS A TRAINING PROJECT. BRIEF DESCRIPTIONS OF EIGHT OF THESE SYSTEMS ARE PRESENTED ALONG WITH PROBLEMS TO BE EXPECTED IN IMPLEMENTING AND OPERATING EVALUATION SYSTEMS. TABLES AND FIGURES ARE PROVIDED. FOR FULL DESCRIPTIONS OF THE EIGHT PROGRAMS SEE NCJ 55723.