NCJ Number
49586
Date Published
1977
Length
13 pages
Annotation
THE DELIBERATIONS OF A SYMPOSIUM WORKING PANEL ON WAYS OF IMPROVING THE DEFINITION OF CRITERIA IN FEDERAL PROGRAM EVALUATION ARE SUMMARIZED.
Abstract
THE PANEL BEGAN BY NOTING THE IMPORTANCE OF NOT REGARDING EFFECTIVENESS AND EFFICIENCY AS THE SOLE CRITERIA FOR EVALUATION PROGRAMS. RECOGNITION OF THE DIFFERENT VALUES AND PERSPECTIVES OF THE MULTIPLE USERS OF MOST EVALUATIONS WAS RECOGNIZED AS A FUNDAMENTAL TASK IN DEFINING CRITERIA. THE IMPORTANCE OF THE DISTINCTION BETWEEN CRITERIA AND MEASUREMENT WAS BROUGHT OUT. CRITERIA ARE STANDARDS OR OBJECTIVES AND ARE FAR MORE ABSTRACT THAN ANY SET OF MEASURES, WHICH ARE FORMS OF INFORMATION THAT ARE RELATED BACK TO CRITERIA. FROM ITS RELATIVELY ABSTRACT CONSIDERATIONS THE PANEL TURNED TO THE IDENTIFICATION OF CONCRETE CRITERIA THAT ARE IMPORTANT TO FEDERAL AGENCY DECISIONMAKERS, (I.E., THE CRITERIA USED TO DETERMINE WHAT DECISIONS WILL BE MADE ABOUT). THESE CRITERIA INCLUDE COMPLIANCE, INFLUENCING OTHER ORGANIZATIONS, INSTITUTIONALIZATION OF PROGRAMS, AND, PERHAPS, PROGRAM COSTS. THE PANEL FOUND IT INTERESTING THAT FEW EVALUATORS ARE INVOLVED IN PROVIDING INFORMATION RELEVANT TO THESE CRITERIA. PANELISTS ALSO CONSIDERED WAYS OF AVOIDING THE ERROR OF SHAPING CRITERIA ACCORDING TO WHAT HOLDS PROMISE OF PROVING SUCCESSFUL (I.E., FITTING CRITERIA TO SUCCESSES). THE PANEL AGREED THAT AN EVALUATION IS JUSTIFIABLE ONLY WHEN IT PRODUCES DECISION-RELEVANT INFORMATION. PANELISTS ALSO NOTED THAT A GREAT MANY DECISIONS REQUIRE ONLY SIMPLE, DESCRIPTIVE INFORMATION ABOUT WHAT IS HAPPENING AND WHAT IS RELATED TO WHAT--INFORMATION THAT IS RELATIVELY EASY TO PROVIDE. THE MORE COMPLEX QUESTION OF WHY SOMETHING IS HAPPENING IS RARELY POSED. (LKM)