NCJ Number
47349
Journal
EVALUATION Volume: 4 Dated: (1977) Pages: 161-164
Date Published
1978
Length
4 pages
Annotation
EXAMPLES OF MISUSE OF EVALUATION STUDY FINDINGS IN PUBLIC DEBATE AND DECISIONMAKING ARE CITED, AND WAYS OF AVOIDING SUCH MISUSE ARE SUGGESTED.
Abstract
EVALUATION FINDINGS MAY BE MISUSED IN A NUMBER OF DIFFERENT WAYS, SOME OF WHICH MAY OR MAY NOT ACTUALLY CONSTITUTE DELIBERATE MUSUSE. THE PROBLEMS THAT CAN ARISE WHEN EVALUATION FINDINGS ARE DISTORTED OR OTHERWISE MISUSED ARE ILLUSTRATED IN THE FOLLOWING STUDIES: A 1974 STUDY ON THE EFFECTS OF JURY SIZE ON TRIAL OUTCOMES; A STUDY CONCERNED WITH THE EFFECTS OF RACIAL COMPOSITION OF CLASSROOMS ON EDUCATIONAL ACHIEVEMENT; AN EVALUATION OF THE EFFECTS OF THE 'SESAME STREET' TELEVISION PROGRAM FOR CHILDREN; THE SURGEON GENERAL'S REPORT ON TELEVISION AND VIOLENCE; AND NEW JERSEY'S NEGATIVE INCOME TAX EXPERIMENT. MAJOR SIGNS COMMONLY ASSOCIATED WITH MISUSE OF EVALUATION FINDINGS INCLUDE THE FOLLOWING: (1) ACTIONS ARE TAKEN BASED UPON ACCEPTING THE NULL HYPOTHESIS OF THE STUDY; (2) NO CAPACITY FOR ADEQUATE TECHNICAL REVIEW OF SOCIAL SCIENCE METHODOLOGIES EXISTS IN THE POTENTIAL USER ORGANIZATION, AND NO OUTSIDE REVIEWS ARE COMMISSIONED; (3) NO ATTEMPT IS MADE BY EVALUATORS OR USERS TO ASSESS THE POSSIBLE NEGATIVE EFFECTS OF A PROGRAM; (4) THERE IS LOW CORRESPONDENCE BETWEEN RESEARCH OPERATIONS AND THE VERBAL LABELS INTENDED TO DESCRIBE THEM; (5) A VERSION OF THE RESULTS IS DISSEMINATED BEFORE A FINAL EVALUATION REPORT IS READY; AND (6) INFERENCE IS BASED ON A SINGLE EVALUATION. THE BEST WAY TO PREVENT SUCH MISUSE MAY BE TO HAVE IMPORTANT PROGRAMS EVALUATED INDEPENDENTLY BY MULTIPLE INVESTIGATORS. (LKM)