RANDOMIZED AND QUASI-EXPERIMENTAL DESIGNS IN EVALUATION RESEARCH - AN INTRODUCTION (FROM EVALUATION RESEARCH METHODS - A BASIC GUIDE, 1977, BY LEONARD RUTMAN - SEE NCJ-49434)

• NCJ Number: 49439
• Author(s): T D COOK; F L COOK; M M MARK
• Date Published: 1977
• Length: 38 pages
• Corporate Author: Sage Publications, Inc
• Publication Format: Document
• Publication Type: Report (Study/Research)
• Language: English
• Country: United States of America

Annotation

METHODS ARE SUGGESTED FOR DECREASING THE AMBIGUITY OF CAUSAL INFERENCES IN EVALUATION RESEARCH. THE FOCUS IS ON DESIGN OF SUMMATIVE EVALUATIONS, INCLUDING TRUE AND QUASI-EXPERIMENTS.

Abstract

EXPERIMENTS CAN BE BROKEN INTO TWO MAJOR CLASSES: (1) TRUE EXPERIMENTS, IN WHICH EXPERIMENTAL UNITS ARE RANDOMLY ASSIGNED TO TREATMENTS; AND (2) QUASIEXPERIMENTS, IN WHICH ASSIGNMENTS TO TREATMENT OCCUR IN SOME NONRANDOM FASHION. FOUR KINDS OF VALIDITY ARE DISCUSSED WHICH AFFECT EXPERIMENTATION INCLUDING INTERNAL, STATISTICAL CONCLUSION, CONSTRUCT, AND EXTERNAL; SUGGESTIONS ARE GIVEN FOR MAXIMIZING THE EXTENT OF EACH. RANDOMIZATION OR TRUE EXPERIMENTATION, ASSURES REPRESENTATIVENESS AND COMPARABILITY IN RESEARCH. HOWEVER, SEVERAL PROBLEMS INTERFERE WITH ITS USE: (1) THE PROGRAM PLANNERS AND STAFF MAY RESIST RANDOMIZATION AS A MEANS OF ALLOCATING TREATMENTS, ARGUING FOR SOME OTHER ALLOCATIVE CRITERION SUCH AS NEED OR MERIT; (2) THE RANDOMIZATIN PROCESS MAY NOT BE CORRECTLY CARRIED OUT, RESULTING IN NONEQUIVALENT EXPERIMENTAL GROUPS; (3) THE RANDOMIZATION MAY BREAK DOWN BECAUSE SOME PEOPLE, MIGHT REFUSE TO PARTICIPATE IN OR MAY DROP OUT OF THE DIFFERENT TREATMENT GROUPS; AND (4) RANDOMIZATION CREATES A FOCUSED INEQUITY BECAUSE SOME PERSONS RECEIVE TREATMENTS THAT ARE MORE DESIREABLE THAN OTHERS, AND THIS INEQUITY CAN CAUSE REACTIONS THAT MAY BE FALSELY INTERPRETED AS TREATMENT EFFECTS. ALTHOUGH TRUE EXPERIMENTS ARE PREFERRED, IF THE RESISTANCE TO RANDOMIZATION IS TOO STRONG, A QUASI-EXPERIMENT MUST BE USED. PARTICULAR DESIGN FEATUES WHICH CAN STRENGTHEN CAUSAL INFERENCE FOR QUASI-EXPERIMENTS ARE DISCUSSED. THESE INCLUDE PRETESTING, USE OF A NO-TREATMENT CONTROL GROUP OR A COHORT CONTROL GROUP, INTRODUCTION OF NONEQUIVALENT DEPENDENT VARIABLES, APPLICATION OF REMOVED OR REPEATED TREATMENT, AND IMPLEMENTATION OF SIMPLE AND COMPLEX INTERRUPTED-TIME-SERIES DESIGNS. BECAUSE THEY ALLOW MORE CONFIDENT CAUSAL INFERENCES, TRUE EXPERIMENTS OR STRONG QUASI-QUASI-EXPERIMENTS ARE RECOMMENDED AS OPPOSED TO PASSIVE CORRELATIONAL TECHNIQUES. REFERENCES ARE PROVIDED. (DAG)