CONTRASTING SOCIAL EXPERIMENTATION WITH RETROSPECTIVE EVALUATION - A HEALTH CARE PERSPECTIVE

• NCJ Number: 53841
• Journal: PUBLIC POLICY Volume: 23 Issue: 2 Dated: (SPRING 1975) Pages: 241-257
• Author(s): N P ROOS
• Date Published: 1975
• Length: 17 pages
• Grant Number(s): 607-1001-22
• Corporate Author: Harvard University Press
• Publication Format: Article
• Language: English
• Country: United States of America

Annotation

THE STRENGTHS AND WEAKNESSES OF SOCIAL EXPERIMENTATION AND RETROSPECTIVE EVALUATION ARE COMPARED, AND THE ROLES EACH APPROACH CAN PLAY IN EVALUATION ARE DISCUSSED.

Abstract

SOCIAL EXPERIMENTATION PROVIDES AN EXPERIMENTAL AND EVALUATION DESIGN IN THE IMPLEMENTATION AND OPERATION OF PROGRAMS. RETROSPECTIVE EVALUATION DEVELOPS AN EVALUATION DESIGN INDEPENDENT OF A PROGRAM'S OPERATION, USUALLY SUBSEQUENT TO A PROGRAM'S IMPLEMENTATION AND OPERATION FOR A PERIOD OF TIME. THESE METHODS ARE COMPARED IN TERMS OF THEIR CONTRIBUTIONS TO INTERNAL VALIDITY (PRECISE MEASURE OF THE PROGRAM'S IMPACT) AND EXTERNAL VALIDITY (MEASURE OF THE PROGRAM'S REPLICABILITY). SOCIAL EXPERIMENTATION CONTRIBUTES TO INTERNAL VALIDITY BECAUSE IT CAN STRUCTURE THE DESIGN OF A TRUE EXPERIMENT (THE USE OF CONTROL AND EXPERIMENTAL GROUPS) INTO THE PROJECT FROM THE OUTSET. IT CAN INCLUDE PROVISION FOR RANDOMIZATION BY CONTROLLING THE SELECTION OF PROGRAM PARTICIPANTS AND SUBJECTS FOR THE CONTROL GROUP. EXTERNAL VALIDITY IS NOT SERVED AS WELL BY SOCIAL EXPERIMENTATION, BECAUSE OF THE NARROW BACKGROUND AND SITE CHARACTERISTICS INVOLVED IN EFFORTS TO ASSURE INTERNAL VALIDITY. ON THE OTHER HAND, RETROSPECTIVE EVALUATION IS UNABLE TO ASSURE INTERNAL VALIDITY BECAUSE OF THE LIMITED CONTROL OF THE EVALUATION PROCESS OVER DATA GATHERING AND EXPERIMENTAL DESIGN IN THE OPERATION OF THE PROGRAM. EXTERNAL VALIDITY , HOWEVER IS SERVED BECAUSE OF THE HETEROGENEITY CUSTOMARILY ASSOCIATED WITH THE SITES AND SUBJECTS INVOLVED IN THE PROGRAM. IT IS CONCLUDED THAT SOCIAL EXPERIMENTATION, WHEN APPROPRIATE SHOULD BE THE EVALUATION METHOD CHOSEN. RETROSPECTIVE EVALUATION, HOWEVER, CAN BE EMPLOYED MORE FREQUENTLY GIVEN THE NATURE OF THE POLITICAL DECISIONMAKING PROCESS USUALLY ASSOCIATED WITH NEW SOCIAL PROJECTS. IT IS ALSO LESS EXPENSIVE THAN THE SOCIAL EXPERIMENTATION METHOD. THE USE OF THE TWO METHODS IN A COMPLEMENTARY WAY IS ALSO DISCUSSED. (RCB)