FEET OF CLAY IN HOBNAIL BOOTS - AN ASSESSMENT OF STATISTICAL INFERENCE IN APPLIED RESEARCH (FROM EVALUATION STUDIES - REVIEW ANNUAL, V 3, 1978 BY THOMAS D COOK ET AL - SEE NCJ-51598)

• NCJ Number: 51603
• Author(s): R A BERK; M BREWER
• Date Published: 1978
• Length: 25 pages
• Corporate Author: Sage Publications, Inc
• Publication Format: Document
• Publication Type: Report (Study/Research)
• Language: English
• Country: United States of America

Annotation

A DIDACTIC REVIEW OF ABUSES OF STATISTICAL INFERENCE IS PRESENTED, AND RATIONALES FOR THE APPROPRIATE APPLICATION OF STATISTICAL INFERENCE TECHNIQUES IN EVALUATION RESEARCH ARE DISCUSSED.

Abstract

CLASSICAL STATISTICAL INFERENCE IS FUNDAMENTALLY BASED ON THE CONCEPT OF LONG-TERM OUTCOMES, THE PROPORTION OF TIMES OVER THE LONG RUN THAT CERTAIN CHANCE EVENTS WILL OCCUR. ONE OF THE MOST BASIC MISUNDERSTANDINGS SURROUNDING STATISTICAL INFERENCE IS THE COMMON ASSUMPTION THAT THE OVERALL VALIDITY OF ONE'S FINDINGS IS ASSURED ONCE STATISTICAL SIGNIFICANCE HAS BEEN ACHIEVED. IN THE ABSENCE OF NUMEROUS INDEPENDENT (TYPICALLY HYPOTHETICAL) REPLICATIONS OF A STOCHASTIC PROCESS, CLASSICAL INFERENCE IS MEANINGLESS. THE RANDOM SAMPLING MODEL IS OF LIMITED VALUE IN MANY PRACTICAL SITUATIONS. IF INFERENCES ARE TO BE DRAWN FROM A SINGLE SAMPLE IN TERMS OF CONFIDENCE ABOUT PARTICULAR ESTIMATES, IT IS BEST TO RELY FORMALLY ON BAYESIAN PROCEDURES. THE RANDOM ASSIGNMENT MODEL ADDRESSES THE DEGREE TO WHICH CHANCE IS RULED OUT AS AN EXPLANATION FOR OBSERVED ASSOCIATIONS BETWEEN THE ASSIGNMENT OF INDIVIDUALS TO GROUPS AND THEIR SCORES ON AN OUTCOME MEASURE OF INTEREST. LIKE THE RANDOM SAMPLING MODEL, HOWEVER, CHANCE FACTORS FORMALLY ENTER AS A CONSEQUENCE OF ACTIONS UNDERTAKEN BY THE RESEARCHER. THE RANDOMIZED COMPONENT MODEL APPLIES TO SITUATIONS IN WHICH NEITHER RANDOM SAMPLING NOR RANDOMIZATION OCCUR BUT WHERE AN OBSERVED RESEARCH POPULATION IS SORTED INTO SUBGROUPS BASED ON NATURALLY OCCURRING PROCESSES AND THE QUESTION IS WHETHER THERE IS SOME SUBSTANTIVE ASSOCIATION BETWEEN SUBGROUP MEMBERSHIP AND SCORES ON SOME OTHER MEASURE OF INTEREST. BOTH THE RANDOM ASSIGNMENT AND THE RANDOMIZED COMPONENT MODELS ARE VULNERABLE TO ALL STANDARD ABUSES OF THE RANDOM SAMPLING MODEL. THERE ARE DEFICIENCIES IN ECONOMETRIC MODELS, AND THE TYPICAL TIME SERIES MODEL IS ONE TYPE OF ECONOMETRIC MODEL. THE USE OF STATISTICAL INFERENCE MODELS IN APPLIED RESEARCH IS EVALUATED. NOTES AND REFERENCES ARE PROVIDED. (DEP)