NCJ Number
62496
Journal
Evaluation and Program Planning Volume: 2 Issue: 2 Dated: (1979) Pages: 137-142
Date Published
1979
Length
6 pages
Annotation
A METHOD IS PRESENTED FOR ENGINEERING THE NECESSARY LEVELS OF MEASUREMENT RELIABILITY TO EVALUATING ONGOING PROGRAMS.
Abstract
TWO STUDIES OF LEVELS OF CLIENT FUNCTIONING AT A COMMUNITY MENTAL HEALTH CENTER (CHMC), IN WHICH AN OUTCOME MEASURE, THE GLOBAL ASSESSMENT SCALE (GAS), DID NOT PERFORM AS EXPECTED, DREW ATTENTION TO THE NEED FOR BETTER CONTROL OF OUTCOME MEASURE RELIABILITIES. DRAWING FROM GENERALIZABILITY THEORY, A STUDY WAS CONDUCTED OF THREE SOURCES OF GAS SCORE VARIANCE--CLIENTS, RATERS, AND TRAINING IN THE USE OF THE SCALE. SEVERAL ESTIMATES OF RELIABILITY (ER'S) WERE DEVELOPED, DEPENDING ON THE MANNER IN WHICH THE GAS RATINGS WERE, OR WOULD BE, OBTAINED IN THE CHMC. IN THE TRAINING STUDY, THE ER IS THE RATIO OF VARIANCE ACROSS CLIENTS TO THE OBSERVED SCORE VARIANCE. THE DIFFERENCES AMONG THESE ER'S CLARIFIED WHY THE GAS HAD LOWER RELIABILITY WHEN USED IN THE CHMC SETTING. THE APPLICATION OF GENERALIZABILITY THEORY TO HARNESS OUTCOME MEASURES IS NOT LIMITED TO CHMC'S OR THE GAS. . THIS THEORY CAN ALSO BE APPLIED IN OTHER SETTINGS TO SELECT APPROPRIATE DATA COLLECTION STRATEGIES. TWO HYPOTHETICAL EXAMPLES ARE DESCRIBED TO ILLUSTRATE THESE OTHER APPLICATIONS. HOWEVER, ESTIMATING OUTCOME RELIABILITIES WILL NOT NECESSARILY IMPROVE OUTCOME MEASURE VALIDITIES. TABULAR DATA, REFERENCE NOTES, AND REFERENCES ARE INCLUDED. (AUTHOR ABSTRACT MODIFIED--PRG)