NCJ Number
74034
Date Published
1977
Length
20 pages
Annotation
A series of experiments was performed to determine the effects on survey results of changes in the wording of survey questions.
Abstract
The experiments were based on decisions that face all survey investigators: whether to use agree-disagree statements or forced choice items, whether to ask open or closed questions, whether and how to balance alternatives offered, whether to include a middle alternative, and whether or not to filter for no opinion. The study also examined the consequences of these decisions not only for univariate distributions but also for an item's relationship to the respondent's educational level. The Survey Research Center's national probability samples were used in the conduct of the experiments, which involved formulation of questions regarding the causes of crime, the political role of women, and other subjects. Results indicated that for agree-type items, for open as against closed items, and for some items involving changes in the tone of a word, three-way interactions with education occurred so that researchers would draw different conclusions from different forms of essentially the same question. The assumption of form-resistant correlations with education appeared to be more justified, however, with questions involving formal verus substantive balance and middle alternatives. The no-opinion filter type of question occupied a middle ground. Evidence also indicated that index construction is an inadequate solution to the question wording problem. Results suggested that question form effects occur often enough that researchers need to be wary of correlations based entirely on a single question form. Tables, a graph, notes, and 19 references are included. (Author abstract modified).