NCJ Number
191736
Journal
Journal of Drug Issues Volume: 31 Issue: 3 Dated: Summer 2001 Pages: 643-664
Date Published
2001
Length
22 pages
Annotation
This article examines the problem of comparing results from three different surveys, all funded by the Federal government.
Abstract
The three surveys are: Monitoring the Future (MTF), funded by the National Institute of Drug Abuse; the Youth Risk Behavior Survey (YRBS), funded by the Centers for Disease Control; and the National Household Survey on Drug Abuse (NHSDA), funded by the National Institute on Drug Abuse and the Substance Abuse and Mental Health Services Administration (SAMHSA). Although these surveys have a variety of purposes, they all collect data about substance use, and they include samples of teenagers. It is in those areas of overlap that they provide an opportunity to examine the comparability of results from surveys and study the reasons why they produce similar or different results. There are three different kinds of results of interest: point estimates, group comparisons, and trends. Because the designs of the three surveys are different, analyses must be focused on those results that are comparable: school enrollment, grade levels, substance use, and current use. The core lessons from the analyses are that it is very difficult to compare results from independent surveys and it is difficult to reach firm conclusions about why survey results differ, when they do differ. The three reasons for this are dissimilar statistics, numerous differences in methods, and lack of good studies. There are many ways in which methodology can affect the final results of a study. More information about the implications of factors such as nonresponse, question wording, and mode of data collection is necessary to make informed decisions when designing and executing surveys. If there were a commitment to invest a portion of survey budgets in studies of methods that are used, to assess the significance of some of the decisions that are made and how they affect comparability with other decisions that might have been made, a knowledge base would be built that would make comparisons of survey results easier. References