STANDARD 4-4-1: Any survey stage
of data collection with a unit or item response rate less than 85
percent must be evaluated for the potential
magnitude of nonresponse bias before the data or any analysis using
the data may be released. (See Standard 1-3 for how to calculate unit
and item response rates.) Estimates of survey characteristics for nonrespondents
and respondents are required to assess the potential nonresponse bias.
The level of effort required is guided by the magnitude of the nonresponse.

STANDARD 4-4-2: When unit nonresponse
is high, nonresponse bias analysis
must be conducted at the unit level to determine whether or not the
data are missing at random and to assess the potential magnitude of
unit nonresponse bias. At the unit level, the nonresponse bias analysis
must be conducted using base weights for the survey stage with nonresponse.
The following guidelines must be considered in such analysis.

GUIDELINE 4-4-2A: Comparisons of respondents and nonrespondents across
subgroups using available sample frame characteristics provide information
about the presence of nonresponse
bias. This approach is limited because observed frame characteristics
are often unrelated or weakly related to more substantive items in the
survey.

GUIDELINE 4-4-2B: Formal multivariate modeling can be used to compare
the proportional distribution of characteristics of respondents and
nonrespondents to determine if nonresponse
bias exists and, if so, to estimate the magnitude of the bias. These
multivariate analyses are used to identify the characteristics of cases
least likely to respond to an interview (such analyses are often referred
to as nonresponse propensity models). Cases are coded as either responding
to or not responding to the interviews and multivariate techniques are
used to identify which case characteristics significantly relate to
unit nonresponse. The predictor variables
should have very high response rates.
This approach may be limited by the extent to which such predictors
exist in the data.

GUIDELINE 4-4-2C: Comparisons of respondents to known population characteristics
from external sources can provide information about how the respondents
differ from a known population. This approach is limited by information
available from existing sources on the population of interest. Known
population characteristics are often unrelated or weakly related to
more substantive items in the survey.

GUIDELINE 4-4-2D: For collections in which successive levels of effort
(e.g., increasing number of contact attempts, increasing incentives
to respond) are employed to reduce nonresponse, comparisons of characteristics
can be made between the later/more difficult cases and the earlier/easier
cases to estimate the characteristics of the remaining nonrespondents.
This approach may be less effective if overall or total response rates
are relatively low or if a collection period is relatively short in
duration. In addition, the assumption that nonrespondents are like those
respondents who are difficult to reach may not hold.

GUIDELINE 4-4-2E: More intensive methods and/or incentives can be used
to conduct a followup survey of nonrespondents
on a reduced set of required response
items. Comparisons between the nonrespondent followup survey and
the original survey can be made to measure the potential magnitude of
nonresponse bias in the original
survey. This approach may be costly and less useful for modeling nonresponse
bias if the nonrespondent followup survey response
rates are also below 70 percent.

GUIDELINE 4-4-2F: The estimated bias can be summarized using the following
measures. One measure is the ratio of the bias to the standard error,
using the base weight. A second measure
is the ratio of the bias to the reported survey mean, using the base
weight. If weighting adjustments are used to reduce bias, these
measures should also be reported using the final weighted estimates.

STANDARD 4-4-3: When item nonresponse
is high, nonresponse bias analysis
must be conducted at the item level to determine whether or not the
data are missing at random and to assess the potential magnitude of
item nonresponse. To analyze potential bias from item nonresponse, the
guidelines below must be considered.

GUIDELINE 4-4-3A: For an item with a low total response rate, respondents
and nonrespondents can be compared on sampling frame
and/or questionnaire variables for which data on respondents and nonrespondents
are available. Base weights must be used in such analysis. Comparison
items should have very high response
rates. This approach may be limited to the extent that items available
for respondents and nonrespondents may not be related to the low response
rate item being analyzed.

GUIDELINE 4-4-3B: Formal multivariate modeling can be used to compare
characteristics of respondents and nonrespondents to determine if nonresponse
bias exists and, if so, to estimate the magnitude of the bias. These
multivariate analyses are used to identify the characteristics of cases
least likely to respond to an item (such analyses are often referred
to as nonresponse propensity models). Cases are coded as either responding
to or not responding to the item and multivariate techniques are used
to identify which case characteristics significantly relate to item
nonresponse. Base weights must be used in such analysis. The predictor
variables should have very high response
rates. This approach may be limited by the extent to which such
predictors exist in the data.

GUIDELINE 4-4-3C: If the overall response rate is acceptable, nonresponse
bias analysis may be conducted using data from survey respondents
only. Unit level respondents who answered the low response rate item
can be compared to unit level respondents who did not answer the item.
Final weights and unimputed variables should be used in such an analysis.
The comparison items should have very high item response
rates. This approach may be limited because it does not directly
analyze nonresponse bias that
may originate because of unit level nonresponse.