As a summary statistic, the weighted total or aggregate income is appealing for its simplicity and its use of all the income data collected by each survey, but its value is heavily dependent on the amount of income captured from the upper end of the income distribution, which holds the least interest for policy analysis. In presenting estimates of aggregate income, we include a breakdown by quintile, which enables us to compare the surveys with respect to their collection of income from different segments of the distribution. We also examine per capita income, which is calculated by dividing the estimate of aggregate income by population size. This corrects the aggregate estimates for slight differences in the size of the population represented by each survey after the adjustment to a common universe described in Chapter III. The estimates of per capita income are presented by quintile as well.

Estimates of aggregate income, for the whole population and broken down by quintile of family income, are presented in Table IV.1 for the five general population surveys. In addition to the dollar amounts, the table presents the estimated amounts as a percentage of the corresponding amounts for the CPS. While the CPS does not represent the gold standard for estimates of income, and we do not mean to suggest that the CPS estimates are the best, the CPS is the official source of household income and poverty statistics for the U.S., so expressing other survey estimates of income as a percentage of the CPS provides a useful standardization.

Table IV.1

AGGREGATE INCOME BY QUINTILE OF FAMILY INCOME: FIVE SURVEYS

Income Estimate

CPS

ACS

SIPP

MEPS

NHIS

Billions of Dollars

Aggregate Income, All Persons

6,468.4

6,346.3

5,766.2

6,257.7

6,116.2

Family Income Quintile

Lowest

370.5

368.7

391.4

360.0

313.7

Second

774.1

778.4

750.8

808.4

717.7

Third

1,090.2

1,087.4

1,008.8

1,144.7

1,058.4

Fourth

1,446.8

1,415.8

1,307.2

1,461.8

1,420.7

Highest

2,786.7

2,696.0

2,308.0

2,483.0

2,605.8

Sum through Four Quintiles

3,681.7

3,650.3

3,458.2

3,774.7

3,510.4

AGGREGATE INCOME BY QUINTILE OF FAMILY INCOME: FIVE SURVEYS (continued)

Income Estimate

CPS

ACS

SIPP

MEPS

NHIS

Percent of CPS

Aggregate Income, All Persons

100

98.1

89.1

96.7

94.6

Family Income Quintile

Lowest

100

99.5

105.6

97.2

84.7

Second

100

100.6

97

104.4

92.7

Third

100

99.7

92.5

105

97.1

Fourth

100

97.9

90.3

101

98.2

Highest

100

96.7

82.8

89.1

93.5

Sum through Four Quintiles

100

99.1

93.9

102.5

95.3

Source: Mathematica Policy Research, from tabulations of calendar year 2002 income from the 2003 CPS ASEC supplement, the 2001 SIPP panel, the 2002 Full-year Consolidated MEPS-HC, and the 2003 NHIS, and prior 12 months income, inflation-adjusted to calendar year 2002, from the 2002 ACS.

Aggregate income ranges from $5.77 trillion in the SIPP to $6.47 trillion in the CPS—a difference of nearly 11 percent. The other three surveys produce estimates that lie within 2 to 5 percent of the CPS. Aggregate income is $6.35 trillion in the ACS, $6.26 trillion in MEPS, and $6.12 trillion in NHIS. Aggregates in the top quintile may be affected by outliers and by differences in survey practice with respect to the topcoding of public use data, documented in Chapter II. For example, the CPS assigns the means of topcoded values as their respective topcodes, which preserves overall means and totals, but not all surveys do this for all income items. For this reason, we summed the survey aggregates through the bottom four quintiles.25 For every survey, the four-quintile sum is closer to the CPS estimate than is the full aggregate, with the MEPS total exceeding the CPS by 2.5 percent. The SIPP total moves to within 1.5 percent of the NHIS total but is still 6 percent below the CPS.

When we examine the results by quintile of family income, we find that SIPP obtains the most income from the lowest quintile, at 105.6 percent of the CPS total. SIPP’s apparent success in collecting income data from the low end of the income distribution begins to erode noticeably by the second quintile, however. In that quintile, SIPP collects 97 percent as much total income as the CPS. This drops to 92.5 percent by the third quintile, 90.3 percent by the fourth and 82.8 percent in the top quintile. MEPS aggregates exceed the corresponding CPS amounts for quintiles two through four while the ACS aggregates lie within a percent of the CPS aggregates (both above and below) through the first three quintiles before dropping to 98 and 97 percent of the CPS in the fourth and fifth quintiles.

This is only the first of numerous tables, and it examines only one dimension of income, but it presents several striking findings that raise fundamental questions about the collection of income data. One such finding is that with a single question NHIS captures 95 percent as much total income as the CPS, despite the latter’s sizable battery of income questions and its status as the official source of income and poverty estimates for the U.S. Second, with far more income questions than any of the other four surveys, SIPP captures 11 percent less total income than the CPS and 6 percent less than the NHIS’s single question. Third, with its massive sample size and an instrument that is filled out primarily by respondents working without the assistance of a trained interviewer, the ACS nevertheless manages to approximate the CPS more closely than any other survey. Fourth, the MEPS person weights used to prepare the estimates in Table IV.1 were post-stratified to CPS totals by demographic characteristics and the distribution of income relative to poverty. What impact does this have on the MEPS estimates of aggregate income? Would MEPS, with its SIPP-like panel design, yield SIPP-like income estimates in the absence of this post-stratification, or does the use of retrospective annual versus monthly income questions trump the panel design?

More generally, what do these findings say about the collection of income data? Does the strategy of asking respondents about their incomes over the prior calendar year or even the past twelve months have a bigger impact on the amount of income collected than the level of detail that is incorporated into the questions? It will become clear as we progress through this chapter that the limitations of a single-question approach are indeed numerous, but this is a separate issue from the retrospective approach. We also have to ask if the SIPP approach of collecting income at four-month intervals and compiling annual totals month by month is inherently inferior, or whether the other surveys share a common upward bias that arises from their retrospective approach. These are compelling questions, and as we walk through the rest of the findings in this chapter it will become apparent that there are areas in which SIPP clearly excels. Nevertheless, we will also see that outside of these exceptions, SIPP’s estimates of income are consistently low.

The boundaries between quintiles (that is, the dollar values of the 20th, 40th, 60th, and 80th percentiles) are themselves informative about the distribution of total family income in each of the surveys. These percentile points are rather similar for the CPS, ACS, and MEPS, but the SIPP quintile boundaries start above the CPS and decline progressively from there (Table IV.2). The NHIS boundaries remain at 92 to 93 percent of the CPS values through the 60th percentile but then rise to nearly 98 percent for the 80th percentile.

The ratio of the 80th to the 20th percentile provides a measure of inequality across the income distribution. The higher the ratio, the more unequally family income is distributed. Given the similarity of their quintile values, the ratios for the CPS, ACS and MEPS are very similar as well. Ratios for the latter two surveys are 97 percent of the CPS ratio of 4.56. The SIPP ratio is much lower at 3.96 or 87 percent of the CPS ratio, reflecting the progressive decline of the SIPP quintiles relative to the CPS values. The NHIS ratio, however, is 6 percent higher than the CPS at 4.83 because the 80th percentile in the NHIS income distribution is relatively higher than the 20th percentile when compared to the CPS.

We obtain similar but more complex findings if we compare per capita income by quintile across the five surveys. Using the ratio of per capita incomes between the top and bottom quintiles as our measure of income dispersion, we find that ACS is just two percentage points below the CPS with a ratio of 7.44 versus 7.57 (Table IV.3). MEPS is now markedly lower with a ratio of 6.90 or 91 percent of the CPS value. SIPP continues to have the lowest ratio at 5.90 or only 78 percent of the CPS ratio. By contrast, the NHIS ratio of 8.34 is 10 percent above the CPS ratio.

Source: Mathematica Policy Research, from tabulations of calendar year 2002 income from the 2003 CPS ASEC supplement, the 2001 SIPP panel, the 2002 Full-year Consolidated MEPS-HC, and the NHIS, and prior 12 months income, inflation-adjusted to calendar year 2002, from the 2002 ACS.

Like aggregate income, per capita income in the top quintile is affected by outliers and topcoding, so we also calculated the ratio of per capita incomes between the fourth and lowest quintiles. Here the patterns are more similar to what we saw with the ratio of the 80th to the 20th percentile, yet there are notable differences. First, the NHIS ratio exceeds the CPS ratio by an even larger amount, being 16 percent higher at 4.55 versus 3.93 for the CPS. In all cases the NHIS results are driven by a very low per capita income in the bottom quintile (and a low 20th percentile). Large ratios result despite the fact that the upper quintiles and percentiles never match the CPS. The MEPS ratio is also higher than the CPS ratio in this case—by 4 percent. The ACS ratio is 99 percent of the CPS ratio, while the SIPP ratio is 85 percent of the CPS ratio.

Overall, then, we see somewhat greater inequality in the income distribution in the NHIS than the CPS and lower inequality in the SIPP. The ACS matches the CPS very closely while the estimates for MEPS show less, about the same, or more inequality than the CPS depending on the ratio we calculate.

To assess the reporting of income in the PSID in comparison with other surveys, we replicated the tables above for the PSID and the three Census Bureau surveys. As we explained in Chapter III, the application of preliminary cross-sectional weights to the PSID yields an estimated population that falls short of the CPS population by 21 million. In part this is due to an omission of unrelated subfamilies and secondary individuals from the CPS-based control totals that were used to post-stratify the PSID weights. In addition to reducing the weighted number of persons, this omission from the PSID is likely to have an effect on the distribution of income because singles—who tend to have lower income than other family units—will be underestimated relative to the CPS. Therefore, we created an additional CPS series, labeled CPS-X in the tables, that excludes unrelated subfamilies and all secondary individuals except those who were identified as unmarried partners. In creating CPS-like families from the PSID families with unmarried partners, we separated the unmarried partners into their own families. We needed their counterparts in the CPS.

Despite 21 million fewer persons, as we noted, the PSID captures 3.9 percent more aggregate income than the CPS, or an additional $253.4 billion dollars (Table IV.4). Compared to the CPS-X series with the aforementioned exclusions, the PSID captures an additional $416.5 billion. The PSID also captures more aggregate income than the full CPS in every quintile, with the biggest difference in the top quintile, where the PSID aggregate is 105.5 percent of the full CPS aggregate.

PSID quintile boundaries are also higher than the quintiles from the full CPS, CPS-X, ACS, or SIPP. The biggest difference occurs at the 20th percentile, where the PSID value of $24,200 exceeds the corresponding full CPS value by $4,200 (or 21 percent) and exceeds the corresponding CPS-X value by $3,300 (Table IV.5). At higher percentiles, the PSID values exceed the corresponding full CPS values by 10 to 13 percent. The ratio of the 80th to the 20th percentile is 8 percent lower than that of the full CPS because the PSID exceeds the CPS by a smaller margin at the 80th percentile than the 20th percentile.

Because the PSID obtains more aggregate income than the CPS from a smaller weighted population, the differences in per capita income are even greater than the differences in aggregate income. The overall per capita income in the PSID, $25,710 is 12 percent higher than both the full CPS and CPS-X per capita incomes (Table IV.6). By quintile the differences grow from 10 percent in the lowest quintile to 14 percent in the highest quintile. Ratios of per capita income between quintiles are only slightly higher than the corresponding CPS ratios, implying that inequality across the income distribution is about the same in the two surveys.

Source: Mathematica Policy Research, from tabulations of calendar year 2002 income from the 2003 CPS ASEC supplement, the 2001 SIPP panel, and the 2003 PSID, and prior 12 months income, inflation-adjusted to calendar year 2002, from the 2002 ACS.

1.The CPS-X estimates exclude all unrelated subfamilies and most secondary individuals (except unmarried partners of the householder) to mimic the population controls applied to the PSID.

Source: Mathematica Policy Research, from tabulations of calendar year 2002 income from the 2003 CPS ASEC supplement, the 2001 SIPP panel, and the 2003 PSID, and prior 12 months income, inflation-adjusted to calendar year 2002, from the 2002 ACS.

1.The CPS-X estimates exclude all unrelated subfamilies and most secondary individuals (except unmarried partners of the householder) to mimic the population controls applied to the PSID.

Source: Mathematica Policy Research, from tabulations of calendar year 2002 income from the 2003 CPS ASEC supplement, the 2001 SIPP panel, and the 2003 PSID, and prior 12 months income, inflation-adjusted to calendar year 2002, from the 2002 ACS.

1.The CPS-X estimates exclude all unrelated subfamilies and most secondary individuals (except unmarried partners of the householder) to mimic the population controls applied to the PSID.

Does the PSID truly capture more income than the CPS or does the PSID sample with its current weights simply overrepresent higher income families? We cannot answer this with the data available to us. We compared distributions of selected characteristics between the PSID and the CPS and found that the PSID had proportionately fewer Hispanics and blacks and slightly more persons with college degrees, but the PSID also had proportionately more persons with less than a high school education, so the comparison was inconclusive.26 Our conclusion at this point is that incomes in the PSID appear to run higher than in any of the other surveys, but given the nature of the PSID sample, this could easily be due to the PSID being less representative of the U.S. population as a whole than the Census Bureau surveys.

Survey Disclaimer

According to the Paperwork Reduction Act of 1995, no persons are required to respond to a collection of information unless it displays a valid OMB control number. The valid OMB control number for this information collection is 0990-0379. The time required to complete this information collection is estimated to average 5 minutes per response, including the time to review instructions, search existing data resources, gather the data needed, and complete and review the information collection. If you have comments concerning the accuracy of the time estimate(s) or suggestions for improving this form, please write to: U.S. Department of Health & Human Services, OS/OCIO/PRA, 200 Independence Ave., S.W., Suite 336-E, Washington D.C. 20201, Attention: PRA Reports Clearance Officer.