Since the end of the 2001 recession, the U.S. economy has performed pretty well in terms of output growth, averaging about 3-1/4 percent a year. But how well has the economy performed in terms of creating jobs?
Since the end of the 2001 recession, the U.S. economy has performed pretty well in terms of output growth, averaging about 3-1/4 percent a year. But how well has the economy performed in terms of creating jobs? To answer that question, most analysts look at two independent monthly estimates of employment published by the Bureau of Labor Statistics (BLS). And the problem is that each sends a different signal about recent labor market conditions. As Figure 1 shows, the so-called payroll survey has been reporting a substantial loss in employment and a slow recovery of labor market conditions, while the so-called household survey indicates much less of a loss and a much faster recovery.
How do these two employment measures differ? Which is a more accurate measure of aggregate employment? This Economic Letter examines the historical and recent behavior of these two employment measures to answer these questions.
The payroll survey estimates the nation’s employment based on responses from a sample of about 400,000 business establishments, which account for about one-third of total nonfarm payroll employment. With a lag of about one year, the BLS revises the payroll estimate to an almost-complete count of U.S. payroll employment; this results in what is known as the “benchmark revision.”
The household survey, in contrast, estimates the nation’s employment based on responses from interviews with approximately 60,000 households; the BLS then inflates the survey data by the most recent estimates of the population. Unlike the payroll survey, the raw household survey data are not revised, but the population estimates used to inflate them are occasionally updated to incorporate new information from censuses and new estimates of immigration.
Beyond these differences, the two employment measures also differ in concept. First, the payroll survey counts the number of jobs, while the household survey counts the number of employed individuals. Therefore, a person with multiple jobs will be counted several times in the payroll survey but only once in the household survey. Second, their scopes are different; while the payroll survey covers only wage and salary workers on nonfarm payrolls, the household survey covers those individuals as well as agricultural workers, the self-employed, workers in private households, unpaid family workers, and workers on unpaid leaves. Finally, payroll employment includes wage and salary workers under the age of 16, while the household survey does not.
Given these significant differences between the two measures, it is not surprising that they produce different estimates for a given month or quarter. Over the long term, however, they should indicate similar paths for employment growth, as their various components should move together with aggregate employment in the long run.
Why, then, have they remained divergent over the last few years? Some have argued that the discrepancy may reflect a tendency for the initial payroll measure to underestimate the number of jobs in the economy, especially in the early stages of economic upturns; for example, Hilsenrath (2003) noted that since the survey covers a fixed set of business establishments, it may miss jobs being created by startup companies. Indeed, the benchmark revision after the 1991 recession did show substantially more jobs in the early months of the recovery. So it is possible that the payroll measure will be revised up to come closer to the measure of the household survey.
This explanation, however, is not, on its face, completely convincing. Figure 2 plots the difference between the initial releases and the revisions for the current recovery (solid thick line), the recovery after the 1990-1991 recession (solid thin line), and the average of the four recoveries before 1990 (dashed line); the shaded area gives the range of all revisions for the four recoveries before 1990, so the top of it is the maximum revision and the bottom is the minimum revision. The difference is indexed, so positive values mean that the revision was higher than the initial release, and negative values mean the revision was lower than the initial release. Clearly, the initial releases of payroll survey do not always underestimate employment in the early phases of the recovery. Although on average the revised level of payroll employment exceeded the initial released level, in some cases the revisions were lower than the initial releases, notably during the recoveries from the 1975 and 1980 recessions. Therefore, the widely cited large upward revision to the payroll data after the 1990-1991 recession might have been a special case.
Moreover, in 1998 the BLS began phasing in a major redesign of the survey aimed at improving the accuracy of the initial releases of payroll employment and reducing the large and systematic benchmark revisions. Therefore, the errors arising from missing job growth in new firms may be much smaller than they were in the recovery following the 1990-1991 recession. As shown by the dotted line in Figure 2, the size of revisions during the current recovery is indeed much smaller than the one after the 1990-1991 recession and is very close to the historical average level during the recoveries following the four recessions before 1990.
A closer look at the household and payroll employment series also suggests that, after adjusting for definitional differences between the two measures, the discrepancy between them decreases substantially. Take September 2003 as an example, when the discrepancy in Figure 1 was the largest. Over the preceding 12 months, the published nonfarm payroll employment fell by 427,000 while the published household employment increased by 261,000. Therefore there is a difference of 688,000 jobs in terms of 12-month employment growth. However, a closer look indicates that the difference mainly comes from two sources: the population updates which led to an increase of 543,000 jobs in the change of household employment, and an increase of 310,000 self-employed jobs. In other words, after excluding the population update and the self-employed from the household survey results, it would show a decline of 592,000 jobs, also indicating a substantial job loss over the preceding 12 months. More thorough studies (for instance, BLS 2004) confirm this “back of the envelope” estimate, finding that, when the household survey is adjusted to match the payroll survey definition more closely, the movements of the two series in the past couple of years do tend to converge.
Which measure is more reliable?
The reconciliation above shows that both employment measures suggested continued weakness in the labor market last year. However, their implications may differ in the long run, and which measure is more appropriate in assessing the labor market remains a judgment call. On the one hand, the household survey has drawbacks that may induce larger biases than the payroll survey. For instance, in the household survey there are a large number of “proxy responses” by household members who may have incomplete information on the employment status of other family members. Moreover, the sample size of household survey is substantially smaller than that of the payroll survey; the larger sample size in the payroll survey will undoubtedly help generate more precise estimates of employment.
On the other hand, the payroll survey has the drawback of missing job growth that is occurring at very young companies, including self-employment, even after the redesign. However, some researchers have pointed out that this drawback may not be so serious. Rissman (2003) treats self-employment as a low-paying alternative to wage work. In particular, workers are assumed to be either employed in wage work, unemployed, or self-employed and looking for wage work. During economic downturns, the likelihood of being laid off rises and the prospect of finding a job offer diminishes, making self-employment relatively more attractive; when the job market becomes more favorable, self-employment becomes less attractive and the number of self-employed persons declines. Therefore, self-employment should be countercyclical. Aaronson et al. (2004) analyze the variations of unemployment and self employment at the state level since 2001 and find that self-employment indeed was countercyclical in the most recent recession: it rose during weak economic times and declined as the wage and salary sector improved. Therefore, increases in self-employment in the early stages of an economic recovery reported in the household survey may indicate a weak rather than an improving labor market, suggesting that the household survey may send a false signal of employment strength.
Tao Wu
Economist
References
[URLs accessed August 2004.]
Aaronson, Daniel, Ellen R. Rissman, and Daniel G. Sullivan. 2004. “Assessing the Jobless Recovery.” Federal Reserve Bank of Chicago, Economic Perspectives 28 (2nd quarter).
Bureau of Labor Statistics. 2004. “Employment from the BLS Household and Payroll Surveys: Summary of Recent Trends.”
Hilsenrath, Jon E. 2003. “Job Gauge Skips Key Group: Start-Ups.” Wall Street Journal, September 22.
Rissman, Ellen R. 2003. “Self-employment as an Alternative to Unemployment.” Federal Reserve Bank of Chicago Working Paper 2003-34 (December).
Opinions expressed in FRBSF Economic Letter do not necessarily reflect the views of the management of the Federal Reserve Bank of San Francisco or of the Board of Governors of the Federal Reserve System. This publication is edited by Anita Todd and Karen Barnes. Permission to reprint portions of articles or whole articles must be obtained in writing. Please send editorial comments and requests for reprint permission to research.library@sf.frb.org