Searching for Value in the U.S. Stock Market


Kevin J. Lansing

Download PDF
(84 KB)

FRBSF Economic Letter 2002-16 | May 24, 2002

The Standard & Poor’s (S&P) 500 stock index closed at an all-time high of 1527 on March 24, 2000. Since then, the index has declined by about 28% to 1097 as of May 14, 2002, roughly where it was four years ago.

The Standard & Poor’s (S&P) 500 stock index closed at an all-time high of 1527 on March 24, 2000. Since then, the index has declined by about 28% to 1097 as of May 14, 2002, roughly where it was four years ago. Falling stock prices have been accompanied by even larger percentage declines in corporate earnings. In 2001, the reported (GAAP-based) earnings of S&P 500 companies totaled $24.69 per share—the lowest earnings figure since 1993 and a whopping 50% drop from 2000 earnings of $50 per share. The collapse in corporate earnings caused the price-earnings (P/E) ratio of the S&P 500 index to increase sharply to a year-end 2001 value of 46. This figure exceeds the P/E ratio of 28 that prevailed at the market peak in March 2000 and is three times higher than the average P/E ratio of 15.2 going back to 1926.

This Economic Letter examines the long-run behavior of the P/E ratio and describes how it might be used to assess the fundamental value of the stock market.

Effect of the rising P/E ratio

According to Ibbotson Associates (2002), the average compound annual return on the S&P 500 (including dividends) was 10.7% from 1926 to 2001. The corresponding return on long-term U.S. government bonds (with a maturity near 20 years) was 5.3%. Hence stocks delivered an annual excess return over bonds of 5.4% during this period.

Ibbotson and Chen (2002) show that the increase in the P/E ratio since 1926 accounts for about one-fourth of the historical excess return on stocks over bonds. This result takes on greater significance when we recognize that the bulk of the net increase in the P/E ratio occurred during the last two decades. Since 1982, there has been a sixfold expansion (from 7.5 to 46) in the “multiple” that investors assign to each dollar of reported earnings. This expansion helped to produce an extraordinary compound annual return on stocks of 15.2% over the period. Given this record, future movements in the P/E ratio (or lack thereof) will likely play an important role in determining how well stocks perform in the coming years.

What accounts for the rising P/E ratio?

Why would investors be willing to pay more for each dollar of corporate earnings than they have in the past? There are several candidate explanations. These include: (1) higher expected future earnings growth, (2) lower perceptions of the risks of holding stocks, and (3) irrational exuberance. Over long periods, corporate earnings growth has tracked the economy’s trend growth rate of productivity. Starting around 1995, the U.S. economy saw a pickup in measured productivity growth that is thought by some to represent a permanent structural change. Improved growth prospects associated with the so-called “new economy” have been cited as justification for the unprecedented valuations assigned to stocks in recent years.

Diminished risk perceptions can also justify higher valuations. All else equal, investors would be willing to pay more for a claim on future earnings if they thought that the risk of suffering a bad outcome was smaller than in the past. Institutional and regulatory developments during the past century and an improved understanding of the economy on the part of policymakers have been cited as factors contributing to a safer environment for stocks. Campbell and Shiller (2001) mention (but do not necessarily endorse) the idea that baby boomers may be more risk-tolerant than earlier generations because memories of the depressed economic conditions of the 1930s have faded. Moreover, baby boomers may view stocks more favorably than bonds because they recall the poor performance of bonds during the high-inflation decade of the 1970s.

The third possible explanation for the rising P/E ratio, advocated by Shiller (2000), is that investors have irrationally bid up stock prices to levels that bear no relationship to the intrinsic values of the underlying businesses (as measured by the expected discounted value of their future earnings streams). Shiller notes that, throughout history, occurrences of major speculative bubbles have generally coincided with the emergence of some superficially plausible “new era” theory. Even with a pickup in trend productivity growth, investors may have overreacted by heedlessly extrapolating the temporary surge in earnings growth of the late 1990s far into the future. Some recent studies provide support for this idea. Chan, et al. (2001) show that equity analysts’ forecasts of long-term earnings growth rates have been consistently too optimistic and have exhibited low predictive power for the actual earnings growth rates subsequently achieved. Sharpe (2002) shows that the dramatic increase in equity analysts’ long-term growth forecasts in the latter half of the 1990s may explain as much as one-half of the rise in the P/E ratio during those years.

A simple valuation model: stock as a “disguised bond”

To gauge the relative merits of fundamental versus bubble explanations for the rise in the P/E ratio, we must apply a valuation model to the aggregate stock market. One simple valuation model compares the earnings yield on stocks—defined as the inverse of the P/E ratio—to the yield on a long-term bond. The logic behind this comparison is nicely summarized by the following quote from famed investor Warren Buffett who describes a stock as a type of “disguised bond” (Loomis, 2001):

A stock…is a financial instrument that has claim on future distributions made by a given business, whether they are paid out as dividends or to repurchase stock or to settle up after sale or liquidation. These payments are in effect “coupons.” The set of owners getting them will change as shareholders come and go. But the financial outcome for the business’ owners as a whole will be determined by the size and timing of these coupons. Estimating those particulars is what investment analysis is all about.

Since the “coupon” payments from stocks are typically viewed as more risky than those associated with bonds, one might expect the earnings yield on stocks to exceed the yield on, say, a long-term government bond which is considered safe from default. On the other hand, the coupon payments from stocks will tend to grow over time with the earnings of the underlying businesses, whereas the coupon payments from bonds are fixed. If the expected earnings growth from stocks exactly compensated shareholders for the extra risk, then a direct comparison between the earnings yield on stocks and the yield on a long-term government bond could be justified.

One drawback of the simple valuation model described above is that it does not allow for changes in investors’ perceptions of the risks of holding stocks versus bonds. Asness (2000) develops a valuation model that addresses this issue. In one version, the earnings yield on the S&P 500 is regressed on a constant term and the following three explanatory variables: (1) the yield on a long-term government bond, (2) the volatility of monthly stock returns over the preceding 20 years, and (3) the volatility of monthly bond returns over the preceding 20 years. The long-term bond yield captures expectations of future economic growth as well as expectations of future inflation. The other two variables capture the slowly changing risk perceptions of successive generations of investors, where risk perceptions are based on each generation’s volatility experience. According to these volatility measures, stocks have become less risky over time while bonds have become more risky (Figure 1). Asness shows that the inclusion of these volatility measures significantly improves the model’s ability to explain movements in the earnings yield and, by extension, the P/E ratio.

Figure 2 plots a variant of Asness’s model where the P/E ratio itself (rather than the earnings yield) is regressed on a constant term and the logarithms of the same three explanatory variables. For comparison, the figure also plots the inverse yield on a long-term government bond. The fitted P/E ratio from the model captures 70% of the variance in the observed P/E ratio over the sample period 1946 to 2001 (monthly data from 1926-1945 are used to compute the initial volatility measures). In contrast, the inverse bond yield alone does a poor job of capturing movements in the observed P/E ratio. These results confirm Asness’s finding of a strong empirical link between valuation ratios and the return volatilities experienced by investors.

In Figure 2, the observed P/E ratio lies above the fitted P/E ratio from November 1998 until the end of the data sample in December 2001. One interpretation of this result is that the stock market has been overvalued for the past several years, i.e., the observed value has consistently exceeded the “fundamental value” implied by the long-standing relationship between the P/E ratio, the bond yield, and the volatility measures. Alternatively, one could argue that the market is properly priced but the valuation model is missing some crucial elements.

Predicting the future

Making predictions about the stock market can be a humbling experience. Still, it may be worthwhile to consider the model’s predictions for the year-end 2002 level of the S&P 500 index. Given a current 20-year government bond yield of about 5.5% and employing the end-of-sample volatility measures for stocks and bonds, the model predicts a P/E ratio of 24.1. Applying this multiple to the S&P’s estimate of $36.34 for reported earnings in 2002 yields a predicted value of 876 for the index—about 20% below the current level. Different predictions would be obtained if any of the model inputs (for example, the bond yield or the earnings forecast) were to change significantly over the coming year. Also note that the market has deviated from the model’s predictions for sustained periods in the past.


Over the long history of the stock market, high P/E ratios have been transitory phenomena. Campbell and Shiller (2001) show that, sooner or later, the P/E ratio has tended to adjust back towards its long-run average. These adjustments have taken place mainly through changes in stock prices (P) rather than through changes in earnings (E). While Campbell and Shiller do not expect a complete return of the P/E ratio to its long-run average, they predict poor returns from stocks in the coming years. The valuation model described here says something similar: we would not expect the P/E ratio to return to its long-run average because the bond yield and the volatility measures are now different from the past. Nevertheless, given the current earnings forecast, the model predicts a downward adjustment in stock prices.

Finally, investors should recognize that the extraordinary returns on stocks recorded over the last 20 years have been driven in large measure by a rising P/E ratio. A believer in efficient markets would not expect the P/E ratio to continue its upward trend because the current market price supposedly already reflects investor risk perceptions and expectations about the future trajectory of earnings. Absent further changes in the P/E ratio, stock prices can rise only as fast as earnings. Since 1926, earnings have grown by an average compound rate of 5.8%. If we add to this figure the current dividend yield on stocks of about 1.2%, we obtain a forecasted total return on stocks of 7% per year—only about one-half the average compound return since 1982.

Kevin J. Lansing
Senior Economist

[URLs accessed in May 2002.]

Asness, Clifford S. 2000. “Stocks versus Bonds: Explaining the Equity Risk Premium.” Financial Analysts Journal 56(2), pp. 96-113.

Campbell, John, and Robert J. Shiller. 2001. Campbell, John, and Robert J. Shiller. 2001. “Valuation Ratios and the Long-Run Stock Market Outlook: An Update.” NBER Working Paper 8221.

Chan, Louis K.C., Jason Karceski, and Joseph Lakonishok. 2001. “The Level and Persistence of Growth Rates.” NBER Working Paper 8282.

Ibbotson, Roger G., and Pend Chen. 2002. “Stock Market Returns in the Long Run: Participating in the Real Economy.” Yale University, International Center for Finance, Working Paper No. 00-44.

Loomis, Carol. 2001. “Warren Buffett on the Stock Market.” Fortune Magazine (December 10).

Shiller, Robert J. 2000. Irrational Exuberance. Princeton: Princeton University Press.

Stocks, Bonds, Bills, and Inflation, 2002 Yearbook. Chicago: Ibbotson Associates.

Sharpe, Steven A. 2002. “How Does the Market Interpret Analysts’ Long-Term Growth Forecasts?” Federal Reserve Board, Finance and Economics Discussion Series Paper 2002-7.

Opinions expressed in FRBSF Economic Letter do not necessarily reflect the views of the management of the Federal Reserve Bank of San Francisco or of the Board of Governors of the Federal Reserve System. This publication is edited by Anita Todd and Karen Barnes. Permission to reprint portions of articles or whole articles must be obtained in writing. Please send editorial comments and requests for reprint permission to