Bank supervisors and stock market investors engage in extensive monitoring of bank holding companies (BHCs), but for different reasons. While investors are looking to ensure that BHC managers maximize shareholder value, bank supervisors monitor BHCs to enforce regulations, gauge their safety and soundness, and guard against broader systemic risk.
- Supervisory surveillance and equity markets
- Can the equity market anticipate BOPEC changes?
- Contributions to an off-site marketing model
- Conclusion
- References
Bank supervisors and stock market investors engage in extensive monitoring of bank holding companies (BHCs), but for different reasons. While investors are looking to ensure that BHC managers maximize shareholder value, bank supervisors monitor BHCs to enforce regulations, gauge their safety and soundness, and guard against broader systemic risk. Despite these differences in motivation, changes in stock prices could be relevant for supervisory concerns.
In this Economic Letter, we present empirical evidence on the potential usefulness of equity market data in the supervisory surveillance of BHCs. We find that changes in stock prices tend to precede changes in supervisory BHC ratings by at least nine months. We also assess the contribution of equity market information in the context of an off-site monitoring model for BHCs. Our results indicate that equity market information can be marginally useful to supervisors, especially since the cost of acquiring and manipulating the data is quite low.
Supervisory surveillance and equity markets
The Federal Reserve is the supervisor of BHCs in the United States. (Note that the BHC is typically the stock-issuing entity within a banking organization.) Full-scope, on-site inspections of BHCs are a key element of the supervisory process. These inspections are generally conducted once a year. At the conclusion of an inspection, the supervisors assign a composite BOPEC rating, which summarizes their opinion of the BHC’s overall health and financial condition. The BOPEC acronym stands for the five key areas of supervisory concern: the condition of the BHC’s Bank subsidiaries, Other nonbank subsidiaries, Parent company, Earnings, and Capital adequacy. BOPEC ratings are assigned according to an absolute scale from the highest rating of one (indicating strong performance) to the lowest rating of five (indicating very poor performance). Note that BOPEC ratings are highly confidential and are not made public.
Between on-site inspections, when private supervisory information cannot be gathered as readily, supervisors monitor BHCs using a well-specified off-site monitoring system; see Supervisory Letters 95-43 and 02-01 issued by the Federal Reserve Board of Governors. Three primary sources of information are used in the surveillance process. One source, known as the BHC Performance Report, is a detailed summary of the quarterly Y-9C regulatory reporting forms filed by BHCs. From this report, certain variables are selected as key performance criteria, and if a BHC fails to meet these criteria in a given quarter, it is noted as an exception that requires further monitoring.
A second source of information for off-site BHC monitoring is the supervisory CAMELS ratings assigned to banks within a bank holding company. As with BOPEC ratings, CAMELS ratings are assigned after bank examinations and are confidential. Since the condition of a BHC is closely related to the condition of its subsidiary banks, the off-site BHC surveillance process includes monitoring recently assigned CAMELS ratings.
A third information source is BHC financial market information, when available. Supervisors monitor BHC stock prices (and other financial market variables). If a BHC exhibits irregular stock price movements, it can be noted as an exception that requires further monitoring during the regular surveillance process.
Using equity market data in the BHC supervisory surveillance process is in keeping with broader efforts to promote market discipline in banking; see Kwan (2002) for a summary. A potential obstacle to using equity market data is the opaqueness of BHC assets; that is, loans, credit lines, and other BHC financial assets may be especially hard for investors to value. If so, signals from BHC stock prices may not be reliable enough for supervisory purposes. Fortunately, most of the recent academic research provides some reassurance concerning the reliability of BHC equity market information. These findings suggest that BHCs are not harder for equity investors to value than nonfinancial firms.
Can the equity market anticipate BOPEC changes?
If equity market assessments are to be useful to BHC supervisors, they must, at a minimum, agree with supervisory assessments a reasonably large fraction of the time. The equity market assessment that we use is based on BHC stock returns leading up to a BOPEC assignment. Large stock returns (either positive or negative) could give supervisors an early warning of changes in the economic environment that are relevant to a BHC’s condition. We examined this possibility by conducting an event study of BHC stock returns leading up to BOPEC assignments. We constructed a model that decomposes BHC stock returns into a systematic component based on general market conditions and an idiosyncratic component that captures individual BHC factors. We examined whether the realized cumulative idiosyncratic returns up to twelve months before the BOPEC assignment behaved abnormally. This approach allows us to examine whether the BHC’s idiosyncratic returns are consistent with the BHC’s assigned BOPEC rating.
In our event studies (see Krainer and Lopez, 2001), we found that the equity market sends a clear signal well in advance of an approaching ratings change. For upgrades, the returns are positive and statistically significant as early as twelve months before the inspection. For downgrades, the returns are negative and statistically significant starting at about nine months before the change. For no change in BOPEC rating, the returns are insignificantly different from zero, implying that the market is not signaling a change. These results suggest that equity market assessments of BHC conditions, as reflected in idiosyncratic stock returns, are consistent with future supervisory assessments.
Contributions to an off-site monitoring model
The second step in gauging the usefulness of BHC equity market information is to assess its contribution when used in conjunction with standard supervisory variables. For this exercise, we turn to the proposed BOPEC off-site monitoring (BOM) model discussed in Krainer and Lopez (2002). The benchmark, or core, BOM model examines the relationship between assigned BOPEC ratings and selected supervisory variables. We extended the model by incorporating BHC equity market variables, such as the systematic and idiosyncratic stock returns discussed above. When the model is estimated over our full sample of BOPEC ratings assigned from 1990 to 1999, the equity market variables are statistically significant and contribute to the model’s empirical fit of the data.
To be useful for supervisory purposes, this extended BOM model also must be able to forecast BOPEC ratings accurately out-of-sample. In order to mimic actual practices, we reestimated the BOM model with and without equity market variables every quarter based on a rolling four-quarter sample of data. The estimated models then were used to generate one-quarter-ahead BOPEC forecasts. The model signals a change in supervisory rating if the BOPEC forecast is more than three-quarters of a rating grade different from its corresponding lagged BOPEC rating. (Note that the forecasts are continuous variables and need not take integer values like the BOPEC ratings themselves.) When compared to all the ratings in our sample at four quarters prior to assignment, the extended model’s forecasts correctly identify 70% of all the BOPEC assignments and about 18% of all BOPEC changes. These percentages increase to 76% and 36%, respectively, at one quarter prior to assignment.
Another dimension of accuracy for the model is the mix of correct and incorrect signals. Given that the model signals, say, a downgrade, what is the probability that the signal will be correct? This dimension of accuracy is measured by the ratio of correct signals of a given BOPEC assignment to the total number of signals of that type. For example, the accuracy of downgrade signals is the ratio of correctly signaled downgrades to the total number of signaled downgrades. The figure presents these percentages for the upgrade, no change, and downgrade signals. These percentages of correct signals are relatively high at four quarters prior to assignment and improve at one quarter prior.
” No change” signals are the most common and are correct about 70% of the time. Downgrade signals are correct about 45% of the time at four quarters prior, and that percentage improves to 66% at one quarter prior. Upgrade signals are correct about 60% of the time at four quarters prior and almost 80% of the time at one quarter prior. These results indicate that forecast signals from the extended BOM model are accurate a large percentage of the time, even up to a year prior to the BOPEC assignments, and could thus be useful for off-site monitoring purposes.
A critical question is whether the model that includes both supervisory and equity market variables provides useful information about BOPEC ratings beyond what is obtained by the model using only supervisory variables. A common way to make such an assessment is to compare statistically the accuracy of the two sets of forecasts, which in this case is the percentage of BOPEC ratings accurately forecasted. By this metric, we find little statistical difference between the accuracy of the forecasts based on supervisory variables alone and that of the model augmented with equity market variables.
This result, however, does not mean that the forecasted BOPEC ratings from the two models are the same. The forecasting literature has shown that combining forecasts from different models can improve certain aspects of forecast accuracy. That appears to be the case here since the two models signal BOPEC changes for different, although overlapping, sets of BHCs. Hence, another way to gauge the contribution of equity market information is to examine the additional forecast signals for public BHCs as generated by the extended model relative to the core model’s signals. At four quarters prior, the extended model signals 72 additional BOPEC changes, of which 27 (about 40%) were correct. The correct signals were almost evenly split between BOPEC upgrades and downgrades. For one quarter prior, 101 additional BOPEC changes were signaled, of which 44 were correct and again almost evenly split between upgrades and downgrades.
Seen in this light, the marginal benefit of adding these additional signals to the core model signals is notable. At four quarters prior, the additional 27 correct signals of BOPEC changes increase the total to 178, a 20% increase. At one quarter prior, the additional 44 correct signals increase the total of correct BOPEC change forecasts by 12% to 394. The benefits from having the additional correct signals provided by these forecasts could very well be worth the supervisory costs of dealing with the additional incorrect signals.
Changes in stock prices for BHCs appear to provide information on their financial condition that is relevant to supervisory concerns. When examined in isolation, we find that equity market variables lead BOPEC changes by at least nine months in advance. Equity market variables are statistically significant in our BOPEC off-site monitoring model estimated over our sample period. The model’s out-of-sample forecasts perform well when compared to actual BOPEC outcomes, but the forecasts were not statistically different from those of the model based solely on supervisory information.
However, the two models did not produce identical sets of forecasts. The marginal contribution of using equity market information is notable since combining the core and extended model’s forecasts increased the number of rating changes correctly signaled by about 20%. We conclude that using equity market variables in this way may have practical value for supervisors for two further reasons. Equity market data are available sooner than supervisory data from quarterly financial statements, which could assist a monitoring model in detecting sudden changes in BHC conditions sooner. Since the cost of incorporating equity market variables into a model, such as the BOM model, is low, even small net improvements in forecast accuracy could be of value.
Jose A. Lopez
Economist
John Krainer
Economist
References
Krainer, J., and J.A. Lopez. 2001. “Incorporating Equity Market Information into Supervisory Monitoring Models.” FRBSF Working Paper 01-14.
Krainer, J., and J.A. Lopez. 2002. “Off-Site Monitoring of Bank Holding Companies.” FRBSF Economic Letter 2002-15 (May 17).
Kwan, S.H. 2002. “The Promise and Limits of Market Discipline in Banking.” FRBSF Economic Letter 2002-36 (December 13).
Opinions expressed in FRBSF Economic Letter do not necessarily reflect the views of the management of the Federal Reserve Bank of San Francisco or of the Board of Governors of the Federal Reserve System. This publication is edited by Anita Todd and Karen Barnes. Permission to reprint portions of articles or whole articles must be obtained in writing. Please send editorial comments and requests for reprint permission to research.library@sf.frb.org