Irreverent Economics

my piece on voxeu.org today

Bank capital has emerged as a key element in the post-crisis financial regulatory reforms. Basel III is now likely to include a 7% equity-to-risk-weighted-assets capital requirement.

7% was a compromise. Some countries wanting more capital now intend to implement stricter standards unilaterally. This is making some of the others unhappy, and a bitter debate has erupted within the EU on whether individual EU member countries should be allowed to require more capital than the Basel III, and hence EU, minimums.

Capital and bank size in the EU

Measured by total assets of domestic banks, the three largest banking nations in the EU in June 2010 were Germany, the UK and France, in that order. The banking statistics come from the ECB. If we divide by GDP, the UK is largest amongst those three at 464%, followed by Germany at 337% and France at 336%. Judging by these numbers, the UK does not seem to be the outsized banking nation it often is made out to be. By including foreign banks, the numbers for France and Germany increase slightly, but the UK goes to 658%.

The ECB provides several different ways to look at bank capital. By taking the measure most relevant to the financial markets during the crisis, tangible equity/tangible total assets, the UK has the best capitalised domestic banks at 4.2% amongst the three countries, followed by France at 3.8%, with Germany having the least capitalised banks at 3.1%.

Minimum capital and Basel III: Some countries view the 7% as insufficient

In the ongoing debate on Basel III, one of the most contentious issues has been the level of bank capital. The Basel III minimum equity capital levels have been set at 7% of risk-weighted assets. Some countries have indicated they want higher minimum capital levels. One might think that the countries making the biggest public noises about problems of excessive risk-taking and speculation would be exactly those demanding higher capital. After all, higher capital directly reduces leverage and risk taking, increasing safety.

Surprisingly, it is the opposite.

• The main champions for more capital are the US, UK and Switzerland,
• The opposition is led by Germany and France.

In the US, an additional 3% may be imposed (Wall Street Journal 2011). Within Europe, the UK has similarly signalled its willingness to do the same (BBC News 2011). That would need to be allowed by EU regulations. France and Germany would like the minimum 7% also to be the maximum, following the so-called “maximum harmonisation” principle. Their public reasoning seems to be based on the public’s belief that their banks weathered the crisis better than the Anglo-Saxon banking nations. However, there is a lingering suspicion that something less straightforward is behind their stance. Germany and France may be opposing higher capital requirements because of hidden vulnerabilities in their banks’ assets. This would both make their capital ratios worse than reported above and the banks more fragile. By contrast, countries wanting more capital already might have stronger banks, partly because they have been more forthcoming in forcing their banks to recognise dodgy assets.

The maximum harmonisation principle for capital is misguided

The maximum harmonisation principle for capital would be sensible if the EU were a single financial market, homogeneous in national attributes such as bankruptcy laws and having a single European supervisory agency. In such a world, variable capital standards undermine the principle of a single financial market.

This, however, is based on a utopian view of European financial markets. After all, the EU does not have a political union, enabling a single EU supervisory agency, nor common bankruptcy laws. Consequently, the composition of assets, and treatment of assets in bankruptcy will be different across borders.

Furthermore, with each state having independent budgets, government policies and development levels, the nature and importance of banking will vary significantly across member states.

Looking at total banking assets over GDP, the relatively smallest banking state is Romania at 64% and the largest is Luxembourg at 1,964%, followed by Ireland at 929% and Cyprus at 928%. For the largest banking states, the financial sector is a significant generator of systemic risk and contributor to the business cycle. Those forces are not as strong in countries with smaller banking sectors, especially where the banks are mostly foreign.

A country with a large banking system needs different approaches to supervision than a small banking state. It needs more protection from financial turmoil and hence it would be prudent for it to require higher capital levels.

Indeed, it is sensible to vary capital levels with the relative size of the banking sector. For these reasons, the maximum harmonisation principle for bank capital is not advisable.
Those wanting low capital may have weak banking systems

The same countries that led the opposition to higher capital standards in the Basel III negotiations now oppose variable capital requirements in the EU, Germany and France.
I suspect they fear that allowing countries to impose more stringent capital standards would expose the weaknesses of their own banking systems. If an important banking nation successfully implements relatively higher capital standards, it directly signals that country’s relative banking strength and a desire not to support the banks in a crisis.

Perhaps, the real reason for the French and German opposition to variable capital standards can both be found in weaknesses in those countries’ bank assets and their willingness to use taxpayers’ money to bail out the banks.

Conclusion

The financial crisis demonstrated the need for improving capital standards, with Basel III a step in the right direction. Countries with large financial systems need to have the freedom to impose stricter controls on risk taking than is the norm. For these reasons, an EU maximum harmonisation for bank capital would be the wrong step.

References

BBC News (2011). “EU block on making banks safer”, 16 June.
Wall Street Journal (2011). “Lenders Dig In on Rules”, 16 June.

From voxeu.org

second part from this post

The appropriate use of risk models: Part II

Risk models are used in four different (though overlapping) situations:

• Routine understanding of risk by banks and supervisors;
• Routine management and control of risk by a banks and trading desks;
• Analysis of systemic and regulatory risk;
• Management and control of systemic risk by supervisors.

We consider these in turn.

Routine understanding of risk by banks and supervisors

The easiest situation is where risk models are used to understand rather than constrain risk, and where risk assessments concern routine rather than extreme risks hence making large amounts of relevant data available.

Because portfolios are not constrained by the models, error maximisation is not an issue, and complex models with good expressive power and a good fit to historic data can be employed, while large sample sizes keep problems of data snooping and over-fitting to a moderate size.

These circumstances are ideal for off-the-shelf models and they should be expected to perform well in this role. Any failures should not be systemically important. Unfortunately such uses of models, although common in academic studies, are likely to be rare in practice.

Routine management of risk by financial institutions

When someone in industry models risk, it is generally to control risk. This introduces error maximisation. The more risk models are used to constrain risk taking, the more fragile error maximisation makes them. Consequently, one should be especially careful to avoid the problem of data snooping.

This suggests particular criteria for evaluating models used to control risk. They do not need fit historic data particularly well. Instead, they need to be robust against currently-unknown future mistakes. A good in-sample fit is not only irrelevant, but even dangerous because it gives more scope for error maximisation. This suggests that models used to constrain risk should be substantially simpler than models used to understand risk. We think this distinction is not widely understood.

Supervisors should be particularly cautious about demanding the use of similar risk models across multiple institutions because by doing so they increase the likelihood that all institutions suffer a failure of risk control at the same time, elevating an individual problem into a systemic one. This was noted by Danielsson and Shin (2003) in their discussion of endogenous risk.

Analysis of systemic risk

The area of systemic risk brings further challenges. Here the question of interest is not the risk of financial institutions failing, but rather the risk of cascading failures. Consequently, a reliable systemic risk model needs to capture the risk of each systemically important institution, as well as their interactions. This will be challenging since such models will reflect the financial system as it is, not as it would be if policymakers acted on the model. Since the models are endogenous to the system they model it is impossible to avoid a substantial subjective judgement on what influence they may have.

Systemic risk is concerned with events that happen during crisis conditions, looking far into the tails of distributions. This makes the paucity of relevant data a major concern, dictating the use of very simple models to keep the adverse effects of over-fitting to reasonable levels. In addition, any reliable systemic risk model needs to address the transition from non-crisis to crisis.

Incentives make the process difficult. During tranquil times, the risk of extreme events is a low priority. Extreme outcomes happen only rarely, perhaps once a decade, and bonus and employment cycles are much shorter than that. It is not in the interest of risk takers, speculating with other peoples’ money, to be overly concerned with extreme risk. Even if the financial institution or the supervisor does have such concerns, such risk taking is difficult to detect when concealed in a turbid alphabet soup of derivatives. In the final analysis, the likelihood of extreme events is often impossible to detect with any model.

Daily 95% or 99% risk levels, such as those in the Basel market risk accords, are of very little direct relevance for systemic risk, and further introduce the problem of error maximisation. We therefore disagree with the widespread assumption that successful models for routine risk can be expected to perform well in systemic risk forecasting. This applies to many systemic risk models currently being proposed by government institutions.

Ultimately, the difficulty of the systemic risk problem suggests that supervisors working on systemic risk should be wary of statistical models of extreme market outcomes. The models may provide a number labelled “systemic risk”, but this does not mean that the number has any meaning. Excessive belief in statistical models will lead supervisors to defend against obsolete threats and leave them blind to the new.

Here be dragons: The challenge of controlling extreme risk

Medieval mapmakers often noted the risk of an unknown kind by the notation “here be dragons”. Attempts at controlling extreme risk should come with a similar warning. Just like the sailors of yesteryear, financial institutions will go into unknown territories and, just like the map makers of that era, modern risk modellers have very little to say.

Financial institutions, willingly or not, will assume extreme risk. This cannot be prevented with any cost-effective methods. Nevertheless, the repeated occurrence of extreme events in financial markets implies that such risks need to be contained and managed. After all, following a crisis event there is generally strong political pressure on supervisors and financial institutions to prevent recurrence.

This however leaves open the question as to the extent to which it can be accomplished. We argue above that it is difficult to forecast extreme risk because of over-fitting. Basing risk constraints on the resulting estimates adds the problem of error maximisation, placing any supervisor seeking to constrain extreme risk taking in a very difficult situation.

Financial institutions and supervisors seeking to control extreme risk-taking should be careful in their use of statistical models, and certainly not use them simply as a substitute for trying to understand what trading strategies are being employed and how they might contribute to some future risk. These models must face all the challenges of those used to understand systemic risk, and in addition will face error maximisation.

Models are of course necessary but they should be extremely simple and founded on very basic measures of asset size and risk. They should have very few (or ideally no) parameters estimated from history, because each parameter increases the scope for data snooping and error maximisation. They must make a limited and wary (or no) allowance for hedging, because in a crisis hedging can be expected to fail.

Unfortunately, the current thrust of regulation seems to be to be in the opposite direction, mandating the use of powerful and expressive models that may provide a good fit to historic data, even deep in the tails, and then use those models as a rigorous constraint on risk taking across many portfolios. This is exactly the most fragile approach.

Some suggestions

As we started this article with a challenge from a risk manager, we want to use this analysis to make specific recommendations on the use of models.

Most importantly we want to underline the need for understanding. Risk estimates do not exist in a vacuum; they are made for some purpose and based on some model. Sensible estimates cannot be made unless both are understood. Ideally risk managers should create their own models as this is the best way to understand the model’s limitations. Outsourcing risk-model creation is outsourcing a key component of risk control and if it is done then substantial efforts must be put into understanding and monitoring the work that has been done.

Any risk forecast should come with robust analysis of forecast uncertainty. This might take the form of the fan charts used by the Bank of England for inflation forecasts. Forecast uncertainty should incorporate statistical uncertainty within the model (such as parameter standard errors), model risk (uncertainty created by using the wrong model) and well as an estimate of the bias introduced by data snooping. Existing statistical methodology allows for such calculations, it is just a matter of mandating their use. Any serious discussion of risk should incorporate explicit estimates of uncertainty.

Of course providing sensitivity analysis for risk forecasts does create problems, particularly making the interpretation of the results harder to communicate to senior management. At the end of the day practical decisions have to be made and few decision-makers are comfortable with the multi-dimensional integrations required when using full distributions rather than point forecasts. Nor can they be expected to enjoy the explicit recognition that every decision may be wrong.

Unfortunately this is the reality of decision making, and obscuring it may make life more comfortable but corrupts the decision-making process. At the very least, there should be lively discussion of uncertainties between risk professionals and any software providers involved, both within individual forms and also in supervisory agencies.

Extreme risk creates additional challenges due to the paucity of relevant data, the correspondingly highly-subjective element in the model, the length of time required to falsify an incorrect model and the seriousness when an extreme event occurs. There is a natural tendency in the industry to sweep problems of extreme risk under the carpet. After all, if the portfolio or the institution does not blow up on my watch, why should I care? Relying on statistical models to assess extreme and systemic risk is likely to provide the false belief that the problem is under control, such as persisted for a decade after Alan Greenspan’s “Irrational Exuberance” comment of 1996.

Macroprudential regulations should focus on prevention and resolution of systemic events. This is very different from trying to smooth out risk-taking on a day-to-day basis, and the latter may well be counterproductive if the result is to lower volatility at the cost of producing fatter tails. More attention should be paid to preventative measures that are model-free or depend only on the very simplest models, such as restrictions on loan-to-value ratios, minimum equity capital based on total (not risk-weighted) assets, such as the leverage ratio, and the Basel 3 liquidity constraints. The on-going work on living wills and resolution is very encouraging; even if much remains to be done. (See e.g. Bassani and Trapanese 2011).

Conclusion

Financial risk models, statistical and non-statistical, are essential for the functioning of financial markets and it would be impossible to manage risk without them. However, there is a tendency both by financial institutions and by supervisors to overstate the reliability of models and underplay their dangers, so it is important to identify their limitations.
We have identified several different reasons why risk models fail in practical use. This allows us to make constructive recommendations on where models can be used with confidence, where problems are likely, on what characteristics models should have and how they should be used, and on how the nature of the intended application will influence all these considerations.

References

Bassani, Giovanni and Maurizio Trapanese (2011), “Crisis Management and Resolution”, forthcoming in M Quagliariello and F Cannata (eds.), Banking Regulation, Riskbooks..

Danielsson, J and HS Shin (2003), Endogenous risk. In Modern Risk Management — A History. Risk Books.

Danielsson, Jon and Robert Macrae (2011), “The appropriate use of risk models: Part I”, VoxEU.org, 16 June.

from Voxeu.org

The appropriate use of risk models: Part I

Financial risk models have been widely criticised for both theoretical and practical failures, especially during the recent financial crisis. In spite of this, all proposals for reforming model use have been resisted. This is not surprising given how deeply ingrained models are in the practice of finance.

Such sentiments are eloquently stated in the conclusion of a comment on a recent Vox article;

“As a risk manager I fully recognise the shortcomings of any model based on or calibrated to the past. But I also need something practical, objective, and understandable to measure risk, set and enforce limits, and encourage discussions about positions when it matters. It is very easy to criticise from the sidelines – please offer an alternative the next time.”

Jan-Peter Onstwedder, Comment on Danielsson (2011)

Our aim here is to respond to the challenges such as those in Jan-Peter’s comment by making specific proposals for how risk models should be used in practice, and identifying how problems with models can be avoided. For a background on the theoretical aspects of risk models see Danielsson (2009, 2011). A practical analysis of models can be found in Macrae and Watkins (1998).

Nature of risk and of risk models

Financial risk is a forecast, not a measurement. Every risk forecast is an uncertain assessment of the underlying risk factors, often with wide confidence intervals, resulting from parameter uncertainty, model error and data snooping, and usually containing an uncomfortably large subjective element. Even nonparametric estimates will require choices such as estimation period.

Financial risk can only be understood in terms of a model. It may be a formal model, but whenever a user adopts some rule for controlling risk there must be a model implied by the rules adopted. For example, lending ratio restrictions imply a simple model that more bank loans lead to greater risk. A more complex model incorporating different levels of loan risk and operational risk is implicit in the Basel II risk weightings.

Despite model dependency and uncertainty, there is a tendency by end users to perceive numbers representing risk as coming from a scientific measurement – using a Riskometer in the language of Danielsson (2009)– rather than from an uncertain statistical procedure. Users need numbers they can use to convince their boss, client or regulator, so users of risk models prefer “objective” risk forecasts whereas forecasts accompanied by qualifications and uncertainties appears less objective.

We suspect this leads users to prefer commercial risk software that provides a single number, unencumbered by confidence intervals even though this makes it particularly hard for users to evaluate the reliability of off-the-shelf models. Where confidence intervals are estimated, their reliability is often suspicious. This is succinctly illustrated by David Viniar, Goldman’s chief financial officer, stating: “We were seeing things that were 25-standard deviation moves, several days in a row” (Financial Times 2007). This can only mean that Goldman grossly underestimated its standard deviations, making confidence intervals far too tight.

Why are uncertainties in risk forecasts so high?

There are several reasons why uncertainties in risk forecasting are higher than is usually assumed:

• The model estimation period is too short;
• There are structural breaks during the estimation period;
• Data snooping and model optimisation occur;
• Portfolios are optimised, maximising errors;
• It is often necessary to forecast extreme risks.

Since the first two issues are well known, we want to focus on the final three.

Data snooping and model optimisation

Every student of econometrics is taught the danger of data snooping. If we run a single regression, we get correct confidence intervals for parameter estimates and forecasts, subject to certain basic assumptions. If, however, we arrive at the same model as a result of optimising a number of explanatory variables and model specifications these assumptions are violated and the confidence intervals will be underestimated. The more complex the model and the smaller the dataset, the larger the underestimation becomes.

The misleading inference that data snooping can cause is demonstrated by Sullivan et al. (1999), who show that apparently statistically significant technical trading rules are not significant if confidence intervals are calculated correctly, taking into account the search for the best model.

Similar effects are at work in risk forecasting. Risk models are routinely validated by back-testing, that is, by examining how well a model forecasts market outcomes that have already happened. If the model performs badly it will be changed, and the end result is certain to perform well in-sample, over the back-testing period.

Such common approaches to risk modelling tell us more about the level of model optimisation than about how the model will perform out-of-sample in the future. Most risk models in practice appear to us to overemphasise their ability to fit past events, rather than out-of-sample risk forecasting. Risk models must be parsimonious, and tested over a variety of market turmoil if they are to minimise the problem of data snooping and model optimisation. The model best at forecasting is unlikely to be the best at capturing historic events with great accuracy.

This imposes a fundamental limit on what risk systems can achieve, especially in a crisis, because parsimonious models cannot deliver great precision but non-parsimonious models are likely to fail out-of-sample.

Portfolio optimisation and error maximisation

A related problem arises from the use of risk models in portfolio optimisation and risk control. Where risk models are a direct input into trading decisions, providing hard constraints on risky positions, the underlying trading process and portfolios will in all likelihood adapt to and exploit model weaknesses.

This problem arises since traders optimise portfolios towards low reported risk (or equivalently low capital usage) and high returns, causing trading decisions to become biased towards assets with under-forecast risk. In other words, the trader maximises exposure to the part of the asset universe with biased risk forecasts, maximising the impact that this error has on the portfolio. Such error maximisation can affect individual trading positions, institutions, and even the financial system as a whole, as illustrated by the recent crisis.

Prior to the crisis, many structured credit products, such as certain CDO tranches, had undeserved AAA credit ratings. As many investors correctly perceived the risk of such AAA tranches as higher than the risk of corporate AAA bonds, their yields were typically somewhat higher than corporate AAA yields. This in turn made such tranches attractive to less sophisticated investors who evaluated risk solely on the basis of credit ratings.

It is not the size of the pricing bias nor the magnitude of the event that is the main culprit here; the CDO market is a relatively small part of total financial assets. The problem is that the presence of tightly-binding constraints based on inaccurate models of risk (and the consequent error maximisation) motivated certain financial institutions to acquire large exposures to these assets. This led to concentrated losses with damaging systemic consequences.

Error maximisation, as an active risk management leads to reduced volatility and fatter tails. The risk in common events is better managed, at the expense of bigger and more frequent extreme events. The more rigorous risk models are used to constrain positions, the more errors will be maximised and the more dramatic will be the consequences when the errors are eventually revealed.

All risk models contain errors and are thus vulnerable to error maximisation. The more widely a model is used and the more tightly a constraint binds, the worse the error maximisation becomes. This argues for heterogeneity in risk models. In the worst case, where a single model or approach is given regulatory force and applied as a hard constraint to many portfolios, a small problem in micro-prudential regulations may be elevated to a systemic level.

Risk managers are well aware of the potential for error maximisation. However, we suspect this is not well understood by senior management nor properly considered by designers of financial regulations.

This imposes a second fundamental limit to what a risk system can be expected to achieve, because risk systems used to constrain portfolios will have been compromised by the implicit optimisation of portfolios to contain assets for which risk systems underestimate risk. Risk systems that have been used to constrain positions will always prove unreliable in a crisis.

Extreme risk forecasts

Perhaps the greatest need for models is in the forecasting of extreme risk or tail risk, especially during periods of financial crisis and extreme market turmoil. This however, is the area where risk models are least reliable because the effective sample size of comparable events is very small. At worst there might be one observation or even zero when we wish to consider events not yet seen.

Over the past half century we have observed fewer than 10 episodes of extreme international market turmoil. Each of these events is essentially unique, and apparently driven by different underlying causes. Trying to get an overall idea of the statistical process of data during those episodes with fewer than 10 episodes of turmoil, all with different underlying causes is difficult to the point of impossible. While it might be possible to construct a model fitting 9 crisis events in a row, there is no guarantee that it will perform well during the 10th.

Nor does it seem likely that we can get much information about price dynamics during turmoil by using the non-crisis data that makes up the bulk of available information since there is ample evidence that market dynamics are very different in times of crisis. Market lore suggests that in a crisis traders rely more on simple rules of thumb (such as “all stocks have a beta of one”, or even “cash is king”) than in more nuanced normal times. This is supported by academic studies, such as Ang et al. (2002), showing that that correlations go to one during crises (manifestation of nonlinear dependence), because of incentives to trade out of risky assets into safe assets when risk constraints bind, causing a feedback between ever higher risk and sharper constraints (see Danielsson et al. 2010).

This is the third fundamental limit to what risk system can be expected to achieve. Regardless of how much data we have, there is never enough to reliably estimate the tails. This is why models for extreme risk can be expected to fail during market turmoil or crises.

Tomorrow we consider how the intrinsic shortcomings of risk models matter for their four main uses. We also make some suggestions on how the financial industry and supervisors should use models in practice

References

Danielsson, Jon (2009), “The myth of the Riskometer”, VoxEU.org, 5 January.

Danielsson, Jon (2011), “Risk and crises”, VoxEU.org, 18 February

Danielsson, Jon, Hyun Song Shin, and Jean-Pierre Zigrand (2010), “Risk Appetite and Endogenous Risk”, Financial Markets Group Working Papers.

Macrae, Robert and Chris Watkins (1998), “A Disaster Waiting to Happen”.

Sullivan, Ryan, Alan Timmermann and Hal White (1999), “Data–Snooping, Technical Trading Rule Performance, and the Bootstrap”, Journal of Finance.

Ang, A and JS Chen (2002), “Asymmetric correlations of equity portfolios”, Journal of Financial Economics, 63(3):443-494.