What does best practice scenario testing look like?

As insurance companies start to implement ICAAP ahead of the 1 January 2013 deadline next year, they are thinking through what best practice looks like; not necessarily to implement it now, but to try to move in  the right direction.

One aspect of the new standards which is new from a regulatory perspective is the requirement to have at a minimum (section 13(d) of the new LPS 110 and 11 (d) of GPS 110):

stress testing and scenario analysis relating to potential risk exposures and available capital resources;

So what is best practice in this area? It very much depends on the institution, its complexity, and the sophistication of its risk management framework. But there are a few aspects that are common.

First, a few definitions. People talk somewhat interchangeably about stress testing, scenario analysis and sensitivities. I’m defining them as follows:

• Scenario analysis involves a story. It involves a company defining a story, for example, what would happen if the stagnation in Japan in the late 80s early 90s came to Australia. That story involves a whole lot of changes to many economic variables – interest rates drop, equity returns reduce, and disability claims might go up. New sales of insurance business might fall. All of those aspects need to be applied to the whole of the insurance company’s business, to illustrate the full effect. The story might take months or weeks to unfold.
• Stress testing (also sometimes called sensitivity testing) involves individual parameters – looking at what would happen to the business if a particular variable change – for example, what happens if interest rates drop by 1%,  or sometimes a range of individual parameters – such as claims rates increasing by 10% with an interest rate drop.
• Reverse stress testing (also called stress to fail) involves working out how the business would fail, for example, just how bad would a pandemic have to be before a life insurer was unable to pay all of its claims?

What is the point of scenario analysis or stress testing?

There are a few benefits from doing this work:

• It gives you a much greater understanding of the  risks associated with the business
• Communication of risks to senior management and Board
• The first step for planning actions in response to the scenarios and stresses
• Reverse stress testing in particular can unearth combinations of risks that will severely damage the business

So if you are planning a program of work to improve your scenario and stress testing capabilities, what should you be aiming for?

There are a few dimensions to think through:

• How many scenarios? What about sensitivity tests?
• How frequently should you do them?
• How quickly should you be able to calculate a scenario?
• Who is the audience?
• How complete is your coverage of your organisation?
• How fully have you covered all the interactions between risks and parts of your organisation?

In my view, best practice in a large, complex company looks a bit like this:

The Board gets to see two or three complex scenarios every year. They have been built up with some good conversation at Board and senior management level, and some wide thinking about all the different things that could go wrong in the organisation. One of those scenarios is a reverse stress test – it will break the organisation. The Board will also see sensitivity tests to key variables regularly; and those  variables will change as changes are happening to the environment of the organisation. With those complex scenarios will come a good discussion from management about the actions that would be taken at key points in the scenario to less the impact on the organisation.

Senior management will have the ability to see far more, and quickly, so that there is a good model enabling them to ask a question about a particular sensitivity or scenario, and a good enough answer to inform decision-making can be available in real time. Senior management will also see sensitivities to the most important variables every time they see capital numbers, so that they will be able to understand where they are in the risk cycle.

The stress and scenario testing will form an integral part of the risk and strategic planning of the organisation. Business planning won’t be regarded as complete without rigorous views of the outcomes under different plausible and implausible scenarios.

None of this comes without a cost. There will be work involved in setting up the infrastructure to enable the best practice view. In some cases, for simpler companies, that infrastructure won’t be that much better than the simple intuitions of management. But it is worth thinking through that best practice view, to think about where the management team is likely to have blind spots. What are the variables that generally get assumed away? What will happen if a particular story from history, or another country happens right here, right now? What can we do about it? How can we sharpen our risk management framework to match the insights we have had from thinking about this?

As companies improve their understanding of the risks associated with their businesses, and the ability to create scenarios, their overall risk management will improve. It is worth building the ability to create these insights, to improve capital management and risk management for the business as a whole.

Updated to add a link: the BIS has published a study on macro stress testing (ie for financial systems as a whole) which is worth a read.