Improving Barclays Bank's management of its exposure to Counterparty Credit Risk
Submitting InstitutionLondon School of Economics & Political Science
Unit of AssessmentMathematical Sciences
Summary Impact TypeEconomic
Research Subject Area(s)
Mathematical Sciences: Statistics
Economics: Applied Economics, Econometrics
Summary of the impact
In response to the deficiencies in bank risk management revealed
following the 2008 financial crisis, one of the mandated requirements
under the Basel III regulatory framework is for banks to backtest the
internal models they use to price their assets and to calculate how much
capital they require should a counterparty default. Qiwei Yao worked with
the Quantitative Analyst — Exposure team at Barclays Bank, which is
responsible for constructing the Barclays Counterpart Credit Risk (CCR)
backtesting methodology. They made use of several statistical methods from
Yao's research to construct the newly developed backtesting methodology
which is now in operation at Barclays Bank. This puts the CCR assessment
and management at Barclays in line with the Basel III regulatory capital
Research Insights and Outputs: The Barclays CCR backtesting
methodology, upon which Qiwei Yao's research had an impact, integrates a
number of statistical methods, underpinned by four pieces of Yao's
research. Each of these four research contributions arose as part of his
long-term focus on statistical inference for time series.
In an effort to reveal dynamic structure beyond linear autocorrelation,
Yao has made substantial advances in developing methods for modelling and
forecasting the future conditional on current and past status, which
reveals various interesting features that are relevant to but absent from
conventional linear time series models. For example, prediction errors for
the future depend on current position and errors are nonlinearly amplified
over time. Yao's work in this area involves a number of co-authors. For
the method used for the CCR project, of particular relevance was his joint
work  published in 2001 with Zongwu Cai from the University of North
Carolina at Charlotte and Wenyang Zhang, then a postdoctoral research
officer at LSE and now Professor in the Department of Mathematics at the
University of York. In terms of technical tools, Yao and his co-authors
have developed several nonparametric and semiparametric methods, including
a method in  for estimating conditional forecasters in the form of
point forecasts, forecast sets and forecasting distributions. To assess
the accuracy of those forecasters, some resampling techniques are used. A
distinguishing feature is to make the resampling adaptive to the
dependence in the data. This is achieved by either reproducing the
dependence in the resampled data or resampling the pre-whitened data.
Yao's second underpinning research contribution in  in 2000 was joint
work with two LSE Visiting Professors, Jianqing Fan and Peter Hall, on a
project investigating the use of traditional Z-tests and t-tests and also
bootstrap calibration in the context of simultaneous hypothesis testing.
One of the interesting findings in  was that with bootstrap methods the
number of simultaneous tests can be substantially larger than the sample
size. In fact, log(v) can be as large as the square-root of the sample
size, where v denotes the number of simultaneous tests. This justifies the
use of bootstrap methods when traditional Z-tests and t-test are
In the third more recent contribution, Yao, jointly with Hongzhi An
(Chinese Academy of Sciences), Da Huang (Yao's PhD student at the time)
and Cun-Hui Zhang (Rutgers University), revisited the classic stepwise
selection methods for regression in the setting of "large p and small n",
see . They showed that model selection consistency can be achieved by
using stepwise selection coupled with appropriately modified information
criteria. They also demonstrated by simulation that their method provides
a much more robust performance than some popular procedures such as LASSO.
This method provides a stable initial screening among a large number of
trades for selecting a counterparty representative portfolio.
Finally the matching quantiles estimation method proposed in  directly
resulted from solving the counterparty representative portfolio selection
problem. A new measure and a new statistical test were also proposed in
 to measure the goodness of match. It was joint work with Nikolaos
Sgouropoulos at QA Exposure of Barclays Bank and Claudia Yastremiz of Bank
Key Researcher: Professor Yao has been at LSE since 2000.
References to the research
 Cai, Z., Yao, Q. and Zhang, W. (2001). Smoothing for discrete-valued
time series. Journal of the Royal Statistical Society, Series B
(Statistical Methodology), 63 (2), pp. 357-375.
 Fan, J., Hall, P. and Yao, Q. (2007). To how many simultaneous
hypothesis tests can normal student's t or bootstrap calibrations be
applied? Journal of the American Statistical Association, 102
(480), pp. 1282-1288. http://eprints.lse.ac.uk/5399/
Evidence of quality: (1) and (2) are in top, peer-reviewed journals.
Details of the impact
Basel III, developed by the Basel Committee on Banking Supervision and
agreed in September 2010, is a set of comprehensive reform measures that
puts in place a global regulatory standard on bank capital adequacy,
stress testing and market liquidity. One of its mandatory requirements is
for all banks to conduct counterparty credit risk (CCR) model backtesting.
However, it leaves each bank to define its own backtesting methodology.
CCR backtesting is intended to ensure that the models provide more timely
and accurate information on a bank's exposure to the risk caused by a
counterparty by comparing the risk measures implied by the bank's pricing
models with the realised exposure based on the traded prices. The main
output of the backtesting is in the form of a "traffic light" system
whereby: "green" signals that there is no evidence against the pricing
models; "amber" signals that observed risk exposure is higher than that
implied by the pricing models but still within an acceptable tolerance
level; "red" indicates that the price models underestimate the risk and
need to be re-calibrated and the capital requirement adjusted. Ultimately,
backtesting results should demonstrate to the regulators (i.e. the Bank of
England in UK) the soundness and conservativeness of the reported exposure
to risk. Backtesting should also be able to identify where pricing models
At the invitation of Barclays Bank PLC, Yao has been participating in the
CCR backtesting project undertaken by the Quantitative Analyst — Exposure
Group at Barclays since January 2012. Yao was invited because of his
relevant research expertise in handling dependence and nonstationarity in
data and his considerable experience in inference with conditional
NATURE AND EXTENT OF THE IMPACT
Based on his previous work in ,  and , and the newly developed
method in , Yao proposed several key statistical methods that were used
in the development of the backtesting methodology outlined in document
[A]; see also [B] and [C]. The methodology has been approved by Barclays
internal governance process and, from September 2013, it has been
implemented as a part of business operations at Barclays. The outputs of
the methodology are now being used by Barclays credit risk managers on a
daily basis to control model risks. The new methodology improves
substantially the CCR assessment and management at Barclays in the ways
described below, and puts the practice at Barclays in line with the Basel
III regulatory capital framework. The resulting improved information about
the bank's exposure to risk mitigates potential future losses and thus
also helps to stabilize the global financial market and protect economic
stability and individual welfare.
DETAILS OF HOW THE RESEARCH UNDERPINNED THE BACKTESTING METHODOLOGY AND
HOW RISK ASSESSMENT HAS BEEN IMPROVED
Yao contributed directly to Barclays backtesting methodology set out in
[A] and [B] (see also [C]). Indeed, Yao wrote the first version of [B].
The key ways in which his research underpinned the methodology and the
benefits arising from his research contributions can be summarised as
1. Yao's research fed into a conditional counter (a new metric for
backtesting) and a simulation-based testing method which result in more
reliable information on risk exposure.
A "binomial counter" had been used by Barclays for analysing
collateralized transactions for which the data from non-over-lapping time
intervals can be treated as independent. A stratified version was
introduced to deal with dependence in uncollateralized transactions.
However, this simple approach, although approved by the Financial Services
Authority, is inadequate where prices across different time horizons are
dependent on each other — which is commonplace. Indeed, there hardly
exists any effective metrics for backtesting with dependent data.
The conditional counter and simulation-based testing method, introduced
on the basis of Yao's research, fill this gap. The conditional counter
method specified in [A] and [B] applies a version of the nonparametric
estimation method proposed in . It effectively looks at the extreme
values of conditional distributions instead of those of unconditional
distributions. The simulation method is more generic. It can be used to
test not only the extreme quantiles but also other features such as the
whole distribution. It can also be easily extended to test the sensitivity
to risk factors. This is significant because an important new requirement
of Basel III is to test various features of the distribution.
A simulation-based testing procedure for calculating p-values using the
bootstrap multiple comparison method of  is generic. It can be used for
testing, for example, a pricing model in relation to an observed trade
path over different time horizons based on any appropriate test statistic.
It takes into account the non-stationarity and the dependencies among
prices at different time horizons, or different price paths, in an
automatic manner (see [A]). This enables Barclays to extract more reliable
information on risk exposure in their daily operation. This test procedure
can also be adapted to identify whether or not pricing models are
overly-conservative, providing a sound scientific basis for Barclays to
adjust is capital reserve.
The proposed simulation-based testing method based on bootstrap
calibration represents the first such generic method to incorporate
nonstationarity and dependence among different trades and/or different
time horizons in an almost automatic manner. This represents a big step
forward in the backtesting techniques used at Barclays.
2. Yao's research contributed some key steps to the methodology for
selecting representative portfolios , which result in the Basel III
standard being met more effectively.
Basel III allows banks to construct representative portfolios for each
counterparty consisting of, for example, a subset of the trades between
two banks. Banks are left to decide the number and trades to be included
in the portfolio, but they have to justify their choices to their
supervisors (the Bank of England in the UK). As the number of trades
between two major banks can easily be in the order of tens thousands or
more, a simple linear regression runs into the so-called "large p and
small n" problem, even after the initial screening and categorisation.
Furthermore, many trades are highly correlated in the sense that the
sparse representation is certainly not unique. Hence some popular
techniques such as LASSO or the Danzig algorithm are no longer applicable.
The procedure adopted by Barclays uses the method of , i.e. a stepwise
sweep coupled with the use of (modified) information criteria to form the
candidate set. To construct a representative portfolio from the trades in
the candidate set, the new Matching Quantiles Estimation (MQE) method
proposed in  is now employed. Barclays has also adopted a new measure
and a test proposed in  to check how well the distribution of a
selected portfolio matches the target distribution at all levels
simultaneously, as required by Basel III. This overcomes problems with
existing estimation methods, such as quantile regression, which can only
check the success of a match at one, or at most, a few fixed levels,
resulting in a representative portfolio that falls short of the Basel III
Construction of counterparty representative portfolios is a new mandated
requirement under Basel III. Some key steps in the way that Barclays has
formulated this construction in its new methodology are attributable to
Sources to corroborate the impact
[A] Barclays QA Exposure Analytics (2012). (This source is confidential.
Please see [C])
[B] Barclays QA Exposure Analytics (2012). (This source is confidential.
Please see [C])
[C] A letter from the Director of the Barclays QA Exposure Analytics on
Yao's contribution on the CCR backtesting. This source is confidential.
Yao received the initial invitation from Barclays to participate this
project for 3 months (January — March 2012). The invitation has been
subsequently extended to December 2013.