Case Study 3: Applications of Computational Optimization under Uncertainty in Decision Support (Computational Optimization)

Submitting Institution

Imperial College London

Unit of Assessment

Computer Science and Informatics

Summary Impact Type

Technological

Research Subject Area(s)

Mathematical Sciences: Applied Mathematics, Numerical and Computational Mathematics, Statistics


Download original

PDF

Summary of the impact

The Computational Optimization Group (COG) in the Department of Computing produced new models, algorithms, and approximations for supporting confident decision-making under uncertainty — when computational alternatives are scarce or unavailable. The impact of this research is exemplified by the following:

  1. Axioma (a firm that provides factor-based risk models and portfolio construction tools for equity investors) now offers insurance to equity portfolios with efficient calculation of coherent risk measures — allowing diverse assets in portfolios.
  2. Commerzbank now has improved risk management for proprietary indices used in funds and options, increasing their revenue of investment strategies.
  3. The consultancy Decision Tree uses scenario tree based valuation of swing options to create better decision-support software, which attracted new clients in the energy sector.
  4. The utility provider Trianel now saves over two million Euros annually by adopting tools that rely on our new optimization techniques.
  5. The energy trading company e&t bought software based on our research that optimizes coal procurement contracts for a 750MW coal-fired power plant.

Underpinning research

Decision support under uncertainty is usually based on deterministic computational models. In many important applications, these models are now becoming so complex that their solution can only be achieved with vast computing resources and speed-up techniques such as parallelization. This has led to more abstract models. The problem becomes worse since solutions based on deterministic models are not reliable predictions of the future and having such reliability is the essence of trustworthy decision-support systems. Therefore, these models need to be revised and resolved frequently, making the overall approach significantly less accurate and feasible. An alternative that we adopt is to devise methods that optimize against an entire set of uncertain values in order to accommodate even worst-case scenarios. The latter is especially relevant in view of recent worst-case realizations of uncertainties in finance and engineering.

The Computational Optimization Group (COG) at Imperial College London has carried out the underpinning research described in this case study. The group has been founded by Professor Berc Rustem in 1995 and was strengthened by the arrivals of Dr Daniel Kuhn in 2006 and Dr Panos Parpas in 2011. The underpinning research strategically addresses the problems of deterministic models by

  • using non-deterministic models to gain reliability of predictions under uncertainty,
  • using approximating models to get better scalability of model solving, and
  • using theoretical results to get confidence measures into the quality of used approximations in relation to the concrete application that they model.

The research breakthroughs achieved are in the areas of robust optimization, stochastic programming, and decision rules. Details of the research are described below.

Risk Management & Optimization in Finance [i, ii, iii]: COG pioneered the development of discrete minimax techniques for rival market forecasts in [6] to inject robustness into portfolio optimization. This includes novel robust portfolios for the cases when the uncertainty set is defined over a continuous domain [1-2]. COG have created efficient approaches for portfolios that are robust to uncertainties in asset returns with performance guarantees, and where the portfolio may contain European-style options. This was achieved with the use of second order cone programming and duality [2]. The approximations developed in [1] enabled the efficient solutions of models and provide efficient and robust decision support with confidence. This research does not only enable the modelling uncertainty of physical events, it also enables the modelling of uncertainty in the underlying probability distributions. This research therefore also renders models for distributionally robust decisions [1].

Risk Management & Optimization in the Energy Sector [iv, v]: COG's energy research has focused on the development of computationally tractable models for multi-stage problems. This has been done through the development of time aggregation methods for continuous time problems in multi-stage stochastic programming [4, 5], and through the use of decision rules and duality theory to validate approximated models and their applicability for swing options in energy trading and production/procurement planning [3]. In order to use multi-stage stochastic programming in applications, it is necessary to discretize continuous stochastic programs so that they become more scalable. Although such approximations may improve performance, they do not give estimates of their quality. COG's research in [5] developed an approach in which an original multi-stage stochastic programming problem was abstracted into two discrete, state-aggregated stochastic programs that provide, respectively, a lower and an upper bound on the optimal value of the original problem. Here, scenario-free stochastic programming uses decision rules in order to characterize the optimal decision. Then duality theory is used to measure the "duality gap" that results from the use of approximations. That measure is an accepted indication of the quality of the approximation. These research outcomes therefore enable trading off the precision of the abstractions with the precision of the computed approximations of optimal values. This also facilitates the assessment of the quality of these approximations in relation to the precise model. The approximations developed in [5] can be computed numerically, since they contain only finitely many constraints and finitely many decisions.

The underpinning research above is grouped into two parts, reflecting impact in two different sectors. However, the underlying techniques are not limited to applicability in those sectors. The grant [vi] below, for example, had led to impact in the Defence sector and enabled some of the underpinning research that is now having impact in software in the defence sector. The nature of that software prevents us from documenting this impact.

References to the research

Publications that directly describe the underpinning research

* References that best indicate quality of underpinning research.

[1] *S. Zymler, D. Kuhn, and B. Rustem. Distributionally Robust Joint Chance Constraints with Second-Order Moment Information. Math. Programming 137(1-2): 167-198, 2013.
http://dx.doi.org/10.1007/s10107-011-0494-7

 
 
 
 

[2] S. Zymler, B. Rustem, and D. Kuhn. Robust Portfolio Optimization with Derivative Insurance Guarantees. European J. of Operational Research 210(2): 410-424, 2011.
http://dx.doi.org/10.1016/j.ejor.2010.09.027

 
 
 
 

[3] G. Haarbrücker and D. Kuhn. Valuation of Electricity Swing Options by Multistage Stochastic Programming. Automatica 45(4): 889-899, 2009.
http://dx.doi.org/10.1016/j.automatica.2008.11.022

 
 
 
 

[4] *D. Kuhn. An Information-Based Approximation Scheme for Stochastic Optimization Problems in Continuous Time. Math. of Operations Research 34(2): 428-444, 2009.
http://dx.doi.org/10.1287/moor.1080.0369

 
 
 
 

[5] D. Kuhn. Aggregation and Discretization in Multistage Stochastic Programming. Mathematical Programming A 113(1), 61-94 (2008). http://dx.doi.org/10.1007/s10107-006-0048-6

 
 
 
 

[6] *B. Rustem and M. Howe. Algorithms for Worst-case Design and Applications to Risk Management, Princeton University Press, Princeton NJ (2002). ISBN: 9780691091549

 
 
 

Grants that directly funded the underpinning research

[i] Uncertainty and Risk Optimisation Algorithms for Food Processing. EPSRC EP/C513584/1, B. Rustem (PI), £239,507, March 2005 — Feb 2008.

[ii] Systems Engineering — Worse-case Analysis and Parametric/Stochastic Programming. EPSRC GR/T02560/01, B. Rustem (PI), £418,439, October 2004 — April 2008.

[iii] Risk Management of Queuing Systems. B. Rustem (CI), EPSRC GR/S27849/01, £237,227 Sept 2003 — January 2007.

[iv] Robust Optimization of Nonlinear Processes under Uncertainty. EPSRC EP/I014640/1, B. Rustem (CI), D. Kuhn (CI), £767,492, April 2011 — March 2015.

[v] Scenario-Free Stochastic Programming. EPSRC EP/H020454/1, D. Kuhn (PI), £99,963, June 2010 — May 2011.

[vi] Software Programming for Decision Making Under Uncertainty. DTC BAE Systems (Operations) Ltd, B. Rustem (PI), £1,207,150, August 2005 — August 2011.

Details of the impact

The research outcomes of our underpinning research give those who need decision support under uncertainty a new tool-box with enhanced capabilities of solving more complex problems and of judging reliably whether these solutions are trustworthy. The transfer of this knowledge and technology is recent and on-going, but it has already had some measurable impact, which is detailed below for each of the five aforementioned impacts:

1. Axioma: Robust decision models require the specification of the set in which the realized value of uncertainty is expected to reside. Our research in [1, 2] now gives similar robustness guarantees when the uncertainty is realized outside of that specified uncertainty set. Since these outcomes were put into the public domain for free usage, Axioma was able to immediately implement and market software based on the model developed in [1]. Axioma "creates flexible tools to help portfolio managers quickly and accurately implement their strategies"[A] Its flagship product is the Axioma Portfolio Optimizer TM. According to the June 2011 Axioma Advisor, [B], the extension of a risk measure to portfolios containing options produced in [2] is now implemented and used on that flagship product. Axioma is a software company whose clients are financial institutions that manage trillions of dollars using these tools.

2. Commerzbank: COG's research results on minimax and optimization [2] are currently used in financial products by research collaborators from Commerzbank's Indices & Strategies team. The mission of that team is to design proprietary indices for products such as funds and options. These are large institutional funds whose clients are professional investors such as pension funds, hedge funds, and very wealthy individuals. COG's research has had direct impact on the management of these funds, as evidenced in [C, D] where the Managing Director, Commerzbank states "The robust optimization concepts described in your papers ... have been carefully studied and some of them are incorporated within several proprietary indices and tool". He goes on to say that "Investors regard these risk management tools as an absolute prerequisite". The latter suggests indirect impact of COG's research outcomes as they provide better assurance of risk modelling. Although Commerzbank cannot reveal even the order of magnitude of the value of these managed funds the Managing Director has stated that "The actual value of the funds being managed by these tools ... is respectable."

3. Decision Trees offers commercial software and consultancy to those who produce or trade in energy. They achieve this by transferring scientific research into usable methods in the energy sector. One of the recent challenges in this sector is the high degree of uncertainty in pricing and reserves of energy, caused by the increased use of renewable but less predictable energy. They therefore recognize the value of stochastic optimization for dealing with such uncertainty in the creation of client plans that can increase profits whilst minimizing risk under such uncertainty. A Manager of Decision Trees GmbH [E] makes clear that our research [3] was instrumental for the creation of one of Decision Tree's products when he states: "we were able to implement scenario tree based stochastic optimization models for a whole variety of practical decision making problems in the energy industry. Mainly, our implementation has been based on your publication in the paper ... " which refers to paper [3]. COG's research then led to indirect impact as suggested by the quote "we have enhanced our stochastic optimization software towards the valuation and operational optimization of natural gas storage and natural gas contracts. ... Amongst these customers is Trianel (Germany), Salzburg AG (Austria), OMW (Austria), and ExxonMobil (UK)." The direct impact COG's research has had in the acquisition of Salzburg AG as a client is described as "We have achieved this great success by the practical application of your original pioneering work on scenario tree based valuation of swing options."

4. Trianel is a German utility provider. Its mission is to coordinate and bundle the interests of municipal and communal energy providers in order to strengthen their independence and competitiveness. They operate a combined cycle gas turbine (CCGT) power plant in Hamm-Uentrop in Northern Germany. Some of the shares of the plant are operated based on deterministic optimization [F][G]. Decision Trees was thus able to apply the stochastic optimization tool described above (see 3) on another share to compare its effectiveness to the standard alternative, noting that "stochastic optimization has achieved 1,4% more profit as compared to the best deterministic optimization. This is equivalent to approximately 2 Million Euros additional profit in 2008 yielded by stochastic optimization" [D]. This is an indirect impact of research outputs in [3-5].

5. e&t is the trading company of Energie Allianz. e&t trades bilaterally and through brokers with all important European power traders on the energy exchanges EEX, EXAA, OTE and IPEX as well as on the European Climate Exchange ECX. Two important services which e&t provides are risk management (which has to deal with uncertainty) and energy wholesales trading (where the use of swing options and stochastic optimization can maintain efficient trades and production). In that context, e&t has purchased a suite of software tools based on [3-5] for the operation of the new 750 MW unit in the coal-fired power plant Walsum in Duisburg [H]. On a daily basis this software assesses plans for yearly coal purchases and the associated purchase contracts.

To summarize, the underpinning research has documented impact in the financial and energy sector, and in companies that provide services for these sectors. That impact is also on-going and likely to persist in the years to come.

Sources to corroborate the impact

[A] http://axioma.com/about_axioma.htm. Archived at https://www.imperial.ac.uk/ref/webarchive/nyf on 22nd Oct2013

[B] Axioma Advisor June 2011: Minimizing Downside Risk in Axioma Portfolio with Options. http://www.updatefrom.com/axioma/2011_q2/research_focus.asp. This document describes the use of COG's research in Axioma's products. Archived on 22/10/2013 https://www.imperial.ac.uk/ref/webarchive/myf

[C] Managing Director, Commerzbank describing the use of COG's research.

[D] Commerzbank iQArts Risk Parity confidential report confirming the use of COG's research in figure 1. Available on request.

[E] General Manager, Decision Trees GmbH confirming the use of COG's research by Decision Trees.

[F] Manager of Plant at Hamm-Uentrop confirming the use of COG's research by Trianel.

[G] Manager of Natural Gas Procurement Portfolio with Take-or-Pay Contracts confirming the use of COG's research by Trianel.

[H] Analyst, e&t Energiehandelsgesellschaft confirming the use of COG's research by e&t Energiehandelsgesellschaft.