Summary of the impact

A series of econometric methods and software, designed by a team of econometricians at Oxford, have been adopted as standard by a large range of governmental bodies, international agencies and businesses. The econometric methods are designed to model and forecast high-dimensional, evolving economic processes facing multiple structural shifts, while the econometric software (PcGive) implements the resulting best-practice procedures. The application of these methods have resulted in more appropriate empirical models, improved robust forecasts, and, consequently, better decision making by these bodies.

Underpinning research

Since the early 1980s, time-series econometricians at Oxford have focused on developing robust techniques for modelling and forecasting important macroeconomic variables, and implementing the procedures in publicly-available software. The key researchers involved in this research at Oxford University are: Sir David Hendry, Professor of Economics and Fellow of Nuffield College since 1982; Jurgen Doornik, Research Fellow at Nuffield College since 1989; and Jennifer Castle, Fellow at Magdalen College since 2006, Postdoctoral Research Fellow, Department of Economics 2003-06.

Research undertaken by this group has established the importance of structural shifts in economic relationships as the main cause of forecast failures. In the face of structural breaks, so-called equilibrium-correction models (ECM - the de facto standard in empirical econometrics) face serious forecasting problems. The reason is that such models are designed to return to equilibrium, but, after a structural shift, this is now the wrong equilibrium. While forecasting performance of ECM models can be improved through `mechanistic' corrections, Oxford researchers have shown that such techniques are outperformed by approaches that model the break process explicitly [R6, R3].

Economies both evolve and experience sudden changes, which interact with the plethora of highly interrelated macroeconomic variables arising from agents' actions, such that many features of econometric models cannot be derived by prior reasoning. Model selection can reduce this complexity, intrinsically involving empirical discovery and theory evaluation [R5]. The model selection and evaluation techniques developed by this research group have been carefully validated, allowing investigators to handle very general models with many variables, long lag lengths, and non-linear functions, while at the same time taking into account outliers, data contamination, and parameter shifts [R1, R2]. These general representations retain the best available theory while being simplified to well-specified representations that are rigorously evaluated against the data to ascertain their validity [R4].

Research on the theory of model selection has been accompanied throughout by the development of publicly available econometric software. PcGive, a user-friendly interactive econometric modelling package for estimation, testing and forecasting, designed by members of the Oxford team, was first released in the mid-eighties and is now in its fourteenth version. Version 10, released in 2001, was rewritten in the matrix programming language, Ox, developed by Doornik, permitting more efficient code development and greater flexibility for users. Econometric model selection packages written in Ox to implement the general-to-specific (GETS) modelling approach described above, have culminated in the recent automatic search algorithm, Autometrics within PcGive (both using and contributing to advances in the theory of model selection [R1, R5, R6].

References to the research

Research Quality

Journal of Econometrics is the top field journal for econometrics. It was rated as "4*" by the ESRC- RES review of UK Economics, and classed as AA in the Combes-Linnemer (2010) ranking of economics journals.

*denotes publication returned as part of RAE 2008

**denotes publication returned as part of REF 2014

Research Grants

This research has been supported by many grants, including a Leverhulme Personal Research Professorship on Forecasting,1995-2000, for Hendry, and the following ESRC awards:

• Automatic Tests of Model Specification (2006-2008, £230K)
• Extending the Boundaries of Econometric Modelling (2004-2007, £133K, rated Outstanding)
• Professorial Fellowship, Hendry: Economic Forecasting (2003-2006, £314K, rated Outstanding)
• Modelling, Forecasting and Policy in the Evolving Macro-Economy (2000-2002, £289K, rated Outstanding)
• Modelling Non-Stationarity in Economic Time Series (1998-2001, £355K, rated Outstanding)
• Econometrics of Economic Forecasting (1996-1999, £291K, rated Outstanding)
• Modelling Cointegrated Processes (1994-1997, £300K, rated Outstanding)

Current support is provided by the Open Society Foundation, and matched by the James Martin Foundation (2010-2015, totalling $10m).

Details of the impact

Research into the theory of modelling and forecasting by the Oxford researchers has informed the development and the use of the PcGive software for empirical analysis. The associated documentation (Doornik, J.A. and Hendry, D.F. Empirical Econometric Modelling using PcGive, Timberlake Consultants Press, London (2013, 2009, 2007, 2001) both introduces the user to the underlying econometric methodology and provides a series of tutorials explaining the detailed use of PcGIve. The automatic search algorithm Autometrics, [R5] has been part of PcGive from version 12 (release date August 2007), permitting econometricians to use automatic methods for model selection and evaluation, and to detect and model structural breaks. PcGive13 (released 2009) extended the application of Autometrics to non-linear models. Automatic model selection procedures save considerable labour time, allowing economists to focus on their economic expertise while widening the scope for empirical discovery when initial ideas are incomplete. The importance of structural shifts as a cause of forecast failure [R3] makes automatic detection and handling of such breaks vital.

PcGive software has been adopted in many governmental institutions and businesses, and is now seen as `the' industry standard. The Bank of England, the Federal Reserve Board, International Monetary Fund (IMF), The World Bank, Inter-American Development Bank and many European central banks use PcGive, implementing the methodological developments as they advanced from the early research into the software. This is confirmed by the data on purchases of PcGive by central banks. Based on information from 2011 and 2012 only, purchases or renewals were made by central banks from: Argentina, Barbados, Bulgaria, Czech Republic, Denmark, ECB, France, Italy, Mongolia, Nigeria, Norway Peru, Philippines, Uganda, Bank of England, the Federal Reserve Board, and FRB Dallas. Other institutional purchases for the same years include: Bank for International Settlements, IMF, US Bureau of Labour Statistics, US Department of Transportation, European Commission, Statistics Netherlands and Norway, The World Bank, The Law Society. While in the same period, companies making purchases or renewals include: Allstate Insurance, Fidelity Investments, and ITAU Unibanco (among others). Motability, a national charity helping disabled people with personal mobility, and one of the largest automobile lease-companies in the UK, recently adopted Autometrics to develop better forecasts three years ahead for the residual values of vehicles returned when their leases had ended. Total non-academic sales were equivalent to almost 2500 single-user copies over 2008-2013. Detailed information on the number and range of sales of the PcGive software and the OxMetrics platform developed at Oxford for the period 2008 to 2013 may be obtained from Timberlake Consultants Ltd, the private distributor [C1]

The impact of the recent developments in econometric methodology and software is evidenced in empirical work using automated general-to-specific model selection (GETS) and Autometrics produced by central banks and international institutions as documented in the following references. A few key examples include:

• Norges Bank employs the modelling software in empirical studies. The Director of Research at the Norges Bank (the Central Bank of Norway) states that OxMetrics is the standard software offered to their economists and is used in econometric research, policy analyses and staff training [C2, C3].
• The Federal Reserve uses the methodology and the associated software to estimate its models and to inform its thinking and approach to modelling [C4, C5].
• MOSES: Model of Swedish Economic Studies", an "aggregate econometric model for Sweden...intended for short-term forecasting and policy simulations" constructed by Sveriges Riksbank (Swedish Central Bank). Autometrics is described as "an essential ingredient in building MOSES" [C6. p. 14].
• The UK Office for National Statistics has used the GETS modelling approach to produce flash estimates of European labour costs for use in policy-making [C7].
• The Agricultural and Rural Development Unit, Africa Region of the World Bank has used the general-to-specific methodology in Autometrics to investigate key food prices in developing economies [C8].
• Researchers at the Czech National Bank used the general-to-specific methodology in Autometrics for testing banking sector resilience [C9].

The successful commercialization of the software has been supported by the provision of tailor- made extensive training courses, designed by the Oxford team. These have been used by central banks and international institutions: for example week-long courses have been run at the IMF (annually, 2002 to 2011), the central banks of Japan (2009) and Brazil (2010), and the ECB (annually, 2010 onwards), as well as annual public attendance courses for academics, students and private users. Timberlake Consultants, jointly with our team and other researchers, organize OxMetrics conferences as a forum where academics and practitioners exchange ideas and experience, annually in Europe (usually London), and biannually in Washington DC [C1].

Private sector companies like Absolute Strategy Research use the team's approaches and software in their commercial activities. Absolute Strategy Research place the econometric methodology developed by Professor Hendry and his team, and the associated computing software, "at the heart of much of the econometric analysis that we undertake" and confirm that it plays a key role in their commercial success [C10]. The Joint Managing Director stated: that "the econometric methodology that [Hendry's team] have developed has helped us specifically in the last year in our modelling of European and Global corporate earnings... as well as in the creation of a suite of Leading Indicators for Eurozone, UK, Asia and the USA." [C10]

Sources to corroborate the impact

[C1] The Director of Timberlake Consultants Ltd, corroborates the number and range of institutions and companies that use the software, participants of training courses, conferences and workshops and feedback from participants (Letter on file).

[C2] Director of Research Department - Financial Stability, Norges Bank, corroborates the impact of the research and software at the Norwegian central bank (Letter on file).

[C3] Hammersland and Traee (2012) The Financial Accelerator and The Real Economy: A Small Macroeconomic Model for Norway with Financial Frictions, Norges Bank Staff Memo no 2. (http://www.norges-bank.no/Upload/Publikasjoner/Staff%20Memo/2012/StaffMemo212.pdf) This is an example of economic modelling undertaken by economists at Norges Bank using the Autometrics methodogy (p.6 and references)

[C4] Senior Economist at the Federal Reserve Board (until 2013) can corroborate the impact of the research and software at the FED.

[C5] Neil R. Ericsson and Erica L. Reisman, (2012), `Evaluating a Global Vector Autoregression for Forecasting', International Finance Discussion Papers 1056, Washington: Board of Governors of the Federal Reserve System (http://www.federalreserve.gov/pubs/ifdp/2012/1056/ifdp1056.pdf) Example of the use of Autometrics to evaluate models used for macroeconomics forecasting (acknowledgements and pp.7-8)

[C6] Bårdsen, Gunnar & den Reijer, Ard & Jonasson, Patrik & Nymoen, Ragnar, (2012), "MOSES: Model of Swedish Economic Studies," Economic Modelling, 29, 2566--2582; also in Working Paper Series 249, Sveriges Riksbank, section 4.2 p.14. www.riksbank.se/upload/Dokument_riksbank/Kat_publicerat/WorkingPapers/2011/wp249.pdf

[C7] Graeme Chamberlin, "European Labour Costs", European and Labour Market Review, 3, 11, November 2009, Office for National Statistics, pp 32-39 (http://www.palgrave-journals.com/elmr/journal/v3/n11/pdf/elmr2009190a.pdf)

[C8] Loening, Josef L., Durevall, Dick and Birru, Yohannes A., "Inflation Dynamics and Food Prices in an Agricultural Economy: The Case of Ethiopia", World Bank, 2009, p.25, and footnote 20. https://openknowledge.worldbank.org/bitstream/handle/10986/4163/WPS4969.pdf?sequence=1

[C9] Adam Gersl & Petr Jakubik & Tomas Konecny & Jakub Seidler, "Dynamic Stress Testing: The Framework for Testing Banking Sector Resilience Used by the Czech National Bank," Working Papers 2012/11, Czech National Bank, 2012, p.14 http://www.cnb.cz/miranda2/export/sites/www.cnb.cz/en/research/research_publications/cnb_wp/d ownload/cnbwp_2012_11.pdf

[C10] Managing Director, Absolute Strategy Research Ltd. corroborates the contribution of the forecasting methodology, the associated software, the conferences and workshops on the work of the company (Letter on file).