Summary of the impact

Weather impacts all of our lives and we all take a close interest in it, with every news report finishing with a weather forecast watched by millions. Accurate weather forecasting is essential for the transport, agricultural and energy industries and the emergency and defence services. The Met Office plays a vital role by making 5-day forecasts, using advanced computer algorithms which combine numerical weather predictions (NWP) with carefully measured data (a process known as data assimilation). However, a major limitation on the accuracy of these forecasts is the sub- optimal use of this data. Adaptive methods, developed in a partnership between Bath and the Met Office have been employed to make better use of the data, thus improving the Met Office operational data assimilation system. This has lead to a significant improvement in forecast accuracy as measured by the `UK Index' [A] with great societal and economic impact. These forecasts, in particular of surface temperatures, are pivotal for the OpenRoad forecasting system used by local authorities to plan road clearing and gritting when snow or ice are predicted [B].

Underpinning research

The underpinning research at Bath started as a systematic study of cheap, flexible, and robust, adaptive mesh redistribution methods with evolving mesh density. These meshes are used in numerical algorithms to compute the solutions of evolutionary partial differential equations (PDEs) in several spatial dimensions. Such PDEs are typically discretised on a mesh and the discrete equations solved numerically. If features of the solution evolve on small time or length scales, conventional methods (based on nearly uniform meshes) may fail, whereas the adaptively redistributed meshes provide accurate robust solutions in a wide range of applications.

The research conducted by Budd (a Professor at Bath since 1995) has been centred on devising methods for moving the mesh so that mesh points are concentrated where they are most needed to resolve fine structures, such as atmospheric inversion layers, without additional computational cost. The advantages of this approach over other approaches are that it is computationally simpler and can be readily inserted into legacy software, the mesh regularity can be controlled a-priori and it can explicitly exploit the structures of the underlying PDE.

The underpinning research has evolved from theoretical ideas on mesh redistribution for PDE solutions, to methods which are now directly used in Met Office data assimilation codes. Bath, in collaboration with Simon Fraser University (Canada), developed a procedure for moving mesh methods in one-dimension that could cope with specific singular PDEs [1]. A major advance in the programme at Bath was the extension of mesh redistribution methods to two and three dimensions by using ideas from geometry and fluid mechanics, facilitating the development of the Parabolic Monge-Ampere (PMA) algorithm [2,3]. This method for mesh redistribution combined the equidistribution of an appropriate monitor of the solution with optimal transport methods and the solution of an associated Monge-Ampere equation. The PMA algorithm was first implemented by a PhD student, Williams (2000-2004), and proved to be effective on model problems. In a more developed form, it was the basis of an invited paper [4], which described in detail how the PMA method could either be used to solve PDEs or to derive meshes to better represent the fine structure in their solutions. This paper was of significant interest to the Met Office as many meteorological phenomena occur on small length scales relative to the overall scale of the Earth.

In 2006, an EPSRC/Met Office CASE student at Bath (Walsh), started a programme of research in close collaboration with the Met Office, developing the PMA algorithm specifically for meteorological problems. The PMA algorithm was applied to improve the numerical prediction of severe storms associated with rapid variations in wind speed and wind pressure. Intensive research in this context led to the identification of appropriate monitors of the atmospheric state which in turn were invoked to obtain effective computational meshes [5]. The PMA algorithm, in combination with these monitors, was then used to generate new meshes which increased the resolution of the atmospheric state close to inversion layers, and ground boundary layers, where there are rapid changes in temperature.

Resolving these temperature changes is important for accurate data assimilation calculations. The successful use of PMA in this context led to its incorporation into Met Office data assimilation software, giving much improved resolution of the vertical atmospheric state. This research was followed up by work in 2011-2012 by Walsh and a further Bath PDRA Browne, funded by EPSRC/Met Office funded Knowledge Transfer awards, leading to the development of PMA for fully three dimensional data assimilation calculations. Budd and Browne, in collaboration with the Met Office, have in this process developed a fast, general purpose adaptive 3D adaptive mesh redistribution algorithm based on PMA [6] which is usable for the UK Area weather forecast.

References to the research

References that best indicate the quality of the underpinning research are starred.

Details of the impact

Data Assimilation is an essential part of the Met Office forecasting procedures [C]. However, a significant problem faced by the Met Office is that of assimilating data in the presence of atmospheric inversion layers or other fine structures. Misrepresenting these layers in the computations, leads to spurious correlations between observed data and the underlying physical structures. This has a negative effect on the assimilation of data (for example from radiosondes) into the forecast, degrading the forecast performance. The nature, and societal impact, of this problem is described in the following two quotes from publications authored by Met Office personnel.

A common problem in forecast case-studies is the misrepresentation of inversions and stratocumulus layers in the assimilation due to inappropriate background error covariances, e.g. smooth and broad vertical correlation functions which do not allow accurate fitting of high resolution radiosonde soundings. This inhibits the ability to diagnose realistic stratocumulus layers and boundary-layer structures which then results in poor forecasts. An example of the impact of this problem in the Met Office NWP system happened in December 2006 when poor visibility at Heathrow led to significant travel disruption during the Christmas period. In this instance radiosonde observations were not able to improve the analysis of the inversion and so the fog was not accurately forecast. Met Office Publication [B] (Emphasis added by case study author.) Accurate representation of the boundary layer in NWP models is important for instance in the forecasting of fog or icy roads. Met Office Publication [B]

To address this problem, Budd and his team at Bath have been working with the Manager of Data Assimilation R&D at the Met Office and his colleagues, to develop cheap and reliable methods to better resolve the atmospheric features and increase the accuracy of the data assimilation methods. In a series of collaborative (and partly Met Office funded) projects [D,E,F], they combined the PMA algorithm, developed at Bath and described in [2-6], with the Met Office data assimilation software. This work used the adaptive mesh transformations generated by the PMA algorithm, to rescale the spatial coordinates used in the data assimilation calculation. This rescaling meant that the vertical correlations of the background error covariance matrix in the inversion or ground boundary layers were much better resolved. In particular this has improved the ability of the assimilation system to accurately use high-resolution information like radiosonde soundings. A key breakthrough in this work was the incorporation of an appropriate monitor function of the potential vorticity atmospheric state (as described in [4,5]) into the rescaling algorithm. The PMA algorithm has proved especially appropriate, flexible and robust for this procedure, and has been particularly suitable when dealing with real meteorological data, which can be very noisy.

The development of adaptive grids based on the parabolic Monge-Ampere equation at the University of Bath provides an affordable technique which can be tuned to meet the challenges of real data. It works through a monitor function, which can be chosen to meet a variety of user requirements, and can be smoothed to ensure the resulting grid can be safely used in an operational environment. [G] Letter from the Manager of Data Assimilation R&D at the Met Office

Adaptive data assimilation software, based directly on the research at Bath described in Section 2, was first incorporated into the operational data assimilation code for the Met Office 4km grid UK models in November 2010 [A,B]. Operational codes make forecasts every six hours [H] and the operational codes, incorporating the PMA algorithm, have been used to forecast the UK weather for the last three years. More advanced Met Office codes based on the results in [6] are being developed and are forming part of the ongoing Data Assimilation research at the Met Office.

The application to data assimilation is particularly suitable for this technique, as the resulting grid is only used to define spatial correlation structures, not to apply a flow solver. As a result this technique has been used in our operational UK analysis for the last 3 years and an extended formulation will be incorporated for trialling within the next 12 months. [G] (Emphasis added by case study author.)

A direct consequence of this work is an improvement of the Met Office forecasting skill in terms of the so-called UK Index (which is a measure of the forecasting skill of limited-area NWP models over the UK and is based on forecasts of selected parameters and for a selected set of positions verified by comparison with available station observations across the UK at 3- to 6-hourly intervals).

... the adaptive mesh transformation led to positive impact in the forecast skill of UK models both in winter and summer. Analysis RMS errors are reduced with respect to radiosonde, aircraft, SEVIRI and ground GPS observations for both periods. Background RMS errors are reduced with respect to aircraft, surface and ground GPS observations for both periods and also with respect to radiosonde observations for relative humidity in the lower part of the troposphere and for potential temperature around the inversions. These results are consistent with the change in the monitor function structures coming from the updated normalization procedure and recalculation of the adaptive mesh within the nonlinear minimization procedure. These led also to improvement of the background state in the full cycled analysis/forecast system and therefore to better representation of the vertical structure of the boundary layer. For these reasons this new version of the adaptive mesh transformation was implemented operationally in the Met Office data assimilation system in July 2011 for UKV and UK4 models. Met Office Publication [A]

Obviously, the improvement of the Met Office forecasting skill has economic and societal impacts. In particular, the enhanced resolution of the ground boundary layer, provided by the PMA algorithm, has led to an improvement in the accuracy of the prediction of fog hazards and road temperatures. These temperature predictions are used, for example, to provide input for the Met Office OpenRoad software [J] that is employed to advise local councils on ice hazards and the need (or not) for road gritting.

The improvement of 2m temperature forecasts is relevant for the Met Office OpenRoad system which provides 24 hour forecasts of road state to companies and local authorities to help maintain essential road services, mainly in winter. For this reason, the adaptive mesh transform was implemented operationally in two Met Office high-resolution limited-area models (UK4 and UK1.5) on 2 November 2010. Met Office Publication [B]

A new method of adapting computational grids to the expected solution is now being exploited in the high resolution analyses used to drive the short-range forecasts for the UK. Particular benefit is found in predicting low-level temperatures, which is very important for maintaining the road network in a safe condition and for predicting fog. [K] Email from the Manager of Data Assimilation R&D at the Met Office

In the winter of 2011/12, the Met Office provided OpenRoad based forecasts for over 350 routes in the UK. The use of OpenRoad reduces the impact of cold weather on road networks, in particular on road safety, and, via more accurate forecasting of road temperatures, leads to a more cost-effective use of grit supplies (gritting can cost a council in the order of £10k to £15k per day). Moreover, since salt is a corrosive substance, avoidance of gritting when it is not necessary, leads to savings for road users in general and to a reduction of damage to the transport infrastructure in particular. In the USA, it has been estimated that the total costs, including these indirect ones, of using salt are three times greater than the direct costs. But even this is likely to underestimate the total costs, as environmental damage caused by salt run off into the ecosystem is not taken into account.

Sources to corroborate the impact

[A] C Piccolo and M Cullen, A new implementation of the adaptive mesh transform in the Met Office 3D-Var System, Q. J. R. Meteorol. Soc, 138, (2012), pp. 1560-1570. DOI:10.1002/qj.1880

[B] C Piccolo and M Cullen, Adaptive mesh method in the Met Office variational data assimilation system, Q. J. R. Meteorol. Soc., 137, (2011), pp. 631-640. DOI:10.1002/qj.801

[C] The importance of data assimilation to the work of the Met Office is described in
http://www.metoffice.gov.uk/research/news/ndp-data-assimilation

[D] 2006-2010 EPSRC CASE Award with the Met Office, £65,000, for Prof C Budd to support the work of E Walsh on `Monge-Ampere methods for adaptive grid generation'.

[E] 2010-2011 EPSRC Knowledge Transfer Grant KTA Grant KTA008_Budd for Prof C Budd £14,748 (to support E Walsh) on `Adaptive numerical methods for weather forecasting'.

[F] 2012 EPSRC Knowledge Transfer Grant KTA092_Budd with the Met Office for £60k, Adaptive mesh data assimilation methods for problems in three dimensions (to support P Browne).

[G] Letter of support from Manager Data Assimilation Research and Development at the Met Office, describing the use and importance of the Bath PMA algorithm.

[H] The nature of an operational forecast is described in
http://research.metoffice.gov.uk/research/nwp/numerical/operational/

[J] The OpenRoad software is described in: http://www.metoffice.gov.uk/roads/openroad

[K] Email from Manager Data Assimilation Research and Development at the Met Office, describing the impact of adaptive methods on forecasting low temperatures and fog.