Summary of the impact

New business models, technological innovations and global markets, demand that engineering firms better manage how they grow and achieve reliability during product development. A reliability growth modelling framework, developed from research at the University of Strathclyde, is being used by and influencing UK industry practice in the aerospace & defence sector. Our model underpins the modern approach to reliability growth management in Aero-Engine Controls (a Rolls-Royce company), Selex ES, and contributes to the Reliability Case required by the Ministry of Defence. The Strathclyde model is included in the international reliability growth standard (IEC 61164) which is adopted globally by manufacturing firms and procurement agencies.

Underpinning research

Context:

The motivation for this avenue of research began in 1996 when insights gained through a PhD studentship, sponsored by Lucas Electronics, generated ideas for changing the way we model growth in product reliability during design and development. The ideas tabled by Strathclyde researchers gained support from industry (in particular Lucas Aerospace, later TRW Aeronautical Systems) within a sector undergoing considerable change. The 1990's saw, for example, the emergence of service level agreements in the commercial aerospace sector; so-called "Power by the Hour" contracts. New generations of aircraft had technological innovations including more electronic components, whose reliability was less well understood than conventional mechanical parts. Commercial and defence customers required suppliers to produce a Reliability Case to document an argument about the product supported by evidence detailing growth in reliability performance during the early product life, especially from design conception through product development. To meet such challenges, the UK aerospace industry required a comprehensive set of tools to support the management of reliability growth. Research to achieve these goals was developed in partnership with the Strathclyde researchers through a funded research project titled, Reliability Enhancement Methodology and Modelling (REMM) — see Section 3.

Key Research Insights / Outcomes:

Our primary research involved: developing a coherent stochastic modelling framework to support useful reliability growth management decisions grounded in the engineering design and development process; designing scientific methods to instantiate the growth model both using subjective engineering judgement and event history data; developing statistical inference methods to support estimation of current and future uncertainties about reliability; providing an updating mechanism to revise reliability estimates in light of new information gained from engineering analysis, test and use; undertaking a designed evaluation study of modelling components, as well as the framework through theoretical, empirical and applied research.

The key outcomes of our research in relation to this impact case are:

a) A framework to construct a stochastic model to describe the reliability of a complex system during its design and development that can be used to support decisions concerning effective and efficient reliability growth (see ref [1] in Section 3).

b) A process for eliciting structured expert engineering judgment during design and development of complex engineering systems that is grounded in the theory of structured expert judgement elicitation (see ref [2]) and evaluated through industry application (see ref [3]) .

c) A stochastic model and associated statistical inferential techniques to support analysis of growth using information gained from engineering analysis and test data generated during the engineering development process (see refs [4] and [5]).

d) A theoretical framework to support decision-makers to assess and manage reliability targets across their supply chain (see ref [6]).

Key Researchers:

The research was conducted by Professor Lesley Walls and Professor John Quigley, starting in the late 1990's and remains on-going. Walls and Quigley have been members of staff at the University of Strathclyde since 1994 and 1997, respectively. Journal articles have been published over the period 2001-2013.

References to the research

Other evidence for quality of research:

Peer reviewed grant funding (1998-2004, in two phases) under the DTI Civil Aviation Research and Development (CARAD) Programme, with an industrial consortium involving TRW Aeronautical Systems (later Goodrich), Smiths Aerospace, FR-HiTemp, Warwick Manufacturing Group, BAE SYSTEMS, Rolls Royce, AgustaWestland, for a collaborative research project, titled Reliability Enhancement Methodology and Modelling (REMM). The Ministry of Defence was represented on the project board. Total project funds £1.95M, of which £550k was awarded to Strathclyde.

Details of the impact

The outcomes of our research have provided the UK aerospace and defence industry with an improved process and useful scientific tools to support evidenced based development decisions for high value products for which reliability is a key business performance measure.

The REMM process has been adopted in its entirety by Aero Engine Controls (AEC), part of the Rolls-Royce group. The AEC Technical Manager with responsibility for New Product Innovation has stated (Source 1) that "due to the insights gained through REMM, my company fundamentally changed its reliability culture". He further explains that: "we have introduced a revised reliability process, heavily based on the REMM principles, on every new product introduction since 2008. In-house engineering procedures have been developed which specify the mandatory use of the REMM process with a design gate checklist..." Further, he comments that: "A particularly important role has been played by the methods developed by Strathclyde for elicitation of engineering judgement and a model to combine information gained from such judgemental data together with statistical mining of operational experience data to provide estimates of reliability."

Our reliability growth modelling has had significant impact because, firstly, it was aligned with the needs of decision-makers and integrated with key stages in the product development cycle and, secondly, the model was generated from sound theory, but operationalized to be populated with available, relevant data so that it was meaningful to the users. The value of our research as an innovation in reliability growth management has been recognised by the professional community through multiple awards. AgustaWestland won the 2011 UK Safety and Reliability Society prize for a paper describing what the firm termed "a design influence process" during the development of a new helicopter. The Principal Reliability Engineer explained that "The basis for much of this analysis was techniques developed during the ...Reliability Enhancement Modelling Methodology (REMM) project" (Source 2).

Our impact has extended beyond the companies involved in the original REMM project. To better understand how they could improve product reliability and support packages, Selex ES engaged with us in 2009. The Emerging Technologies Manager (Source 3) stated that "The REMM modelling process has now been put in place with the assistance of Strathclyde to address the necessary building blocks for reliability process improvement." He then explains that the model is allowing Selex ES to "prioritise aspects of design to address for reliability improvement" and concludes that "the implementation of REMM has been recognised by winning bronze at the 2010 Finmeccanica Innovation Award scheme".

Our modelling framework has also impacted the Reliability Case presented by firms to the Ministry of Defence (MoD). Airborne Systems Ltd. (at the time called Irvine Aerospace) used our modelling for a novel reliability assessment allowing the MoD to better identify and assess the risk associated with procuring a high value aerospace product. Source 4 is an invited discussion piece in a professional trade publication with a description of the application of the methods to Airborne Systems Ltd. Acknowledging the impact of REMM on industry, the MoD's Principal Reliability Engineer (Source 5) stated that "the fuller impact of the research is being observed in the evidence of analysis presented in the Reliability Cases submitted to the MoD for review."

Our process and methods also have more general applicability to other high value manufacturing industries beyond the aerospace and defence sector because our research is one of three growth models in the reliability standards produced by the International Electrotechnical Committee (IEC). The IEC is the international standards and conformity assessment body for all fields of electrotechnology. International standards by their very nature have a global impact since standards are mandated or advised by customers within contracts and can be transferred through the supply-chain so that SMEs as well as multi-nationals benefit. All international reliability standards are developed by the IEC, but are also distributed through national agencies, such as the British Standards Institute (BSI). The current reliability growth modelling standard, IEC 61164, is dual numbered as BS EN 61164 by BSI (Source 6). This issue of the standard is the one used by UK and international firms over the period 2008 to the present. A revision of the international reliability growth management and modelling standard is under development; Prof Walls is the UK expert on the project team (Source 7). Elements of our REMM research have also been included in an informative annex, called the `elicitation of Bayesian priors for reliability modelling', of the 2013 edition of IEC 61710, a standard which covers general analysis of reliability data from system test and operation (Source 8).

The Chairman of the IEC Technical Committee (TC) with responsibility for international reliability standards (i.e. TC 56), himself an experienced reliability practitioner with a distinguished career in Danish companies such as Nokia and Bang & Olufson, has commented (Source 9) that the "academic research of Profs Walls and Quigley is fundamental not only to providing industry with sound, robust modelling mechanisms, but also making these models accessible and understandable through an exposition within the language of standardisation."

Sources to corroborate the impact

1. Letter from Technical Manager New Product Introduction, Aero-Engine Controls, UK.
2. http://www.nottingham.ac.uk/engineering/conference/ar2ts/prize-winners-2011.aspx documenting Safety and Reliability Society prize awarded to Agusta Westland.
3. Letter from Emerging Technologies Manager, Selex ES, UK.
4. Quigley, J.L. and Bedford, T.J. and Walls, L.A. (2009) Empirical bayes estimates of development reliability for one shot devices. Safety and Reliability: the UK Safety and Reliability Society, 29 (4). pp. 35-46. ISSN 0961-7353.
5. Letter from Principal Reliability Engineer, Defence Equipment and Support, Ministry of Defence, UK.
6. http://webstore.iec.ch/webstore/webstore.nsf/Artnum_PK/31953 and http://webstore.iec.ch/preview/info_iec61164%7Bed2.0%7Den.pdf giving link to purchase and extract of the current international standard on reliability growth methods IEC 61164.
7. http://www.iec.ch/dyn/www/f?p=103:14:0::::FSP_ORG_ID,FSP_LANG_ID:3997,25 and
8. http://www.iec.ch/dyn/www/f?p=103:14:0::::FSP_ORG_ID,FSP_LANG_ID:2335,25 documenting expert role played by Prof Walls on IEC standards
9. Letter from Chair of International Electrotechnical Committee Technical Committee 56 on Dependability, Denmark.