Summary of the impact

Our research in complex user interface design for environments where there is need to support a high cognitive overload as well the need to support variability of user interface design has led to impacts on the design of products and services in two organisations addressing the domains of crisis management software and nuclear power plant industry. In the former case, research arising from our investigations of user behaviour in complex systems has been applied by adopting our Variable Uncertainty Framework (VUF) into the software product `VSL Planner' developed by VSL Systems AB and the `XVR' product developed by E-Semble Systems. The XVR product has recently been sold to the London Fire Brigade. Related to this, our research on complex task analysis, specifically in abnormal situations resulted in a second conceptual framework that extended the Task Complexity (TACOM) methodology that has been used to inform and guide the Korean Atomic Energy Institute in their work on designing and implementing modern control rooms for atomic power stations in South Korea. The safety critical nature of these interfaces means the potential impact of this work is very high, as any failure could have catastrophic consequences.

Underpinning research

The Interaction Design Centre (IDC) at Middlesex has been researching user interface design issues since 1995. The Centre has previously been led by Professors Harold Thimbleby (now at Swansea) and Ann Blandford (now at UCL) and is currently led by Professor BL. Wong.

Since its inception IDC has been successful in obtaining over 25M in research funding from a range of funding bodies including ESPRC, JISC, Eurocontrol, and various EU FP programmes to study human computer interaction requirements in a range of domains such as low literacy skills, ambulance dispatch, and first responder crisis training. Recently the centre has focussed on the visualisation needs in security domains where the data characteristics such as velocity, volume variability are predominant. The range of domains has necessitated engagement with a broad group of stakeholders including users with low literacy, older populations and intelligence analysts at both national and international levels.

In the context of these domains a range of task complexity studies and human-system interface studies have been conducted leading to outputs that include software tools with new visual paradigms [7] and new methodologies [4].

Tasks that require a high cognitive overload have been explored in contexts such as information seeking behaviours on web sites supporting the services of the National Citizens Advice Bureau (CAB) [6] and in the design of crisis management training situations as part of research carried out in the EU funded CRISIS (CRitical Incident management training System) project. The latter research led Professor BL. Wong and his team to the development of the Variable Uncertainty Framework — a conceptual tool for managing variability and complexity of training scenarios along dimensions such as: situational complexity, the number of events occurring simultaneously, and randomness of events [2]. Our earlier research on how situation awareness processes and their antecedents such as in the context of emergency medical dispatch [1] provided an earlier intellectual contribution to the VUF.

The research expertise and environment in the area of complex user interface design in the Interaction Design Centre was instrumental in Dr Dong-Han Ham being funded by the Korean Atomic Energy Institute (KAERI) on a contract research project (`Development of Model-Based Method for Evaluating Cognitive Task Complexity and Human-Machine Interface Designs in Nuclear Power Plants'. Amount: GBP 75,000, Period: March 2007 ~ February 2010. Grant code: 2010-00001029). In this project, Dr Ham conducted research for complex user interface design requirements for nuclear power plants (NPP). This research developed a novel conceptual framework for deriving various complexity factors and understanding complexity-related issues with a specific aim to improve TACOM (TAsk COMplexity) — a widely used method for evaluating procedure-based tasks in NPPs in South Korea. The research paid particular attention to the sufficiency and efficacy of TACOM's three methodological tools, the process, the cognitive task analysis method and the set of guidelines. The research outcomes were reported to KAERI in a confidential report [3] and a subset of the findings were published more widely [4] [5].

References to the research

This research was based on competitively funded projects, with robust peer review systems. The outcomes from the research were published in leading peer review journals and conferences in the field.

Grants

1. Development of Model-Based Method for Evaluating Cognitive Task Complexity and Human-Machine Interface Designs in Nuclear Power Plants'. Amount: GBP 75,000, Period: March 2007 ~ February 2010. Grant code: 2010-00001029).

2. CRISIS (CRitical Incident management training System); FP7 Security Call; Ref: 242474

Details of the impact

Systems such as that found in nuclear power plants and air traffic control place a high cognitive load on human operators as they perform tasks that require information search, integration and deep inference leading to a critical decision. Mastering such complex environments requires extensive training and increasingly, computer-based simulations form an essential element of such training. Although improvements in display technology are important, underlying issues prevalent in risk laden environments means that improvements do not necessarily provide for productive and reliable user performance. In particular, design and evaluation of task procedures, information displays for complex systems and associated training have become a critical issue. As the task complexity issue is very important to improve the safety of these (safety-critical) complex systems many organizations utilizing these complex systems require methods for evaluating task complexity or evaluating interface designs for this requirement.

Our research on information seeking behaviour, goal oriented domain analysis techniques and the influence of user interfaces on critical decision making has addressed the issue of cognitive overload in safety critical systems and how to provide training to support users of such systems. The research described earlier has led to two overarching frameworks, both of which have had direct impact on external research users.

Frameworks for measuring complexity of user interface tasks

In the first instance, within the context of the nuclear power industry, the analysis provided by Dr Ham on the efficacy of widely available TACOM methodology for abnormal situations has further materially, informed and guided the Korean Atomic Energy Institute in their work on designing and implementing modern control rooms for atomic power stations in Korea.

Guidance has been provided in two specific areas. First, a conceptual framework for complexity factors that was developed formulated on a sound theoretical basis of cognitive systems engineering and system thinking has now been made available to KAERI. Secondly, practical guidance for using task complexity factors in the design and evaluation of task procedures and information displays for NPPs has also been utilised by KAERI.

Much of the guidance has had its main impact since March 2010 following the completion of a number of task complexity studies. The research outputs and case studies developed in the underpinning research are now used as a significant reference point for the on-going research activities of Korea Atomic Energy Research Institute (KAERI).

Following the research undertaken by Dr Han, task complexity has been regarded as a critical component that should be addressed for the systematic design and evaluation of task procedures and information displays in complex systems particularly for NPPs.

Additionally the research outputs have offered a new way of addressing complexity-related research issues in NPPs. For example, the research outputs have helped systems designers and human factors engineers working in NPPs to evaluate the design of task procedures and information displays from the perspective of cognitive systems engineering.

Colleagues working at KAERI (Dr. Wondea Jung (wdjung@kaeri.re.kr) and Dr. Jinkyun Park (kshpjk@kaeri.re.kr)) have been the primary beneficiaries of the impact. They have been able to utilize the research and develop further research outputs based on the initial work done at Middlesex [S3].

Frameworks for designing variable training scenarios addressing complex crisis situations

The CRISIS project led by Prof. Wong developed the Variable Uncertainty Framework — a conceptual tool for managing variability and complexity of training scenarios along dimensions such as: situational complexity, the number of events occurring simultaneously, and randomness of events [2].

VSL Systems AB) a commercial software services company has implemented key functions of the framework in their software product: `The VSL Planner' including the following: time-based triggers initiating events at a precise time or between time-intervals; events triggered from other events; manual triggers and others. VSL acknowledge that the ideas of the VUF are powerful and provide a simpler means of increasing the variability of the training situations with minimum user interaction [S4]. VSL are in the process of commercialising their products.

Additionally, E-Semble (http://www.e-semble.com), another simulation based training product organisation has also incorporate a part of the VUF (spatial aspect triggers) by developing a bespoke software development kit (SDK) into their XVR product.

As part of the investment into improved training facilities for London Fire Brigade, Babcock, the infrastructure services company selected E-Semble to implement the XVR platform into two newly built Incident Command Simulation (Sim) suites located in London. The XVR platform generates a range of scenarios including those derived from the CRISIS project and its deployment is reported in a press release to the trade magazine Fire & Rescue [S5]

Sources to corroborate the impact

S1. KAERI is a research led organisation and as such it conducts foundational and applied research relevant to the nuclear power industry. In the initial case KAERI funded Dr Han to perform research (`Development of Model-Based Method for Evaluating Cognitive Task Complexity and Human-Machine Interface Designs in Nuclear Power Plants'. Amount: GBP 75,000, Period: March 2007 ~ February 2010. Grant code: 2010-00001029) to conduct research for complex user interface design requirements for nuclear power plants (NPP).

S2. This research (in particular the papers [4,5,6,7] were used and co-developed by KAERI). These papers were cited further in other research.

S3. Letter from researcher at KAERI describing influence of research on designing for task complexity.

S4. Letter from Dr John Jenvald, CEO, VSL Systems AB.

S5. E-Semble Press Release; 30-09-2013. http://www.e-semble.com/en/About_E- Semble/News/News_overview/Press_Release_Babcock_invests_in_immersive_training_fo r_the_London_Fire_Brigade_.html

The press release was also reported in the Fire & Rescue Magazine, Page, 5, December Issue, 2013.

S6. Letter from Martin Boosman, Founder, Director of Strategy, E-Semble Systems.