Submitting InstitutionMiddlesex University
Unit of AssessmentComputer Science and Informatics
Summary Impact TypeTechnological
Research Subject Area(s)
Information and Computing Sciences: Information Systems
Medical and Health Sciences: Public Health and Health Services
Economics: Applied Economics
Summary of the impact
The case described shows that our research reporting on the limitations
and challenges associated
with traditional approaches to presentation and management of information
search results for
different types of users has led to design and implementation of
non-conventional user interface
technology - INVISQUE (Interactive Visual Search and Query Environment
). This technology
has had a direct influence and usage in a range of domains including
information seeking in
Citizens Advice Bureaux and in the security domain both nationally (UK
Ministry of Defence) and
internationally (US Dept. of Homeland Security). The research has created
a community of
practice around the emerging field of Visual Analytics and has formed the
basis of a successful
FP7 project (EC grant €13.1M) bringing together a consortium of 18
leading Visual Analytics researchers (e.g. PNNL, University of Konstanz,
City University London),
and police end-user partners, from across the UK, Europe and the US.
The use of lists to present information search results is a common
approach (e.g. Google search
results). However there is evidence that users often get lost or lose
sight of their original goals after
drilling down, backtracking and so on . When further considerations
such as the domain of
search, and the type of user (e.g. the level of literacy) are taken into
account then limitations of
such presentation formats become further pronounced. Researchers at
Prof Ann Blandford, Prof Harold Thimbleby (1999-2003), and more recently
Prof William Wong
(since 2003) have been researching the design of user interfaces to
present complex information in
novel visual forms taking into account the complexities associated with
types of users. The impacts
described in this case study arise from several strands of research
In 2005 two key papers reported on research exploring information seeking
behaviours utilised in
digital libraries (DL) and websites supporting the services of the
National Citizens Advice Bureau
(CAB). The papers noted that experts could deploy a repertoire of simple
but effective strategies
that could be used to generate design ideas for user interfaces that aid
`ordinary' users.  .
Specific cases identified included guidance on how to integrate adaptive
help systems into DL
systems. Both papers utilised empirical data collected in situ through
observation studies, cognitive
task analysis and think aloud techniques.
Whilst one study  explores strategies based on expert users, other
research  investigates the
effects of low literacy skills on information retrieval. The results
indicated the increased likelihood of
getting lost; taking longer; the inability to backtrack amongst other
issues. The latter issue was also
reported in a subsequent report produced for the Joint Information Systems
following a study examining search behaviours of first year business
students in higher education.
Additionally and grounded in HCI and cognitive systems engineering, the
need for novel interaction
and visualisation techniques to support spatial-temporal reasoning in
complex environments has
led to applications such as air traffic control where research on novel
representations  has been conducted. More recent research exploring
techniques for displaying
multiple dimensions of information across physically distinct layers of
information has been
reported in .
Research in this area is broadly coalescing around the notion of visual
analytics - an area of
interaction design that has now become firmly established at Middlesex.
Visual analytics is the emerging science of analytical
reasoning supported by highly interactive
visual interfaces that combine a range of automated analysis techniques
with user driven
visualisations. The integrated technologies collectively enable real-time
analytic interaction with
very large complex datasets from multiple sources to support effective
and decision making.
A research outcome from our activities in visual analytics is INVISQUE, a
interface technology. The original prototypes were developed in 2009
following funding from JISC
and were refined and applied to the areas of digital resources in digital
libraries  and in the
context of low literacy users . Evaluation of INVISQUE showed
significant improvements in
performance. The non-conventional INVISQUE UI, in the context of low
literacy users, was found
to significantly improve task completion and success in finding
information, reduce lost-ness and
back-tracking. INVISQUE helped them see what the system retrieved, and how
grouped the results, showed links and revealed associations between data.
The results indicated
the applicability of this technology to the security domain hence the
impact on that community.
References to the research
This research was based on competitively funded projects, with robust
peer review systems. The
outcomes from the research were published in leading peer review journals
and conferences in the
1. Kodagoda, N. and B. Wong. Effects of low & high literacy on user
information search and retrieval. in Proceedings of the 22nd British HCI
Conference on People and Computers: Culture, Creativity,
Interaction-Volume 1. 2008.
British Computer Society.
2. Kodagoda, N., et al. Interactive visualization for low literacy users:
from lessons learnt to
design. in Proceedings of the SIGCHI Conference on Human Factors in
Systems. 2012. ACM.
3. Stelmaszewska, H., A. Blandford, and G. Buchanan, Designing to change
information seeking behaviour: a case study. Adaptable and Adaptive
Systems. London: Information Science Publishing, 2005: p. 1-18.
4. Stelmaszewska, H., et al. Electronic resource discovery systems: from
user behaviour to
design. in Proceedings of the 6th Nordic Conference on Human-Computer
Extending Boundaries. 2010. ACM.
5. Wong, B.W., et al. 3D-in-2D Displays for ATC. in 6th EUROCONTROL
Research Workshop. 2007.
6. Wong, B.W. and M. Varga. Black Holes, Keyholes And Brown Worms:
Challenges In Sense
Making. in Proceedings of the Human Factors and Ergonomics Society Annual
2012. SAGE Publications.
Related funded research projects associated with research all peer
reviewed and competitively
won and most supported by expert advisory committees are:
1. JISC Rapid Innovation: INVISQUE, Interactive Visual Query Environment;
2009. PI: Prof. William Wong.
2. EPSRC: Making Sense: 10 UK partners; 245000 GBP; 2010-2013; Co-PI:
3. UKVAC (UK Visual Analytics Consortium) Phase 1, NVAC; 2010-2011;
$250,000; PI: Prof.
4. TSAM: Team Sense Making Assessment Method. Partners with BAe Systems,
Associates, MoD CDE. 50000 GBP; 2011.
5. UKVAC (UK Visual Analytics Consortium) Phase 2, US Dept of Homeland
Home Office; 2012-2013; $500,000 + 400,000 GBP ; PI: Prof. William Wong.
6. VALCRI: Visual Analytics in Criminal Intelligence Analysis; EU FP7.
PI: Prof. William Wong.
Details of the impact
Visual analytics (VA) continues to mature as a subject area within
Interaction Design and
Middlesex has increasingly provided international leadership in this area
both to researchers and
the wider research user community. Impacts arising from our research in VA
of software at government research labs in the security sector; supporting
low literacy users such
as those using services located in Citizens Advice Bureaux (CABs) and
leadership at both a
national and international level in VA.
A key aspect of applied aspects of the research detailed earlier is the
need to support `theory
building' (such as that found in intelligence analysis) using visual
mechanisms for reasoning and
narrative construction based on data that has been retrieved and organised
by the various
automated techniques e.g. automated knowledge extraction and thematic
clustering. Our ideas
have been implemented in the INVISQUE technology leading to impact in
Firstly, INVISQUE enables visual support for human reasoning actions to
support theory building
and this has applications in the intelligence domain. Secondly, as this
has developed, the research
has contributed to and established a community both nationally and
internationally in the area of
visual analytics. Thirdly, recognising that the software technology
originates from research
exploring issues facing low literacy users, the software has also
applicability in CABs.
The research is being used by the UK MoD to help them define
theory-building tools for their
intelligence community. In our work, we refer to the concept of a
Reasoning Workspace and how
such a Reasoning Workspace is formed in design. The constituent elements
are underpinned by
the research reported from 2005 onwards that identified insights into how
expert users bring to
bear techniques to support their information seeking behaviours. The
research also identified the
need for novel visual representations and the need to support backtracking
within such visual
representations. Some of these findings were most clearly noted when first
working with low
literacy users but now have their use in a security setting.
While the software arising from the research, INVISQUE, can be provided
on request, it has been
made available to both the Pacific North Western National Laboratory
(PNNL) and the UK Ministry
of Defence (DSTL). At PNNL, a variant of INVISQUE is installed and used as
tool to other technologies developed by PNNL [S3]. DSTL has used the
software for experiments
with security related information. The usage of the technology is reported
in an internal manuscript
and confirmation of the activity noted by DSTL [S4]. Thus the primary
beneficiaries of this research
are the security organisations of the UK and US. The impact of the
research is largely the design
and principles underpinning the technology.
The novelty of the INVISQUE environment and the supporting research led
to the first UK
workshop/conference on the recently established field of visual analytics.
This workshop in 2010,
at Middlesex, was attended by delegates from both the UK Government and US
Dept of Homeland
Security and was a direct consequence of the research presented from Wong
and others. The
workshop provided the basis for the formulation of the UK Visual Analytics
Consortium led by Prof.
Wong with funding from Department of Homeland Security and the UK Home
UKVAC is funded by to carry out research in technologies based around
INVISQUE and to provide
advanced training on related technologies for visual analytics. Other
members of this consortium
include Oxford, Imperial, UCL, Bangor and Swansea.
The UKVAC has run three International Summer Schools and an accompanying
2012 and 2013) and is the premier UK located event in this area attracting
delegates attending the
summer school from the intelligence, finance and academic communities
The INVISQUE technology has also been applied in Citizens Advice Bureau
settings. Further we
planned to develop a deployable version for the National CAN to begin in
Feb 2012. However due
to budget cuts at the CAB, the project was postponed until further funding
could be found. A
number of commercial organisations have expressed various levels of
interest in this technology,
including Microsoft, IBM i2, BAE Systems, Detica, Taylor and Francis
(which was prepared to
adopt the INVISQUE user interface concept - but had to cancel the project
as a result of funding
cuts) [S1], JISC and the British Library.
Professor Wong has played a pivotal role in ensuring that INVISQUE has
also had impact on the
wider international community through the offices of the NATO IST110
committee and is
presenting a keynote at the NATO Visual Analytics Symposium. This impact
is evidenced by the
supporting letter from Prof Margaret Varga, Chairman of NATO IST100/RTG055
INVISQUE and its underlying concepts have been the basis for a successful
grant proposal to the
European Commission. The project VALCRI (Visual Analytics in Criminal
Intelligence Analysis) has
been costed at €16.8 mil (EC grant €13.1 mil) and involves 18 industrial,
research and police end-
user partners across the UK, Europe and the US [S5].
Sources to corroborate the impact
The nature of the primary beneficiaries means that it is difficult to get
confirmation from security related
organisations on the use of technology.
S1. Taylor and Francis: Emails to confirm the interest of publishers in
the Invisque platform.
S2. Pacific Northwest National Laboratory (for the US Dept of Homeland
Security). Statement of
S3. Pacific Northwest National Laboratory: Letter confirming use of
Invisque at PNNL.
S4. DSTL: Letter describing role of Invisque platform at
S5. VALCRI funding letter.
S6. Supporting letter from NATO IST110 / RTG055