Increasing society’s capacity to tackle complex, socio-technical dilemmas
Submitting InstitutionOpen University
Unit of AssessmentComputer Science and Informatics
Summary Impact TypeTechnological
Research Subject Area(s)
Information and Computing Sciences: Artificial Intelligence and Image Processing, Computer Software, Information Systems
Summary of the impact
Compendium software is used to map dialogue and information around
socio-technical dilemmas with economic, public policy, educational and
health impacts. In Australia, urban planners attribute stakeholder buy-in
to dialogue mapping with Compendium. In the USA, a deadlocked
environmental planning process used it to make progress, while Southern
California Edison use it to manage environmental policy. In the NHS, it
can map therapeutic group dynamics, while in Germany, a journalist
summarised a medical ethics case to parliament with it. More than 170
companies and individuals have endorsed Compendium, a striking application
being to control Attention Deficit Hyperactivity Disorder (ADHD) at work.
Our research into sensemaking focuses on computational support for human
interpretation and action, when confronted by overwhelming complexity.
Professor Simon Buckingham Shum (1995-present) has led the investigation
into the human and technical factors that promote (or impede) the adoption
of software tools to assist in this process. Specifically, we have focused
on the design and use of visual software that combines formal modelling,
information management, reflective dialogue, rigorous argumentation and
visual `knowledge cartography'.
Our research into hypermedia discourse, design rationale and argument
visualisation underpins a software tool called Compendium, and
associated methodologies for its effective use. Embodying insights from 20
years' research, Compendium's data model is designed to capture multiple
`minds' when deliberating a complex dilemma (in contrast to mindmapping,
where typically only one mind is expressed).
Our research has conducted empirical studies of both novice and expert
Compendium users. Hypertext functionality helps to manage myriad
connections between ideas and information elements. It is this attention
to holding in one place multiple perspectives across many conversations,
about complex, multimedia information, which underpins Compendium's
distinctiveness and impact.
Our informatics perspective emphasises that there are cognitive, social
and political ramifications to capturing and visualising conversational
contributions. Critically, we have developed an account of the skillset
required to augment meetings with shared visual representations, a
practice that has been the object of close analysis.
Methodologically, this has been action research, in which the software is
deployed by ourselves and many others in authentic contexts, enabling us
to refine both the underlying assumptions and the software. The evidence
base has not only been documented academically, but also informs a set of
validated practices for effective use of the tool, delivered through
training by ourselves and two consultancies.
The research input has been critical. Reflective practice, combined with
video analyses of users, informed understanding of the initial learning
curve [3.1], but also the nature of fluency and expertise with the tool
[3.2], findings that are possible only with a longitudinal research
programme studying a robust tool in authentic usage.
Visualizing Argumentation [3.3] includes several chapters on
Compendium, establishing the field, and becoming the standard reference.
The research took a further step when Compendium's argument maps were
integrated with a videoconferencing platform, enabling us to investigate
the affordances of semantically indexed video replays of meetings,
generated automatically from the meeting metadata [3.4].
This longitudinal research programme enabled us to reflect on lessons
learned over 15 years, in a chapter for the primary source on Software
Engineering Design Rationale [3.5], with the editorial concluding that
Compendium is the most mature tool available in terms of real-world
Simon Buckingham Shum (1995-present), Anna De Liddo (PhD 2008; RA
2008-present), Clara Mancini (RA 2002-2006), Albert Selvin (PhD 2003-2011)
References to the research
[3.1] Buckingham Shum, S., MacLean, A., Bellotti, V. and Hammond, N.
(1997) `Graphical argumentation and design cognition' Human-Computer
Interaction, vol. 12, no. 3, pp. 267-300. http://doi.org/10.1207/s15327051hci1203_2.
[3.2] Selvin, A.M., Buckingham Shum, S. and Aakhus, M. (2012) `The
practice level in participatory design rationale: studying practitioner
moves and choices' in Carroll, J.M. (ed.) Creativity and Rationale:
Enhancing Human Experience by Design, London, Springer. ISBN
978-1-4471-4111-2. Reprinted from Human Technology, 2010, vol.
6, no. 1, pp. 71-105. Eprint: http://oro.open.ac.uk/20948.
[3.3] Kirschner, P., Buckingham Shum, S. and Carr, C. (eds) (2003) Visualizing
Argumentation: Software Tools for Collaborative and Educational
Sense-making, London, Springer-Verlag. www.VisualizingArgumentation.info.
[3.4] Buckingham Shum, S., Slack, R., Daw, M., Juby, B., Rowley,
A., Bachler, M., Mancini, C., Michaelides, D., Procter, R., De Roure, D.,
Chown, T., and Hewitt, T. (2006). Memetic: An Infrastructure for Meeting
Memory. Proceedings of COOP 2006: Conference on Cooperative Systems
Design. IOS Press: Frontiers in Artificial Intelligence and Applications,
Vol.137, Editors: Parina Hassanaly, Thomas Herrmann, Gabriele Kunau,
Manuel Zacklad. pp. 71-85. Eprint: http://oro.open.ac.uk/23373
[3.5] Buckingham Shum, S., Selvin, A., Sierhuis, M., Conklin, J., Haley,
C. and Nuseibeh, B. (2006) `Hypermedia support for
argumentation-based rationale: fifteen years on from gIBIS and QOC',
in Dutoit, A., McCall, R., Mistrik, I. and Paech, B. (eds) Rationale
Management in Software Engineering, Berlin, Springer-Verlag,
pp.111-32. Eprint: http://oro.open.ac.uk/3032.
Open Learning Network Project (2009-2012): Funded by the Hewlett
Foundation, £2M e-Dance Project (2008-2009): Funded by joint
AHRC/EPSRC/JISC Arts & Humanities e-Science Programme, £337,017
ECOSENSUS Project (2005-2007): Funded by ESRC e-Social Science Programme,
MEMETIC Project (2005-2006): Funded by JISC, £200,656
Co-OPR Project (2004): Funded by DARPA £230,000
CoAKTinG Project (2002-2004): Funded by EPSRC, £517,139
ScholOnto Project (2001-2004): Funded by EPSRC, £315,929
Details of the impact
Through design-based research, working with clients confronting complex
challenges, Compendium has evolved from a proof-of-concept e-Science
demonstrator in 2002 to usage across education, business and civic
Software adoption: Compendium has attracted more than 100,000
unique downloads since 2003 and has an active user community (1867
members, July 2013) supported by the Compendium Institute (CompendiumInstitute.net).
The open source developer community took ownership of the software in
early 2013, and maintains the software on the CompendiumNG website (compendiumng.org):`Compendium
is considered so valuable by its user community that it has self-organised
to ensure that it remains a living software application.' [5.1].
Our user survey [5.2] (launched January 2011) has elicited more than 170
endorsements of its importance in the personal and professional lives of
many people, with the primary impact being more efficient information
management and deliberation processes. The testimonials database shows the
sector breakdown: Personal: 32%, Education: 27%, Business: 17%, Not for
Profit: 14%, Other: 8%, and Government: 1%.
Economic impact (SMEs): Compendium sits at the heart of several
consulting firms, for example:
`CogNexus Group has used Compendium software exclusively to support Issue
Mapping and Dialogue Mapping services and training since 2008. During this
time we have trained over 80 people in the art of Issue Mapping using
Compendium, in areas as diverse as public health, energy production, water
usage issues, education, and consulting. We see Compendium as the premier
tool for supporting the large maps typical of the real-world use of Issue
Mapping and Dialogue Mapping. Because Compendium supports capabilities
that no other software does, our work would not be possible without it!'
Of the many public testimonials on the Compendium Institute website, one
`Compendium means more than a tool for our project; it is a full work
philosophy when choosing options in our complex decision trees while
designing projects, developing research discussions, following academic
debates, planning activities and evaluating alternatives.' (BambHaus,
January 2011 [5.2])
Public policy impact: Compendium was used during 2010-11 for
participatory urban planning in Perth, Australia by Seven Sigma
`The Stirling Alliance utilised dialogue mapping to help resolve the long
standing Stephenson Reserve issue that could not be resolved using
traditional methods. Dialogue mapping significantly shortened the time
frames to discuss multiple options.' [5.4].
Southern California Edison uses Compendium to capture rationale and index
documents in environmental policy management, documenting the 1993-2003
period in [5.5], and a decade later they continue to use it. They hired
their own developer to add data scalability and local area networking to
the codebase, returning these improvements to the open source release.
Since 2012-present, Compendium supports dialogue mapping which led to
breakthroughs in planning the Sacramento-San Joaquin Delta, as evidenced
in the meeting maps and report from the Delta Dialogues [5.6].
Health impact: Group psychotherapists struggle to find ways to
evidence the impact of their practice in the increasingly quantitative
terms required by the NHS. Compendium was piloted in 2010 successfully by
psychotherapists, who convert their usual written notes from analytic
group sessions into interactive maps that reflect the group dynamics and
enable the data to be interrogated. This work attracted interest at the
professional conference of the Society for Psychotherapy Research,
and was subsequently published in a leading practitioner journal, Group
The user survey shows that Compendium also serves to support many
individuals' personal lives. For instance, vital cognitive planning
support for one user (unknown to us) with ADHD, who said: `IBIS has
changed my life. [...] I have ADHD and mapping out problems with IBIS
helps me to slow down, orientate and stay focused. [...] I use Compendium
in meetings and in GTD (Getting Things Done) weekly reviews' [5.2].
Education and public understanding of science: Compendium has been
adopted by several university courses as an official tool, including
Imperial College London, Department of Mechanical Engineering, who
required its use in student design projects in 2010-11 [5.8].
Compendium was selected for the British Library's Growing Knowledge foyer
exhibition, communicating to the public how digital tools are transforming
scholarly research (October 2010-July 2011) [5.9].
Compendium enabled a journalist to synthesise material in an efficient,
accessible way as part of engaging the public, scientists and politicians
in a topical debate on synthetic biology [5.10]:
`The online maps did a great job in organizing and guiding the
conversations with our experts. Our goal (which was accomplished) was to
create a map that all contributors agreed upon (in terms of fair and
thorough representation of arguments). The results were presented at a
Parliamentary evening event in Berlin in November 2010.'
Sources to corroborate the impact
[5.1] CompendiumNG: Dev community takes the codebase forward. Compendium
Institute news story: http://compendiuminstitute.net/news/rostra/news.php@r=55&t=2&id=54.htm
Contact: CompendiumNG lead developer
[5.2] User community testimonials (January 2011-present):
Contacts: CogNexus Group consultant; ADHD sufferer; BambHaus
[5.3] Culmsee, P. and Awati, K. (2011) The Heretic's Guide to Best
Practices: The Reality of Managing Complex Problems in Organisations,
Bloomington, IN, iUniverse.
[5.4] Seven Sigma's use of Compendium for Dialogue Mapping:
[5.5] Conklin, J. (2003) `Dialogue Mapping: reflections on an industrial
strength case study' in Kirschner, P.A., Buckingham Shum, S. and Carr, C.
(eds) Visualizing Argumentation, London, Springer-Verlag. http://www.visualizingargumentation.info.
[5.6] Delta Dialogues, Groupaya (2012) Meeting Maps: http://delta.groupaya.net/meetings
[5.7] Compendium for mapping group dynamics. Compendium Institute news
[5.8] Compendium in Civil Engineering. Compendium Institute news story:
[5.9] Arts and Humanities e-Science Support Centre, e-Dance
[5.10] Argument Visualization in Online Science Debates. Compendium
Institute news story:
Contact: Science Journalist