Summary of the impact

Compendium software is used to map dialogue and information around socio-technical dilemmas with economic, public policy, educational and health impacts. In Australia, urban planners attribute stakeholder buy-in to dialogue mapping with Compendium. In the USA, a deadlocked environmental planning process used it to make progress, while Southern California Edison use it to manage environmental policy. In the NHS, it can map therapeutic group dynamics, while in Germany, a journalist summarised a medical ethics case to parliament with it. More than 170 companies and individuals have endorsed Compendium, a striking application being to control Attention Deficit Hyperactivity Disorder (ADHD) at work.

Underpinning research

Our research into sensemaking focuses on computational support for human interpretation and action, when confronted by overwhelming complexity. Professor Simon Buckingham Shum (1995-present) has led the investigation into the human and technical factors that promote (or impede) the adoption of software tools to assist in this process. Specifically, we have focused on the design and use of visual software that combines formal modelling, information management, reflective dialogue, rigorous argumentation and visual `knowledge cartography'.

Our research into hypermedia discourse, design rationale and argument visualisation underpins a software tool called Compendium, and associated methodologies for its effective use. Embodying insights from 20 years' research, Compendium's data model is designed to capture multiple `minds' when deliberating a complex dilemma (in contrast to mindmapping, where typically only one mind is expressed).

Our research has conducted empirical studies of both novice and expert Compendium users. Hypertext functionality helps to manage myriad connections between ideas and information elements. It is this attention to holding in one place multiple perspectives across many conversations, about complex, multimedia information, which underpins Compendium's distinctiveness and impact.

Our informatics perspective emphasises that there are cognitive, social and political ramifications to capturing and visualising conversational contributions. Critically, we have developed an account of the skillset required to augment meetings with shared visual representations, a practice that has been the object of close analysis.

Methodologically, this has been action research, in which the software is deployed by ourselves and many others in authentic contexts, enabling us to refine both the underlying assumptions and the software. The evidence base has not only been documented academically, but also informs a set of validated practices for effective use of the tool, delivered through training by ourselves and two consultancies.

The research input has been critical. Reflective practice, combined with video analyses of users, informed understanding of the initial learning curve [3.1], but also the nature of fluency and expertise with the tool [3.2], findings that are possible only with a longitudinal research programme studying a robust tool in authentic usage.

Visualizing Argumentation [3.3] includes several chapters on Compendium, establishing the field, and becoming the standard reference.

The research took a further step when Compendium's argument maps were integrated with a videoconferencing platform, enabling us to investigate the affordances of semantically indexed video replays of meetings, generated automatically from the meeting metadata [3.4].

This longitudinal research programme enabled us to reflect on lessons learned over 15 years, in a chapter for the primary source on Software Engineering Design Rationale [3.5], with the editorial concluding that Compendium is the most mature tool available in terms of real-world deployment.

Key researchers

Simon Buckingham Shum (1995-present), Anna De Liddo (PhD 2008; RA 2008-present), Clara Mancini (RA 2002-2006), Albert Selvin (PhD 2003-2011)

References to the research

Grants

Open Learning Network Project (2009-2012): Funded by the Hewlett Foundation, £2M e-Dance Project (2008-2009): Funded by joint AHRC/EPSRC/JISC Arts & Humanities e-Science Programme, £337,017

ECOSENSUS Project (2005-2007): Funded by ESRC e-Social Science Programme, £45,663

MEMETIC Project (2005-2006): Funded by JISC, £200,656

Co-OPR Project (2004): Funded by DARPA £230,000

CoAKTinG Project (2002-2004): Funded by EPSRC, £517,139

ScholOnto Project (2001-2004): Funded by EPSRC, £315,929

Details of the impact

Through design-based research, working with clients confronting complex challenges, Compendium has evolved from a proof-of-concept e-Science demonstrator in 2002 to usage across education, business and civic society.

Software adoption: Compendium has attracted more than 100,000 unique downloads since 2003 and has an active user community (1867 members, July 2013) supported by the Compendium Institute (CompendiumInstitute.net). The open source developer community took ownership of the software in early 2013, and maintains the software on the CompendiumNG website (compendiumng.org):`Compendium is considered so valuable by its user community that it has self-organised to ensure that it remains a living software application.' [5.1].

Our user survey [5.2] (launched January 2011) has elicited more than 170 endorsements of its importance in the personal and professional lives of many people, with the primary impact being more efficient information management and deliberation processes. The testimonials database shows the sector breakdown: Personal: 32%, Education: 27%, Business: 17%, Not for Profit: 14%, Other: 8%, and Government: 1%.

Economic impact (SMEs): Compendium sits at the heart of several consulting firms, for example:

`CogNexus Group has used Compendium software exclusively to support Issue Mapping and Dialogue Mapping services and training since 2008. During this time we have trained over 80 people in the art of Issue Mapping using Compendium, in areas as diverse as public health, energy production, water usage issues, education, and consulting. We see Compendium as the premier tool for supporting the large maps typical of the real-world use of Issue Mapping and Dialogue Mapping. Because Compendium supports capabilities that no other software does, our work would not be possible without it!' [5.2].

Of the many public testimonials on the Compendium Institute website, one consultancy states:

`Compendium means more than a tool for our project; it is a full work philosophy when choosing options in our complex decision trees while designing projects, developing research discussions, following academic debates, planning activities and evaluating alternatives.' (BambHaus, January 2011 [5.2])

Public policy impact: Compendium was used during 2010-11 for participatory urban planning in Perth, Australia by Seven Sigma consultants [5.3]:

`The Stirling Alliance utilised dialogue mapping to help resolve the long standing Stephenson Reserve issue that could not be resolved using traditional methods. Dialogue mapping significantly shortened the time frames to discuss multiple options.' [5.4].

Southern California Edison uses Compendium to capture rationale and index documents in environmental policy management, documenting the 1993-2003 period in [5.5], and a decade later they continue to use it. They hired their own developer to add data scalability and local area networking to the codebase, returning these improvements to the open source release.

Since 2012-present, Compendium supports dialogue mapping which led to breakthroughs in planning the Sacramento-San Joaquin Delta, as evidenced in the meeting maps and report from the Delta Dialogues [5.6].

Health impact: Group psychotherapists struggle to find ways to evidence the impact of their practice in the increasingly quantitative terms required by the NHS. Compendium was piloted in 2010 successfully by psychotherapists, who convert their usual written notes from analytic group sessions into interactive maps that reflect the group dynamics and enable the data to be interrogated. This work attracted interest at the professional conference of the Society for Psychotherapy Research, and was subsequently published in a leading practitioner journal, Group Analysis [5.7].

The user survey shows that Compendium also serves to support many individuals' personal lives. For instance, vital cognitive planning support for one user (unknown to us) with ADHD, who said: `IBIS has changed my life. [...] I have ADHD and mapping out problems with IBIS helps me to slow down, orientate and stay focused. [...] I use Compendium in meetings and in GTD (Getting Things Done) weekly reviews' [5.2].

Education and public understanding of science: Compendium has been adopted by several university courses as an official tool, including Imperial College London, Department of Mechanical Engineering, who required its use in student design projects in 2010-11 [5.8].

Compendium was selected for the British Library's Growing Knowledge foyer exhibition, communicating to the public how digital tools are transforming scholarly research (October 2010-July 2011) [5.9].

Compendium enabled a journalist to synthesise material in an efficient, accessible way as part of engaging the public, scientists and politicians in a topical debate on synthetic biology [5.10]:

`The online maps did a great job in organizing and guiding the conversations with our experts. Our goal (which was accomplished) was to create a map that all contributors agreed upon (in terms of fair and thorough representation of arguments). The results were presented at a Parliamentary evening event in Berlin in November 2010.'

Sources to corroborate the impact

[5.1] CompendiumNG: Dev community takes the codebase forward. Compendium Institute news story: http://compendiuminstitute.net/news/rostra/news.php@r=55&t=2&id=54.htm
Contact: CompendiumNG lead developer

[5.2] User community testimonials (January 2011-present):
http://compendiuminstitute.net/community/community.php
Contacts: CogNexus Group consultant; ADHD sufferer; BambHaus consultant

[5.3] Culmsee, P. and Awati, K. (2011) The Heretic's Guide to Best Practices: The Reality of Managing Complex Problems in Organisations, Bloomington, IN, iUniverse.
http://hereticsguidebooks.com/sample-chapters/

[5.4] Seven Sigma's use of Compendium for Dialogue Mapping:
http://www.sevensigma.com.au/what-we-do/sensemaking.html

[5.5] Conklin, J. (2003) `Dialogue Mapping: reflections on an industrial strength case study' in Kirschner, P.A., Buckingham Shum, S. and Carr, C. (eds) Visualizing Argumentation, London, Springer-Verlag. http://www.visualizingargumentation.info.

[5.6] Delta Dialogues, Groupaya (2012) Meeting Maps: http://delta.groupaya.net/meetings

[5.7] Compendium for mapping group dynamics. Compendium Institute news story:
http://compendiuminstitute.net/news/rostra/news.php@r=55&t=2&id=55.htm

[5.8] Compendium in Civil Engineering. Compendium Institute news story:
http://compendiuminstitute.net/news/rostra/news.php@r=55&t=2&id=53.htm

[5.9] Arts and Humanities e-Science Support Centre, e-Dance project:
http://www.ahessc.ac.uk/e-dance

[5.10] Argument Visualization in Online Science Debates. Compendium Institute news story:
http://compendiuminstitute.net/news/rostra/news.php@r=55&t=2&id=47.htm Contact: Science Journalist