Summary of the impact

The Mixed Reality Laboratory (MRL) has collaborated with the artists Blast Theory to transform UK theatre and drive innovation in games and television. The underlying research was published as a series of papers in ACM Transactions on CHI and at the ACM's CHI conference between 2003 and 2012, with two CHI papers winning best paper awards. By creating and touring a series of innovative and technically advanced performances, we have enriched cultural life and influenced a rising generation of UK artists. By then jointly collaborating with companies such as Microsoft, Sony, Nokia and the BBC, we have innovated new games and television formats. Evidence of this impact is to be found in: performances touring to 40 venues in 18 countries to be experienced by over 200,000 people; 160 reviews in international press; artistic and industry awards; direct involvement of industry partners in follow-on commercial projects; and consultancy.

Underpinning research

During this REF period, the Mixed Reality Lab has collaborated with the cutting-edge UK artists Blast Theory to explore how interactive technologies can transform the experience of theatre, games and television. This work has addressed how mixed reality technologies can underpin new forms of interactive experience in which audience members become active protagonists and performance and gameplay spill out onto the city streets.

This collaboration involved a tight iterative loop between the MRL's underlying research and Blast Theory's artistic practice: a technological innovation feeds into multiple performances, while naturalistic studies of these performances then shape further research. Research papers often reflect on past projects while establishing new concepts and techniques for future ones. The following are especially significant academic papers that have shaped Blast Theory's practice while also having a wider impact on the games and television sectors as we describe later.

• We were the first to establish the approach of designing ambiguous interactions in a CHI 2003 paper [1]. This laid the foundations for a series of Blast Theory productions that deliberately made use of ambiguous information, instructions or framing (separating what is fictional from what is real) to create powerful public experiences.
• We introduced the idea of Pervasive Games to a readership of 100,000 computing professionals in our 2005 Communications of the ACM article [2]. This drew on and inspired a series of Blast Theory productions and also underpinned the development of commercial prototypes with Sony and Nokia in the EU Integrated Project on Pervasive Games.
• From 2001 onwards our research established the mixed reality techniques to connect physical environments to their online counterparts. Our 2006 ACM Transactions on CHI paper [3] described how these techniques were used to create Blast Theory's production Can You See me Now?, a game in which performers equipped with GPS on the streets of a city were chased by online players in a virtual model of the same city. A study of several performances informed new approaches to designing around the inaccuracy and lack of coverage of GPS that fundamentally reshaped their approach to later works such as A Machine to See With, I'd Hide You, You Get Me, and The Goody Bullet.
• From 2007 we developed the `trajectories' conceptual framework to guide the design of mobile cultural and entertainment experiences. Our award-winning CHI 2009 paper [4] that first introduced trajectories drew on earlier Blast Theory works, while the concepts were used to design later productions, and were also taken up by the BBC as we describe below.
• Recent research has employed Blast Theory's production Ulrike and Eamon Compliant to illustrate the deliberate use of discomfort to create entertaining, enlightening and socially bonding cultural experiences. This has been reported in an award-winning paper [5] at CHI 2012 and in the cover article of Communications of the ACM in September 2013.

The key researchers involved in this work were: Benford (Prof), Greenhalgh (Prof), Rodden (Prof), Crabtree (transitioned from Research Fellow to Associate Professor and Reader over this period), Koleva (Research Associate to Associate Professor), Flintham (PhD student to Lecturer), Reeves (PhD student to EPSRC Early Career Research Fellow) and Marshall (PhD student to Leverhulme Fellow). They are all currently still working at the MRL.

References to the research

Citations as reported by Google Scholar on 21st September 2013

Papers marked * were awarded best papers at the CHI conference (top 1% of all submissions)

This research was funded through a series of EPSRC, RCUK, TSB and EU awards:

• The Equator IRC (EPSRC, 2000-2007, £10.5M total award, GR/N15986/01)

• The eRENA project (EU, 1997-2000, £1.1M)

• The Integrated Project on Pervasive Games (EU, 2004-2008, £3.5M)

• The Participate project (TSB/EPSRC, 2005-2008, £2M, EP/D033780/1)

• The Horizon Digital Economy Research Centre (RCUK, 2009-2014, £11.5M, EP/G065802/1)

There was also additional support through two AHRC grants, support for touring from the Arts Council of England and British Council, and direct industry funding from Microsoft.

Details of the impact

This body of research has impacted on three sectors of the cultural and creative industries:

1. Theatre — by establishing concepts and technologies to mix fictional virtual worlds with physical settings and by collaborating with artists to create a new theatrical form called Mixed Reality Performance.
2. Games — by working with major industry players to explore how these same concepts and technologies can enable the emergence of a new form of mainstream entertainment called Pervasive Games.
3. Television — to engage TV producers and broadcasters to further expand this impact into the world of television, enabling new modes of audience participation.

Impact 1 — Transforming theatre through Mixed Reality Performance

The performing arts are an essential aspect of our cultural and economic life, contributing to our emotional wellbeing, providing a hothouse for nurturing creative talent, and driving innovation in more mainstream entertainment such as games and television. The UK is internationally renowned for the quality and innovation of its theatre, and exports many productions worldwide.

During this REF period we have worked with Blast Theory to develop, refine and tour ten theatrical productions: Can You See Me Now? (based on the research in [1,2,3,6]), Uncle Roy All Around You [1,2,4,6], Day of the Figurines [4,6], Rider Spoke [4,6], Ulrike and Eamon Compliant [4,5,6], Flypad [4,6], A Machine to See With, I'd Hide You [6], You Get Me [6], and The Goody Bullet [6]. This extensive body of work has led to the recognition of Mixed Reality Performance as an emerging theatrical form (comprehensively described in [6]). In theatre, the best evidence of impact is to be found in successful touring, critical acclaim, awards and influencing other artists:

Touring. According to Blast Theory's performance records (corroborating evidence [A]) these works have toured to 38 venues in 16 countries and been experienced by over 150,000 participants during this REF period. Commissioners include prestigious international venues (Royal Festival Hall, Royal Opera House), festivals (Edinburgh, Venice Biennale, Sundance Film Festival, Ars Electronica, Linz Capital of Culture, TRUST: Media City Seoul), and galleries (The Barbican, Sydney Museum of Contemporary Art, Tate Britain, Victoria and Albert Museum, British Library). We delivered a permanent installation at The Public, a new multi-million pound arts centre in West Bromwich, while documentation of performances has been displayed at the British Library, Stanford Library and the Baltimore Contemporary Museum.

Critical acclaim. Blast Theory's press archive [B] documents 160 articles about these works from across the globe, reflecting the cultural significance of this work. In April 2009, The Guardian included our joint work Desert Rain in its "Ten Experimental Theatre Productions that Transformed Theatre" [C], and Matt Adams from Blast Theory was invited onto BBC Radio 4's Start the Week in May 2013 to discuss how mixed reality performance is transforming British theatre.

Awards. Ulrike and Eamon Compliant [5,6] was nominated for the 2011 annual Doc/Fest award; You Get Me was Nominated in the NetArt category for the 14^th Annual Webby Awards; and Rider Spoke was nominated for a Total Theatre Award at the Edinburgh Festival Fringe, in 2009 [B].

Influencing other artists. This body of work has inspired a new generation of artists to work in Mixed Reality Performance. With support from The Arts Council and NESTA, the Mixed Reality Laboratory has directly collaborated with a further seven artists during this REF period to create over ten additional touring performances. Our collaborations with Plan B involving the public capture and visualisation of GPS trails toured to Berlin, Brussels, Leipzig, London, Sao Paulo, Hamburg, Birmingham, Athens (USA), Kiel, Leuven, and Sunderland, being experienced by over 25,000 people [D]. Active Ingredient produced a series of performances that drew on personal biodata and environmental data and that toured the UK and to Paris, Barcelona and Rio, being experienced by over 11,000 people [E]. We worked the performance group Urban Angel to create an `alternate reality game' that reached over 3000 direct participants [F]. Artworks created in collaboration with emerging artist Theresa Caruana reached over 800 members of the public and attracted support of over £10,000 from arts funders [G]. Finally, we have worked with Francesca Beard on the development of Storyverse, an innovative performance project that has recently toured to Uganda and Sudan with support from the British Council [H].

Impact 2 — Transforming the games industry through Pervasive Games

An innovative aspect of these performances has involved adopting the form of computer games that bridge between online and mobile play. As a result, the games industry has viewed them as prototypes of future game formats and has collaborated with us, or directly funded us, to drive their wider impact on the games industry. Indeed, we were the first to popularise the term Pervasive Games in an article in Communications of the ACM in 2005 [2]. Evidence for this impact on the games industry is to be found in collaborations with major games companies and industry awards:

Funded collaborations. The EU Integrated Project on Pervasive Games [I] that ran through 2008 involved us working jointly with Sony to develop Rider Spoke [6] and with Nokia to develop Day of the Figurines [4,6]. In 2010, we were awarded a £25,000 consultancy contract by Microsoft to help them develop a commercial mobile training game. We have since won funding for a new €10M European Integrated Project that will run from 2013 to 2017 to work with technology and games companies to establish a new technology platform for pervasive realty games.

Industry awards. Ulrike and Eamon Compliant won the `Best Real World Game' award at the prestigious 6th International Mobile Gaming Awards in 2010 [J]; Rider Spoke was nominated for the 2008 European Innovative Games Award; and Blast Theory and Nottingham were awarded the Digital Collaboration Award at the Inaugural DiMAS (Digital Media Awards South) in 2008 [B].

Impact 3 — Transforming television through new forms of participation

Our research has also impacted on the television industry by inspiring new approaches to viewer participation. Evidence can be found in funded collaborations with TV companies:

• Nottingham and Blast Theory were partners in the TSB-funded Participate project that ran through 2008 in which we worked with the BBC and British Telecom to develop platforms for mass-participation television campaigns [J].
• We received an award from TSB to work with TV production company Something Else to develop formats that engage online viewers with video streamed from the city streets. The result was broadcast on the BBC's online platform `The Space'. YouTube have since invested $40,000 into extending this approach to enable spectators to crowdsource coverage of public events such as marathon races.

Benford was appointed as the BBC's first Visiting Professor in 2013 [K]. The BBC User Experience Design group subsequently adopted our work on trajectories [4,6] as a core concept in their Northstar project that is focusing over 100 professional user experience designers on future `One Service' TV experiences. This has led to a £10K consultancy contract to develop 5 learning trajectories as part of the BBC's new Knowledge and Learning mobile app.

Sources to corroborate the impact

[A] Letter with corroborating evidence from Blast Theory, 2nd April 2013.

[B] Blast Theory website: http://www.blasttheory.co.uk.

[C] Guardian article on ten experimental works that transformed British theatre, 1st April 2009.
http://www.guardian.co.uk/stage/2009/apr/01/experimental-theatre-spill-festival

[D] Letter with corroborating evidence from Plan B, 3rd September 2013.

[E] Letter with corroborating evidence from Active Ingredient.

[F] Letter with corroborating evidence from Urban Angel.

[G] Letter with corroborating evidence from Theresa Caruana, 9th September 2013.

[H] Letter with corroborating evidence from Francesca Beard.

[I] iPerG project website: http://iperg.sics.se/index.php.

[J] Participate project website: http://www.participateonline.info.

[K] BBC blogpost and video on Steve Benford's impact as a Visiting Professor, 8th April 2013.
http://www.bbc.co.uk/rd/blog/2013/04/steve-benford-being-a-visiting-professor-at-bbc-rd.