Summary of the impact

Alliance researchers have devised and applied technologies that bridge the gap between the real and virtual worlds, linking digital data to physical entities. The ability to embed personal stories in objects and places has impacted on the way National Museums Scotland sources and displays collections, while Oxfam has used the research to bring added value to donated goods, leading to an increase in store sales of 53% over a week-long period. Mobile Visual Search technology has been taken-up by global brands and advertising agencies, including Nike, Disney, Vodafone, Nokia, Tesco, P&G, King & Partners, Mocom and Ogilvy, leading one industry expert to describe it as "the new model of marketing mobility". The work has led to a patent, the receipt of several awards, and influenced the formation of a spin-off company, Mobile Acuity (with revenue of over £0.5M to date), which has secured a major investment of over £1M, including from international corporation, [text removed for publication], to invest in the US and East Asia.

Underpinning research

The Internet of Things is anticipated as society moves to a ubiquitous form of computing in which every device is `on' and connected to the internet. The `things' are the objects we use and encounter every day, many of which can now be tagged and indexed during production so that consumers can read them using mobile devices. The process was originally seen as merely a logistical framework, but researchers at the University of Edinburgh (UoE) identified the potential for more social, cultural and commercial applications, including in the built environment. The research enables mobile phones to recognise or contextualise objects through the use of Quick Response (QR) codes or Mobile Visual Search (MVS) functionality.

Alliance researchers are credited with creating the world's first MVS system, based on image recognition software, which allows users to be proactive in associating photographs of real (offline) things with digital (online) media. Its earliest form was Spellbinder: a way of embedding digital content in an object or place simply by taking a photograph of it using an ordinary camera phone. Spellbinder was developed between 2004 and 2007 by a team led by Dr Mark Wright (1991-2013) and Dr Anthony Ashbrook (1996-1999; 2004-2005), with trials conducted in collaboration with ODEON cinemas, Pepsi and The Assembly venues. The research resulted in a successful patent application (Publication no: EP1847112 B1) and the formation, by Wright and Ashbrook, of Mobile Acuity, a commercial company which offers two MVS solutions to the market.

Spellbinder was developed through action research in an academic environment to allow its creators the time to explore the `idea space' around the technology, i.e. to investigate what it might mean to a variety of users and communities. While working on the project, Wright was also a Research Fellow on Branded Meeting Places (2006-2008) funded by the Research Councils' Designing for the 21st Century programme. The study, led by Prof Richard Coyne (1995 -present), examined the potential of new technologies to support social interaction in city spaces. Working on both projects simultaneously allowed Wright to think about how Spellbinder could be utilised to encourage more participatory exploration of the urban environment and to redefine `place' as a dynamic experience mediated by the tools of mobile and social media.

In Branded Meeting Places, the researchers explored the use of mobile phone applications to `tag' places: planting invisible clues in physical or online environments that could prompt memories as to what took place there. Workshops were conducted over the period 2006-2008; several of which involved Prof Chris Speed (2008-present) as participant. The concept of digital tagging became a major theme for Speed as the Principal Investigator of TOTeM (Tales of Things and Electronic Memory). This project was developed during a five-day EPSRC Design in the Digital World `sandpit' in December 2008, supported by the Technology Strategy Board.

TOTeM allowed people to attach personal stories, via digital media, to existing artefacts, including buildings and open spaces. The project ran from August 2009 to December 2012 with Dr Andrew Hudson-Smith of University College London as Co-Investigator, responsible for developing the technology. Speed investigated the technology's creative use, working with stakeholders to explore its deployment in different heritage, social and industrial contexts. Other consortium members were Brunel University, The University of Dundee and The University of Salford.

A critical aspect of the Alliance's research was to gain a better understanding of the public's concerns around Radio-Frequency Identification (RFID), as well as the value of artefacts and how technology can unlock people's relationship with them. Supported by the National Co-ordinating Centre for Public Engagement, one of the RCUK's `beacon' projects, this was achieved primarily through live `memory capturing' workshops at events such as the Edinburgh International Science Festival, the Scottish Mental Health Arts and Film Festival and Hands on History, led by the BBC. The research found that people respond to physical objects more deeply when they can disconnect them from guarded, public environments and handle them personally. This led to the development of `ghost objects'; touchable substitutes for priceless artefacts.

References to the research

Publications

Funding
Scottish Enterprise provided £120,000 proof-of-concept funding for the advancement of Spellbinder from 2004 to December 2005. Mobile Acuity was founded by Wright and Ashbrook in 2006 as a spin-out from the University of Edinburgh, backed by a private investor and has recently secured a major investment, including from [text removed for publication], to expand operations into the US and East Asian markets.

Branded Meeting Places, with Coyne as PI and Wright as Co-I, was awarded a total of £328,297.88 by the Arts and Humanities Research Council (AHRC) and the Engineering and Physical Sciences Research Council (EPSRC) in 2006 under its joint `Designing for the 21st Century' programme.

TOTeM: Tales of Things and Electronic Memory, with Speed as Principal Investigator, was funded by the Engineering and Physical Sciences Research Council (EP/H007318/1) following the Design in the Digital World sandpit (December 2008). The funding totalled £1,190,567, of which £247,878 was awarded to Edinburgh College of Art.

Details of the impact

This body of research has brought the social, urban and digital environments together in a way that is meaningful to both industry and consumers. It has demonstrated that new technologies can be a mechanism for storytelling, in the traditional sense, and a force for positive social and behavioural change. It has allowed retailers, gallerists and museum curators to address the toxic mix of factors which, in recent years, have led to the noticeable decline of the retail and cultural core of many urban areas. By incentivising physical presence in a shop or museum with virtual rewards, it has facilitated the return of footfall back to the heart of towns and cities.

In an article entitled Digital World Now Coaxing Shoppers Back to Stores (June 2012), Women's Wear Daily featured Mobile Acuity's work with New York digital agency, King & Partners. The article carried a quote from King saying "It's time to run back to brick-and-mortar, make it smart and really give smart tools to sales associates to focus on the real physical store" (see 5.1 below). In September that year, King & Partners launched the mobile application `Agent 3.1' for fashion label Phillip Lim. Using Mobile Acuity's On-Device MVS, developed from Spellbinder, Phillip Lim customers could scan brochures, print advertisements and billboards to reveal additional content about the label and its offer, including its first graphic novel (promoting its Fall 2012 collection).

In an article in InsideFFM (October 2012), a spokesperson for Phillip Lim said "Agent 3.1 the app will certainly feature in future campaigns and initiatives - we are really excited about its applications. The possibilities are really changing the way we can look at overall communications and engagement. We have had over 1000 downloads of the app and the number of shares of content on social media networks really surpassed our expectations - it has really been a lesson in... keeping your followers excited about what you are providing them". (See 5.1).

The Agent 3.1 application was well received within the industry for the power and choice it gave to the consumer. One expert, a Mobile and Emerging Technologies Strategist for [text removed for publication], is quoted as saying it succeeded in "entirely flipping the script on what we have come to believe are proper app interfaces. [Using the Mobile Acuity MVS] the brand and its agency chose to put ownership and control of the user experience in the hands of the actual user... As simplistic as this sounds, fundamentally, it is actually pretty revolutionary in digital fashion; the new model of marketing mobility." (See 5.1).

Agent 3.1 used Mobile Acuity's On-Device MVS technology. The company also licenses its platform to clients, enabling them to add visual search capabilities to their own products via Cloud MVS, its `point and click' solution for large-scale campaigns. This solution has been taken up by leading digital media delivery company, 7digital, which, in 2011, integrated Mobile Acuity Cloud's MVS Application Programming Interface (API) with its own API to develop a Visual Music Discovery (VMD) service. With existing retail partner, HMV, it created the first licensed VMD application; the HMV `listening post'.

The `listening post' app is one of range of HMV initiatives designed to help the business adapt to the new retail environment. The app is a contemporary reinterpretation of the physical `listening post' that used to be characteristic of HMV high street stores before floor and shelf space was compromised by the need to expand the company's offer beyond music. The technology enables consumers to listen to music previews by simply taking a photo of a CD cover. 7Digital has praised the "speed and accuracy" of Mobile Acuity's product and said "This functionality is a great addition to our leading API, allowing developers and service providers to further integrate innovative music exploration and purchasing into their applications. Being able to immediately see, hear and even purchase the music that you are reading about in a flyer or see on a fly poster is a really engaging proposition for music lovers" (see 5.2).

The success of the MVS solutions developed from Spellbinder is reflected in the diversity of their applications. They have been used by Tesco in a Grocery app for the mass market and by high-end brands, such as Nike, to reward niche audiences. The former contract was awarded by competitive tender in 2011, with Tesco rating the Mobility Acuity solution better than any offered by a range of international competitors, demonstrating, as it did, extra capabilities beyond those called for in the brief. In the case of Nike, the MVS was used to increase brand awareness by allowing users to create a Limited Edition version of a trainer inspired by a photograph they took: an application for which the agency client, AKQA, won The Revolution Award for Innovation in 2009 and a D&AD Yellow Pencil Award in the same year.

TOTeM, too, has been particularly diverse in its impact. In May 2010, the tagging of donated goods was piloted in an Oxfam shop in Manchester as part of Future Everything, an award-winning new media festival and showcase. Over the course of the week-long pilot (called Remember Me), sales in the store increased by 53%, by 47% over the following two weeks and by 19% the week after. The event, which involved patrons recording downloadable audio `stories' about their donations, was covered by the New York Times and the BBC (there have now been over 80 print, broadcast and on-line articles published on the research). (See 5.3).

On the strength of Remember Me, Oxfam's Board of Directors invited the researchers to develop the Shelflife iPhone application and website (http://shelflife.oxfam.org.uk/). Over 1,600 objects were donated and tagged across ten pilot stores in the Manchester area. Feedback suggested that adding stories to secondhand items increased their worth and longevity and reduced disposability. The pilot's success led to Oxfam partnering further research, The Internet of Secondhand Things project, funded by the Engineering and Physical Sciences Research Council (EP/K012819/1).

In February 2011, TOTeM was awarded the Best Internet of Things Application at the BE2 Awards for Social Media in the Built Environment. In April the same year, the researchers co-created Tales of a Changing Nation, tagging 81 objects in the National Museum of Scotland (NMS) and allowing visitors to add their own stories using smart phone applications. Transforming the way in which used NMS `handling objects' and engaged with the public about communities and nationhood, Tales of a Changing Nation was showcased at the international MuseumNext conference in Barcelona (May 2012, see 5.4), at the British Museum, Science Museum and National Media Museum and published in the Museums Association Journal. The [text removed for publication] NMS described the work as "a great example of how museums can not only give visitors more information about objects and stories, but also involve our visitors in adding their own responses to the objects, whether personal reflections or additional resources " (see 5.5).

Sources to corroborate the impact

5.1 A testimonial from the Founder of King & Partners has been published in the following...

http://www.wwd.com/retail-news/direct-internet-catalogue/digital-world-now-coaxing-shoppers-back-to-stores-6002645?full=true See also a testimonial from a spokesperson for Phillip Lim in...

http://insidefmm.com/2012/10/phillip-lim-marketing/ See also a quote from a Mobile and Emerging Technologies Strategist for Acquity Group in...

http://www.luxurydaily.com/3-1-phillip-lim-bolsters-print-outdoor-with-mobile-scanning-app/

5.2 The testimonial from 7Digital forms part of a press release issued by the company on 15/02/11.
See... http://www.mobileacuity.com/wp-content/uploads/2013/03/MA-HMV-7Digital-Case-Study.pdf

5.3 Contact details for Oxfam [text removed for publication] have been provided separately. With regards to the Remember Me project at the Oxford Road shop in Manchester see also a BBC website article, featuring quotes from this source...
http://news.bbc.co.uk/local/manchester/hi/people_and_places/newsid_8680000/8680310.stm
... and the New York Times article referred to in Section 4...
http://www.nytimes.com/2010/09/05/magazine/05FOB-Consumed-t.html?_r=3 .

5.4 Contact details have been provided separately for [text removed for publication] the Manchester Museum, participant in the MuseumNext conference, Barcelona 2012.

5.5 Contact details for the [text removed for publication] National Museums Scotland have been provided separately. See also an article on The Guardian's website...

http://www.guardian.co.uk/edinburgh/2011/apr/14/edinburgh-national-museum-scotland-qr-codes