Summary of the impact

Research at the University of Cambridge Department of Engineering on computer vision tracking led to the creation of Extra Reality Limited in 2010, which was subsequently acquired by a new company called Zappar Limited in May 2011. Zappar employs 17 staff and had revenue of GBP612k in the financial year 2012/13, an increase of 35% on the previous year.

Over 50 different brands have used Zappar's augmented reality application across more than 300 offerings in over 17 countries to deliver entertainment-based marketing interactions from 2011 to 2013. [text removed for publication] Examples of partners include Disney, Warner Brothers and Marvel. Zappar has changed attitudes in the media sector by showing that "augmented reality is finally ready for prime time" (President, Creative Strategies Inc, Time Online, 2012).

Underpinning research

Dr Roberto Cipolla became a Lecturer in the University of Cambridge Department of Engineering (DoEng) in 1992 and was promoted to Professor in 2000. He has led work in the Computer Vision group, including work on recognition, since 1992. Dr Tom Drummond joined the Computer Vision group in 1998 as a postdoctoral Research Associate working for Cipolla. He became a Lecturer in 2001. He specialised in visual tracking and Augmented Reality (AR).

Cipolla was the Principal Investigator (PI) for two grants undertaking fundamental research on video servoing to guide robots to their target using an image-based feedback control loop (EPSRC 1996-1999 and EC 1998-2001). The research proposed new techniques for extracting geometrically salient curves from arbitrary viewpoints and matching them with curves based on geometric invariants. This enabled feature-based tracking, which was essential for creating a solution that could operate with real-time frame rate images and modest computing resources. The results of theoretical work were tested in experiments; the experiments drove further theoretical work, as the test subjects became more complex and the conditions under which tracking had to be maintained became more demanding. The results were published in 2000 and 2002 (Ref 1 and 2) by Drummond and Cipolla. They represented the state-of-the-art for real-time and robust 3D model-based tracking.

This research success led to Drummond winning further funding as the PI for two grants: "Hybrid tracking and spatial reference technologies for new human machine interfaces" sponsored by ABB (2005-2007) and "Human Computer Interfaces for Command and Control" sponsored by Boeing (2007-2010).

Drummond's ABB grant made advances in using structure-from-motion techniques to create a 3D model from a sequence of images, which enabled robust tracking without the need to generate 3D CAD models and register them to image information (Ref 3).

Drummond's Boeing project aimed to use augmented reality to provide an improved interface for emergency services and their coordinators. The research resulted in systems that could, for instance: track a paper map on the command room table and project accurately on its surface additional live information about the emergency and the deployment of services; and overlay an image of a scene viewed live on a smartphone with additional information about hazards, water hydrants, etc. Theoretical work was tested in experiment in order to find viable solutions.

Drummond's team had to research new methods of tracking that would be very accurate, fast, robust and computationally efficient. The work with smartphones used a novel edge-based tracker that used a textured 3D model that made scale-based detail culling automatic, improved matching through appearance-based edge signatures and allowed the use of more commonly available models (Ref 4).

Drummond's continued focus on smart phone applications, with Boeing's support, resulted in an approach that used the novel idea of Histogrammed Intensity Patches. These patches are independent histograms of the quantised intensity of a group of pixels. They can be stored in less than 60kb, yet provide enough information for fast feature recognition, which was further accelerated by Drummond's tree-based lookup scheme. In comparison with a leading competitor's solution in 2008, Drummond's solution was more robust and reduced both runtime and memory

Cipolla took over as PI for Drummond's research grants, when Drummond left the DoEng in 2010 to become a Professor at Monash University, Melbourne.

References to the research

*indicates papers best demonstrating the research quality.

Details of the impact

Drummond encouraged and supported students Simon Taylor and Connell Gauld to commercialise his group's algorithms and know-how. The three jointly founded Extra Reality Limited in June 2010. Extra Reality developed the DoEng smartphone-based tracking research into an exemplar proof- of-concept AR application (app), called `Popcode'. The app, a free download for Android-based smartphones, and later iPhones, delivered context-related information about an item using a mobile phone. Users opened Popcode on their phone when they saw the Popcode logo printed on a magazine page, poster or leaflet. They pointed their phone camera at the logo. Popcode recognised information hidden in the logo and downloaded the relevant AR content. The user then turned their phone camera from the logo to whatever picture was printed alongside the logo, watching this picture on their phone. Popcode overlaid the image with animations, information and interactive content. The underpinning research ensured that the AR system was efficient, robust and fast enough to provide a high-quality experience on a smartphone. Early projects created a high profile for the company. For instance, the University of Cambridge used Popcode in 2011 to augment the brochure for its Science Festival, which attracts over 35,000 visitors.

VEEMEE Limited, a creative media/marketing company with significant links to both fast-moving consumer goods businesses and creative media companies, recognised the value of Popcode and in May 2011 formed Zappar Limited, as a joint venture with the shareholders of Extra Reality, for the purpose of acquiring Extra Reality Limited. [text removed for publication] Zappar Limited purchased the intellectual property (IP) created by the DoEng research from Cambridge Enterprise, while Boeing retained a non-exclusive licence to the IP created with its sponsorship.

To date, Zappar has not received any external venture capital funding and has been self-financed from its operating revenue. The company has two models for generating revenue from projects for clients: (1) production advances for AR-enabled products plus royalties on sales achieved; (2) fixed project fees. Zappar's revenue in the financial year 2012/13 was GBP612k, which it reinvested in product development, recruitment and global expansion. As of June 2013, it employs 17 people, an increase of approximately 13 since 2011 (Ref 7).

Zappar is now established as a full-service company that provides both the AR delivery platform and the creative content for clients. The Zappar process is essentially unchanged from the Popcode process and remains underpinned by DoEng research results. Each user experience of using Zappar is called a Zap.

A Zap lasts on average 90 seconds, 60% of all Zappar users zap the same content more than once and the experience requires active participation (Ref 7). Firstly, as a comparison, the average duration of conventional online video advertisements is 24 seconds (Ref 8), which is less than a third the length of an average Zap. Secondly, only a fraction of viewers watch online video advertisements to completion. Thirdly, the video advertising experience is passive, whereas a Zap actively engages the user. This means that Zappar creates longer, more repeated and more active engagement than conventional methods of advertising. Zappar sees the benefit to its brands partners and licensees is through its ability to add incremental engagement and therefore media value to marketing activity as well as creating new product lines, stock-keeping units (SKUs) and incremental sales for license holders (Ref 7).

Zappar has created over 300 different experiences for over 50 different brands and licensed properties in 17 countries around the world over the last two years (Ref 7). It has created partnerships with licensees (principally T-shirt manufacturers), which has led to relationships with licensors, such as Warner Brothers and Disney. Current Zap-enabled products include T-shirts, caps, accessories, greetings cards, books, CDs, posters, puzzles, calendars, magazines and toys. These goods are used to deliver added value content for some of the world's most popular properties, including, Marvel's Spiderman, the pop group One Direction and Skylanders, which is an interactive console game created by Activision (Ref 7). [text removed for publication] Zappar client work since its launch in 2011 includes the following:

• Over 1.5 million Zappar-powered tee-shirts have been sold in retail outlets such as JCPenney, Macy's, Wal-Mart, Target, ASDA, C&A and Woolworths in countries that include Australia, Canada, Mexico, South Africa, the UK and the US (Ref 7)
• Zappar was used for the limited edition picture box set version of the 2012 album "Take Me Home" by the pop group, One Direction. Buyers gained access to exclusive content, brought to life using Zappar, which included behind-the-scenes videos and photos of the group. [text removed for publication]
• Zappar was used for exclusive features with Fabulous (a free magazine supplement in News International's "Sun on Sunday" paper with a readership of over 4 million people). Features for Leona Lewis and One Direction were enhanced with Zappar to promote their new albums (Ref 7). The editor of Fabulous stated: "We wanted to offer our print readers an enhanced experience and Zappar's cutting-edge technology has enabled us to do this. We look forward to pushing the boundaries with them in the future" (Ref 9)
• A cover promotion for Rogue magazine in association with "Magnum" ice cream, featuring the actress and TV presenter Solenn Heussaff, used Zappar (Ref 7)
• The Tatler lifestyle magazine used Zappar to augment its content for the the Queen's Diamond Jubilee in 2012 and Valentine's Day in 2013 (Ref 7)
• Zappar was used in the promotions for the UK release of Warner Bros' "Happy Feet 2" film in 2011, which included a Zappar-powered posters that triggered a dancing penguin and the ability to view the film's trailer (Ref 7)
• A similar poster-based promotion was used for the cinema release of the film "Harry Potter and the Deathly Hallows: Part 2", "Sherlock Holmes: a Game of Shadows" and "Green Lantern" for Warner Bros (Ref 7)
• Zappar engaged with ASDA in March 2013 to promote an in-store Easter egg hunt [text removed for publication]. Head of Events Marketing at ASDA stated, "The AR Easter Egg Hunt is a great way to get across the ASDA personality and another example of how ASDA stores are a destination for mums and kids. We are really excited to be embracing this new technology and incorporating it into the ASDA app. Hopefully everyone enjoys meeting the Easter Bunny!" (Ref 10). Zappar, with ASDA, won the Award for Best Practical Case Study at the 2013 Augmented Reality Summit in London in June 2013 for which the judges' criteria were: concept of use of AR; imagination, innovation and uniqueness; commercial feasibility; market perception; and achieved results/return on investment (Ref 11).

Time Online ran a feature on Zappar in October 2012 in which the President of Creative Strategies Inc, an established Silicon Valley industry columnist, presented his independent view: "A few weeks back, executives of a U.K. company called Zappar came to my office to show me their augmented reality technology. I have been looking into augmented reality for over a decade and have only seen false starts. But for the first time, I finally saw a technology and app infrastructure coming together that suggests to me that augmented reality is finally ready for prime time." He added, "Also, of the AR apps I am aware of, Zappar seems the furthest along in getting big support from ad agencies, Hollywood, game makers and others" Ref (12).

Zappar hired two further developers in April 2013 to work on its new self-serve Zapcode Creator system, which will allow consumers and businesses to create, publish and share their own AR content on the Zappar platform.

Sources to corroborate the impact

7. Founder and Managing Director at Zappar
8. "comScore Releases March 2013 U.S. Online Video Rankings", Press release by comScore Inc, 25 April
   2013, http://www.comscore.com/Insights/Press_Releases/2013/4/comScore_Releases_March-2013_U.S._Online_Video_Rankings
9. "One Direction AR magazine cover", Article in Mobile Entertainment, 1 October 2012, http://www.mobile-ent.biz/news/read/one-direction-ar-magazine-cover/019510
10. "Zappar and Asda bring to life the classic easter egg hunt", Zappar press release, 13 March 2013, http://www.zappar.com/blog/zappar-and-asda-bring-to-life-the-classic-easter-egg-hunt/
11. "The Award Winning Zappar App™", Zappar press release, 21 June 2013, http://www.zappar.com/blog/the-award-winning-zappar-app/
12. "With Zappar, Augmented Reality Is Ready for Prime Time", Feature by President of Creative Strategies Inc, 22 October 2012, Time Online, http://techland.time.com/2012/10/22/with-zappar-augmented-reality-is-ready-for-prime-time/