Summary of the impact

The Boujou special effects software was developed from research carried out at the Department of Engineering Science. It enables sophisticated computer generated imagery (CGI) to be quickly and easily added to `real' film footage, facilitating the visual effects that feature so importantly in films such as Harry Potter and X-Men. The software has become an essential tool used by film-makers, TV advert producers, and video game manufacturers, and for instance played a pivotal role in helping `The Curious Case of Benjamin Button' win the 2009 Oscar for Best Visual Effects. Between 2008 and 2013, sales of Boujou totalled £1.37 million and this software boosted productivity and profitability right across the global digital entertainment industry.

Underpinning research

The research that underpins this revolutionary `move matching' software was carried out between 1995 and 1999 by Prof. Andrew Zisserman, FRS (who joined the Department in 1987 as an EPSRC Advanced Research Fellow and is now Professor of Computer Vision Engineering). Between 1999 and 2001, he and his team built software libraries to tie in with Boujou's release onto the market.

• Move matching is a technique that enables `real' film footage to be enhanced by special visual effects. It does this by taking into account the changing perspective that inevitably occurs whenever a camera moves around during the filming of a sequence. Prior to Boujou's development, move matching involved a painstaking (and costly) manual process. Alternatively, special camera equipment or calibration objects were needed — which was also an unattractive, expensive option. Working in the field of multiple view geometry, Zisserman devised a practical, affordable solution to the problem by developing (i) a theoretical understanding of how the geometry of multiple views works and (ii) software algorithms capable of calculating 3D geometry from multiple 2D images automatically, rapidly and using only a PC or laptop.
• In 1995, Zisserman and his team were working on guidance techniques for robots, exploring how the 3D geometry of a scene could be calculated from a stream of 2D images. A mathematical model was constructed that could relate multiple 2D images to one other, regardless of whether they comprised a sequence that had been shot by a single camera or produced by a number of different cameras [1].

• The research then addressed how the mathematical relationship between multiple 2D images could be estimated from real data, tackling the many measurement errors (e.g. missing data, mismatched image points) that contaminate the system. This was achieved by detecting outliers, and building a robust estimation computation. One crucial step involved changing video processing from sequential (i.e. solving for each frame, given the solution for the previous one) to batch (i.e. solving for all frames simultaneously); with this latter approach removing many of the problems previously encountered [2].
• The research then focussed on how the software could be used `in the wild', by identifying ways of overcoming problems arising with certain types of video sequences and 3D configurations [3]. This represented the critical move that transformed the original research software into a move matching version fit for real-world use: `Boujou'. The team went on to develop MLESAC [4], an algorithm for estimating multiple view geometric relations and a significant improvement over the standard RANSAC method previously used.
• The theoretical understanding developed in the course of the research, together with the algorithms for automatically estimating 3D geometry from multiple 2D images, were set out in a book first published in 2000 [5]; a second revised (paperback) edition followed in 2004.

Members of the team that developed the software also included: Paul Beardsley, Research Assistant 1995-1996; Andrew Fitzgibbon, Research Assistant 1995-2005; and Phil Torr, Research Assistant 1995-1997 and now Professorial Research Fellow at Oxford University.

References to the research

(best indicators of research quality are marked `Q')

Grants in support of this research: (Principal Investigator: Prof. Andrew Zisserman)

• EC ACTS: VANGUARD, 1995-1999, ECU 591,000 (Oxford component) (ref: AC074).

• EPSRC LINK: 2000-2003, £265k (refs: GR/M82790/01).

Details of the impact

Boujou was the first automatic camera tracking system developed worldwide, and has delivered a step change in the way that visual effects are incorporated into films, adverts, games, TV programmes and other environments where virtualisation plays a core role.

Societal Impact — quicker, cheaper production of more convincing special effects

By eliminating the need to rely on ponderous, expensive ways of computing a camera's movement, Boujou has made it simple and affordable to add sophisticated computer graphics to film: "on long, complex shots, using Boujou can cut two to three weeks off the completion time" [6(a)]. The software won a Technical and Engineering Emmy in 2002 [7] for its "revolutionary impact on the creation of complex visual effects". Since then, it has established itself as the preferred software for numerous film and visual effects companies and is used on almost every cinema film with CGI content. Key examples in the period 2008 to 2013 include:

Film	Awards/quotes
The Curious Case of Benjamin Button (2008)	[2009] Oscar for Best Visual Effects [8].
Tropic Thunder (2008)	Visual Effects Supervisor, CIS Vancouver [9]: “There were around 110 shots that required tracking. There was one shot in particular with Robert Downey Jr. coming in via helicopter and it crashes. The final edit looks seamless. Boujou has always been my weapon of choice
Angels and Demons (2008)	Nominated for a Visual Effects Society award in 2010.
Harry Potter and the Deathly Hallows Part 1 (2010) & Part 2 (2011)	Both won Oscars for Best Visual Effects. Part 2 also won a Best Special Visual Effects BAFTA.

Boujou's ubiquity is also reflected in its extensive use by individuals making their own films. Videos from all over the world acknowledge its contribution (see http://vimeo.com/tag:boujou) and many YouTube videos show how to use it (e.g. http://www.youtube.com/watch?v=7PsGdjwht_A). In addition, Hollywood's Gnomon School of visual effects offers professional training for users (see teaching DVDs for sale at http://www.robots.ox.ac.uk/~vgg/publications/2000/Torr00/torr00.pdf).

Economic Impact — commercial success and wealth creation

Software libraries developed as part of the underpinning research were licensed to 2d3, a company created in 1999 by Vicon (part of the Oxford Metrics Group) to sell Boujou to the film industry. In 2009, Vicon took over direct ownership of Boujou and, to date, has secured £1.37 million in revenue through sales of 654 licences for the software [10], purchased by:

- Post-production companies in the film and advertising industries, with Boujou making a key contribution to a 2009 Kleenex advertisement, for instance [11].

- High-profile television companies such as the BBC and CNN [10].

- Companies active in the computer game and architectural markets, e.g. Ubisoft and EA Games.

- Universities, where it has been used as a teaching tool, in sectors outside science and engineering, e.g. the Univ. of York, Dept. of Theatre, Film and Television http://www.york.ac.uk/tftv/about/building-facilities/equipment/; Florida State Univ., College of Motion Picture Arts http://film.fsu.edu/About/Facilities/Visual-Effects-Lab.

The commercial benefits of using Boujou, have continue to affect the worldwide entertainment industry. By using it, special effects sequences have become cheaper to produce and therefore much more ubiquitous, enhancing the entertainment value of films, TV shows etc., and translating into increased audiences and higher profits. Endorsements include:

- "We are constantly amazed at what Boujou is able to track. In fact, it has become a verb in the jargon at Ring of Fire: `Let's Boujou it!'" [6(b)].

- "Ease of use is a hallmark of Boujou, yet the software can do incredible things. The beauty of the application is that it just does what it needs to do: produce a moving camera from film or video without the need for the artist to track input manually" [6(c)].

- "Every 3D project that comes here, including Benjamin Button, starts with Boujou in our tracking department. Once shots are tracked, we can export cameras into Maya and use those 3D cameras as if we were on set with the director" [12].

Sources to corroborate the impact

6. http://www.awn.com/articles/technology/lets-boujou-it/page/3,1.contains three statements (a) CG Supervisor for The Orphanage Inc. corroborates that Boujou can cut production time, (b) Senior VFX Producer on `Ring of Fire' (2013 film) and (c) CG Artist, Zoic Studios both corroborate the impact Boujou has had within the film industry.
7. Article on Technical and Engineering Emmy in 2002. http://www.awn.com/news/awards/2002- engineering-emmy-awards-2d3s-boujou-and-apples-finalcut-pro
8. Use of Boujou on `The Curious Case of Benjamin Button' see:
   http://www.vicon.com/Content/PDFs/CCBB.pdf and
   http://www.omg3d.com/news/2009/88/boujou-helps-hydraulx-create-seamless-effects-for-the-curious-case-of-benjamin-button
9. Use of Boujou on `Tropic Thunder' see: http://www.metrics.co.uk/boujou/contact/101309.html
10. Chief Executive of Oxford Metrics Group (OMG), the holding company for 2d3 and Vicon. Corroborates the sales that Boujou has generated.
11. http://press.trustcollective.com/uploads/client_placements/rhino-3dworld_kleenex_PRINT_10.6.09.pdf
12. Lead Camera Tracker, Hydraulx, who worked on `The Curious Case of Benjamin Button'. Corroborates the impact Boujou has had within the film industry.