Summary of the impact

Research at Kingston University into methods for tracking pedestrians and monitoring crowds using computer vision techniques has been translated into commercial products by Ipsotek Ltd and BAe Systems, resulting in economic benefits to these companies from sales of these products.

These products have been sold to high-profile customers including the London Eye, the O2 Arena and the Australian Government, providing significant commercial benefits, employment and growth for both companies, as well as providing an economic impact for these customers.

Underpinning research

From 1997 to the present, the Digital Imaging Research Centre (DIRC) at Kingston University has been carrying out research into how computer vision methods can be used for tracking pedestrians and detecting potentially dangerous crowd situations in public places.

From 2002 to the present, DIRC carried out research into new methods to extract useful information on crowd behaviour and dynamics from CCTV images, addressing the challenges posed by the visual clutter typical of crowds [1]. The new method used motion vectors for detecting crowded situations, and therefore could be used in the (MPEG-2) compressed domain, which thereby improves efficiency. This involved significant experimentation in major European underground train stations (London, Paris and Rome) [4]. These experiments demonstrated the feasibility of the detection of crowded scenes via automatic video analytics. The work done on using global image cues to measure crowdedness, static objects and predominant directions of movement, and to detect situations that might need operator intervention, formed the basis for industrial-level systems taken forward by Ipsotek Ltd, of which a co-author of [1] was (and is) the Technical Director. The research by DIRC identified the feasibility of processing imagery from standard CCTV systems to monitor people, highlighted challenges and novel ways to measure crowd dynamics without having to rely on individual pedestrian detection, and demonstrated the feasibility of such methods in real scenarios.

At the same time, DIRC worked on individual pedestrian detection and tracking [2]. Based on ideas developed earlier by Prof. Ellis and Dr Makris on learning transition times of moving objects between cameras [3], a multi-camera tracking system was proposed and tested in 2002. The key feature of this system was the capability to learn the relations between cameras using unlabelled training data, thus rendering it suitable for deployment in arbitrary environments without expensive and time-consuming configuration. Indeed, a co-author of [3] was recruited by Ipsotek and is still employed by this company, commercially exploiting this technology. The work was also submitted as part of an international challenge organised by NIST and the Home Office.

Dr Makris and Nebel were the academic supervisors for a KTP between DIRC and Ipsotek that transferred the above knowledge generated at DIRC to Ipsotek to improve several key components of the analytics technology: people tracking, people counting, and counting of other objects.

Key researchers: Sergio Velastin (2001 - 2012, Professor of Applied Computer Vision), Tim Ellis (2002 - present, Professor), Dimitrios Makris (2003 - present, Reader) and Jean-Christophe Nebel (2004 - present, Reader)

References to the research

Details of the impact

Sergio Velastin co-founded Ipsotek Ltd, a company providing video analytic products, in 2001, where he remains a consultant for R&D. The company now has 20 full-time employees. Ipsotek was engaged in a Knowledge Transfer Partnership with Kingston University from 2007 to 2009, resulting in at least £200K additional turnover.

The Ipsotek KTP developed people-counting technology based on the underpinning research described in [4] and [5]. This technology has been in use since 2006 at venues such the O2 Arena in London, the LG in Birmingham (NEC), Heathrow Airport, and various properties owned by the Casino Group. At the O2, it successfully coped with capacity crowds at events such as the Led Zeppelin concert at the O2 in 2007 and the Prince concert in 2011. The Ipsotek KTP has also provided an object-counting component, to differentiate between different categories. This has since been modified to create an `anti-backtrack' system for airport arrivals, which was been deployed at Edinburgh, Birmingham and Bristol airports in 2010-2011.

The intrusion detection technology deployed at the London Eye was developed to address the specific challenges of this environment: a round-the clock requirement to monitor land and tidal water approaches. The methods described in [2], [5] and [6] were used as the underpinning research to develop robust event (intruder) detection system, installed in 2007, and extended in 2011, also to process data from thermal cameras. Similar installations at Colt telecom, after more research and development, achieved i-LIDS primary accreditation.

A further set of research contracts developed a system based on the pedestrian tracking technology described in Section 2 above, especially the publications of Makris and Ellis. This involves an operator selecting a person on the screen and then an automated system tracking that person in a multi-camera network. The idea was taken forward to development in 2009-2011 through research contracts with Ipsotek (who became main supplier and IP owner) and BAE Systems (who are supporting the commercialization of this technology), and has become known as the "Tag-and-Track" system. The work was demonstrated successfully at Manchester Airport in 2010. It has won awards such as "Winner of UK SME category for the Global Security Challenge 2011" and runner up prize at "UK Home Office OSCT INSTINCT TD2 Challenge 2010". Ipsotek was awarded `CCTV System of the Year' at IFSEC 2012.

The link between Ipsotek Ltd. and BAe Systems also resulted in an installation of an advanced video analytics system including intrusion detection and abandoned package detection at the Australian Parliament in 2009, a £2M export project. This used components of the technology described in the previous two paragraphs.

The DIRC underpinning research has made a "significant impact" to the growth in Ipsotek's turnover, from approximately £400K to £1M. Furthermore, the totality of the economic impact also encompasses the customers at whose sites the Ipsotek technology was installed. At the O2 arena, the people-counting technology is used to calculate the tariffs charged to the organizers of events, and to the advertisers promoting products at these events. It enables precision scheduling of the site staff, to be on hand at the busiest periods. At the London Eye, the presence of the intrusion detection system enables other activities to be scheduled alongside this facility, while maintaining an institutional awareness of safety and security.

Sources to corroborate the impact

1. Finance Director, Ipsotek Ltd: All aspects of impact