Faster CCTV video content analysis

Submitting Institution

Royal Holloway, University of London

Unit of Assessment

Physics

Summary Impact Type

Technological

Research Subject Area(s)

Information and Computing Sciences: Artificial Intelligence and Image Processing
Medical and Health Sciences: Public Health and Health Services


Download original

PDF

Summary of the impact

The explosive growth in the number of CCTV cameras has meant that analysing the volume of data produced has become almost unmanageable. Dublin based start-up Kinesense Ltd was incorporated in 2009 by Dr Mark Sugrue, who had carried out his PhD in Video Analytics at Royal Holloway. New methods to detect motion, track objects and classify behaviour in CCTV now enable the efficient scanning of video for important events. Kinesense Ltd has developed a range of forensic video analysis tools, which reduce the time required to search and analyse video footage by up to 95%. It has attracted investment funding of over €820,000, employs 7 full time staff and has made sales to police forces and security agencies in over 17 countries. Kinesense products benefit law enforcement professionals and organisations by providing more efficient surveillance and detection of criminal activity, allowing better use of investigator time, reducing the length of criminal investigations and increasing their success rate. The general public worldwide benefits from increased crime detection and the consequent prevention and reduction of criminal activity.

Underpinning research

Professor E. R. Davies (Royal Holloway Department of Physics until 2008, currently Emeritus Professor) is an international authority on machine vision. He led the Signal Processing and Machine Vision research group in the Department, until his retirement in 2008. He was awarded Distinguished Fellow status by the British Machine Vision Association in 2005, and was elected Fellow of the International Association for Pattern Recognition in 2008. He is author of Machine Vision: Theory, Algorithms, Practicalities (Academic Press, 4th edition 2012), Image Processing for the Food Industry (World Scientific 2000), Electronics, Noise and Signal Recovery (Academic Press, 1993).

The underpinning research on motion detection was undertaken by Davies in conjunction with his PhD student Mark Sugrue (2003-2007), as part of an EPSRC Basic Technology grant [G1], Next generation artificial vision systems: reverse engineering the human visual processes.

Traditionally the analysis of video surveillance or CCTV footage has been carried out by trained personnel who view lengthy footage aiming to identify and time-stamp key events for use as evidence in court, or to aid the police during an investigation. This laborious process is time consuming and liable to human error. The field of video analytics has grown, driven by the need to efficiently exploit the increasing coverage by CCTV, in order to better promote safety, security and the detection of crime.

The key advance at Royal Holloway that led to the impact described in this case was to devise a totally new approach for the analysis of CCTV footage, based on algorithms which mimic the human visual system (HVS). This was inspired by the near perfect ability of the HVS to detect and track moving objects that we are all familiar with. Reverse engineering the HVS resulted in a new paradigm that differs fundamentally from conventional approaches, and achieves major improvements in efficiency and speed. Prior video analytics techniques were based on simple motion detection; they detect when groups of pixels change and use the size of the group as the key characteristic to distinguish between frames. This primitive approach produces limited and unreliable search results. The background modelling paradigm struggles with signal-to-noise fluctuations and changes of shape of moving objects make them difficult to track.

In the new approach initial detection of moving objects was achieved using a new motion distillation paradigm, which employs spatio-temporal wavelet decomposition of video. It was demonstrated that this method is more robust than traditional background modelling techniques, while being computationally less expensive. As with the HVS, the approach uses a dual channel tracking architecture. The motion channel, generated through motion distillation, handles object detection and initialises tracking. The form channel is used to resolve tracking ambiguities and occlusions. A new approach was also made to the analysis of human behaviour in the footage. Objects were characterised into vehicles, pedestrians, runners, groups, and unknown.

The research at RHUL [1-6] which formed the basis of Sugrue's PhD, led to a licensing agreement between Royal Holloway and Amideon Systems Ltd (a company providing electronic solutions to the aerospace and civil security sectors) in March 2008. Amideon evaluated the performance of the code using real-world CCTV footage, which had not been available at Royal Holloway. The initial investigation using the low quality, low frame rate footage typical of most security CCTV capture was unsuccessful. Sugrue, informed by these challenges and building on the principles of the original Human Visual System inspired algorithm, and the knowledge and experience gained at Royal Holloway, developed a new successful proof-of-concept product that was now able to deal with real CCTV footage.

In late 2009, Sugrue and his business partner Sarah Doyle formed Kinesense Ltd and worked closely with the Irish Police (An Garda Síochána) during 2010 to turn the initial proof-of-concept into a functioning real-world product. Kinesense has attracted €820,000 investment from private investors, venture capitalists and Enterprise Ireland's High Potential Start-Up (HPSU) funding programme. The product was launched in the UK in 2011 and is now in use by 20% of UK police forces, helping solve hundreds of serious crimes.

References to the research

Selected peer-reviewed research papers, and academic publications related to impact; * denotes a publication indicative of the quality of the underpinning research.

1. 2008 Next Generation Artificial Vision Systems: Reverse Engineering the Human Visual System, Chapter 9: Motion detection and tracking by mimicking neurological dorsal/ventral pathways, Artech House Series Bioinformatics & Biomedical Imaging, Mark Sugrue & E.R. Davies

2. 2007 Motion signals for provision of rapid discernment of pedestrians and pedestrian behaviour, Electronics Letters 43, Issue 23 (2007) 1267-1269, Mark Sugrue & E.R. Davies

 
 
 

3. *2007 Contrast independent motion detection using 'inverse pair' spatio-temporal edge detectors, Electronics Letters 43, Issue 24 (2007) 1346-1348, Mark Sugrue & E.R. Davies

 
 
 
 

4. *2006 Motion distillation for pedestrian surveillance, Sixth IEEE International Workshop on Visual Surveillance, Graz, May 1, 2006, Mark Sugrue & E.R. Davies

5. 2005 Tracking in CCTV Video Using Human Visual System Inspired Algorithm, Visual Information Engineering 2005, Mark Sugrue & E.R. Davies

 

6. *2005. Image Analysis in Crime: Progress, Problems and Prospects, Proc. IEE Int. Symposium on Imaging for Crime Detection and Prevention (ICDP 2005), IEE, London (7-8 June), pp. 105-112 (2005) E.R. Davies

 

Research grant related to impact

G1. EPSRC Basic Technology Grant, GR/R87642/01, PI E.R. Davies, Next generation artificial vision systems: reverse engineering the human visual processes (2003-2007) £257,985.

Details of the impact

"When business development manager Sarah Doyle first saw the video tracking technology created by Mark Sugrue as part of his PhD thesis, she says she recognised its commercial potential instantly. As a result she left her job [...] and teamed up with Sugrue to develop the technology into a marketable product." (Irish Times interview, October 2012).

The Chief Technology Officer of Kinesense Ltd, states in a letter of support: "I strongly believe that the insights and education I gained at Royal Holloway Physics Dept helped me to take a radically new approach to the technical problem of CCTV video, and to help build a successful start-up company that is providing both employment and real social impact, helping police solve serious crime around the world."

Kinesense Ltd (http://www.kinesense-vca.com/) was incorporated in 2009 in Dublin, Ireland, led by Chief Executive Sarah Doyle and the Chief Technology Officer, who obtained his PhD in video analytics from Royal Holloway. The company specialises in video content analysis, and has developed a tool for video retrieval and analysis aimed at the law enforcement and security markets.

The company's technology allows users to search through video footage using filters to pinpoint certain types of activity. The system works by indexing the CCTV footage and then allowing the watcher to pinpoint areas of interest, for example movement near a door or someone wearing a particular colour. The technology identifies objects based on their unique motion pattern, and can reliably identify humans, vehicles, colour and direction of movement.

An officer investigating a break-in can use the system to pinpoint a period on a CCTV tape that is of interest rather than watching through the entire tape. For example, if an ATM was vandalised by someone in the middle of the night, the investigating officer could use the software to detect at which times there was human movement near the machine. The efficient algorithms mean that, with modest computing power, multiple video files can be batch-processed at speeds far higher than could be achieved with a human reviewing the footage, leaving just the critical sections of the video to be watched by the investigating officer.

Kinesense Ltd Chief Executive Officer Sarah Doyle estimates that the time required to search a piece of footage can be reduced by 95 per cent using their software. The improvement in speed of processing raw footage has been of benefit to the Major Crimes Division of the Irish Police force who said, about the Kinesense Law Enforcement (LE) product, "Kinesense LE enabled us to deliver a complete video timeline of a high-profile murder case to the investigation team within 5 hours of the actual shooting event."

The company's technology, which is sold as helping the global war against crime, is currently being used by police forces, security agencies, counter-terrorism units and serious crime units in Denmark, Ireland, the UK, Venezuela, Canada and North Africa. Kinesense Ltd Sales Director Tony Cahill has added more than 10 countries to this list so far in 2013 - Italy, the Czech Republic, Slovakia, Turkey, Malaysia, Singapore, Indonesia, Brunei, Australia, the United Arab Emirates and Brazil.

Kinesense has also expanded its customer base beyond law enforcement agencies. British Telecom bought the technology to help stem losses of up to £2,000 a day through the theft of copper wire. The telecoms company found thieves were stripping the valuable copper from utility poles and sub-stations, causing disruptions to their services. The Kinesense technology allowed BT to analyse CCTV footage in 30 minutes compared with eight hours previously, speeding up the opportunity to arrest and prosecute thieves.

Kinesense Ltd was initially supported by a €200,000 investment from Enterprise Ireland under the High Potential Start-Up (HPSU) funding programme. In September 2012 Kinesense Ltd secured a further €620,000 syndicated investment to focus on its international growth strategy. The funding round was led by Kernel Capital through the Bank of Ireland Seed and Early Stage Equity Fund, which contributed €500,000. The remainder of the money was contributed by the Irish BES (Business Expansion Scheme) and angel investors.

By October 2012 Kinesense Ltd employed 7 full-time staff, including another Royal Holloway Physics PhD graduate from the Signal Processing and Machine Vision group, Dr. Daniel Ellin, as Lead Software Engineer. The company is now actively recruiting additional staff to support its growth.

Sources to corroborate the impact

The Chief Technology Officer and Director of Kinesense Ltd can corroborate the relationship between the underpinning research and the impactful activities of Kinesense.

Copies of all documents can be made available to the panel.

Documents corroborating impact of Kinesense:

Documents corroborating investment in Kinesense: