Summary of the impact

Video surveillance or monitoring is an important ingredient of modern life. Research conducted by the 2017Centre for Information, Intelligence and Security Systems` (CIISS), into improving the reliability of automated detection of visual entities in videos, has made an impact on public services and on practitioners (increased speed and quality, lower labour cost — Beneficiaries: U.K. Police; police investigators) and their health (mitigation of potential physical or psychological harm — Beneficiaries: police investigators), on society (reduction of a factor associated with crime rates and legal costs — Beneficiaries: the public; tax-payers), and on business (creation of a spin-out company - Adaptive Video Analytics Technologies Ltd — Beneficiaries: UK; and influence on management decisions about technology choices — Beneficiaries: Serco Group plc (HMP Dovegate)).

Underpinning research

2.1 Research challenge
Research conducted by CIISS, into video analytics, aims to develop innovative techniques which meet the need for automated video analysis in challenging indoor and outdoor operational settings, such as environments with changing imaging conditions, like unwanted illumination variation. Intelligent video analytics software, for applications such as visual surveillance or similar image understanding tasks, requires the ability to analyse videos using a fully or partially automated process. This process should extract salient information about the content of the images (for example, the objects displayed therein), such that the extracted information is as much as possible unaffected by changes in imaging conditions. However, existing solutions fall short of achieving reliable object detection under such conditions. Hence, CIISS engages in research to develop new image analysis algorithms for adaptive video analytics.

2.2 Spectral 360^®
Visual entities are generically referred to as "objects" by the image processing community. The detection of objects in images is a corner stone of video analytics systems, and reliable object detection enhances the effectiveness of these systems. CIISS researchers developed a breakthrough object detection algorithm (Spectral 360^®) which emulates the ability of human vision to detect objects under illumination changes or in the presence of other severe imaging conditions. Spectral 360^® is designed to address the main sources of deficiencies (such as changing illumination) which hinder the performance of video analytics systems. Spectral 360^® is a novel algorithm for object detection in images. It uses a physics-based model devised to determine the real colour of visual entities shown in an image, so as to improve the reliability of automated object detection under changes of illumination.

Field testing has demonstrated that this novel technique can detect events missed by a vigilant human operator (Sedky, 2010). The technique produces more accurate results, with higher object detection capability and more resilience to changes of imaging conditions, than other techniques available in the public domain (details are given in Section 3.2).

The key features of Spectral 360^® are its resilience to illumination changes, and its high capability to detect objects. It works with minimal intervention from a human operator, and offers reliable performance under adverse illumination conditions without a change of its parameters. These capabilities result in improved operational quality and efficiency, which builds on enhanced object detection, a reduced false-alarm rate, a reduced missed-detection rate, high adaptability to environmental conditions, and a low number of video frames required for training the system.

2.3 Key researchers and dates
CIISS research into reliable video analytics began in 2004 and it is still on-going. The key researchers are: Sedky (initially a PhD student, subsequently a lecturer), Chibelushi (Reader in Digital Media Processing), Moniri (Professor in Digital Processing Systems, who is returned under the 2017General Engineering` submission), and Badawi (former PhD student). They remain employed at Staffordshire University, except for Badawi who joined the Arab Academy for Science, Technology and Maritime Transport (Egypt) as lecturer, after his graduation from the University.

References to the research

3.1 References to research outputs
It should be pointed out that a significant part of the underpinning research has not been published due to commercial sensitivities; nonetheless, the research has produced a patent (EPO Patent No. EP2374109 A1) (Sedky, Chibelushi, and Moniri, 2008), peer-reviewed journal and conference papers (Badawi et al., 2012; Sedky, Chibelushi, and Moniri, 2005), and PhD theses (Badawi, 2011; Sedky, 2010). Technical details of the research are given in the first four references below; the fifth reference is included to show that explicit attention was given to the commercial relevance (and by implication, to the impact) of the work, throughout the research.

3.2 Evidence of the quality of the research
The quality and originality of the research both stem from the development of Spectral 360^® as a significant part of the original contribution to knowledge from doctoral research, which included commercial relevance as a key consideration throughout the project. The world-leading status of the research outcomes is evident from results of a worldwide open competition of image change detection algorithms, where Spectral 360^® has been tested rigorously against other algorithms (25 algorithms, at the time of writing) from all over the world, on a comprehensive benchmarking set of challenging videos. Spectral 360^® emerged as the winning algorithm and it has been the best performing algorithm since February 2013, the month it was uploaded onto the benchmarking website (results are available at http://www.changedetection.net/ [Accessed on 14/02/2013, 12/08/2013 and 11/11/2013]). The competing algorithms were produced by world-renowned institutions, including Massachusetts Institute of Technology, Technische Universität Berlin, Technische Universität München, Queen Mary University of London, Fraunhofer IOSB, intuVision Inc., National Research Center of Italy, University of Amsterdam, ICAR-CNR, Kyushu University, Brunel University, State University of New Jersey, Mitsubishi Electric Research Laboratories, École Polytechnique de Montréal, University of Washington, and Université de Sherbrooke.

The following grants were secured from the Higher Education Innovation Fund (funder: HEFCE), and from Advantage Proof of Concept Fund (APOCF) grant (funder: Advantage West Midlands) to pave the way for the commercial exploitation of Spectral 360^®.

• C.C. Chibelushi (Principal Investigator), M. Moniri (Co-Investigator), M.H. Sedky (PhD Student), HEIF3-Commercialisation of Video Surveillance Prototype, HEIF 3 grant - Funder: HEFCE, February 2008 - July 2008 (£20,000.00).

• C.C. Chibelushi (Principal Investigator), M. Moniri (Co-Investigator) , M.H. Sedky (PhD Student), Adaptive Video Analytics Technology: IP Landscape Assessment, Market Assessment and Business Plan, Advantage Proof of Concept Fund (APOCF) grant - Funder: Advantage West Midlands (local RDA), July 2009 - September 2009 (£9,414.00).

• M.H. Sedky (Principal Investigator), C.C. Chibelushi (Co-Investigator), M. Moniri (Co-Investigator), HEIF4-Commercialisation of Video Surveillance Prototype, HEIF 4 grant - Funder: HEFCE, January 2010 - December 2010 (£43,541.00).

• M.H. Sedky (Principal Investigator), Design and Implementation of a Video-based People Counting Solution for Serco, Innovation Voucher - Funder: HEFCE (£4,000.00). Client: Serco Group plc - HMP Dovegate.

Further research and development is on-going, with a £79,200 grant awarded to Adaptive Video Analytics Technologies Ltd, by the Technology Strategy Board (TSB) and the Defence Science and Technology Laboratory (Dstl), to integrate Spectral-360® into an asset protection system.

• M.H. Sedky (Principal Investigator) - Technical Director, Adaptive Video Analytics Technologies Ltd, 2017Spectral-360: A Physics-based Video Analytical System for Asset Protection`, Funders: TSB; Dstl, Project number: 33839-238212, Phase 1: 02/10/2013 to 31/10/2014 (£79,200). [Subject to satisfactory outcomes, the project would be funded for its second phase (running from 01/05/2014 to 30/09/2015) (£400,000 to £500,000)].

Details of the impact

Spectral 360^®, integrated into adaptive video analytics tools (referred to herein as "AVA software"), has had the impacts listed below.

4.1 Impacts on public services
Improved delivery of a public service (increased speed and quality; lower labour cost): AVA software significantly speeded up and reduced the cost of investigations undertaken by law enforcement officers. In two examples, AVA software automated the extraction of video evidence (much shorter than the original footage of the event under investigation); retaining only evidence of criminal activity by removing all segments which could not possibly contain any evidence.

A case investigated by Staffordshire Police in 2012 illustrates the labour savings which can be achieved with AVA software [3] (source listed in Section 5). Without AVA software, investigators would have been required to trawl manually through the 4,032 hours of video footage for this case; an error-prone and costly labour-intensive task. AVA software reduced the video to only about 30 hours to be manually reviewed by an investigator. This selective reduction enabled the investigator to prepare efficiently and accurately the evidence to go to court. The greater-than-hundredfold reduction of video viewing translated into a significant saving, estimated to be at least £24,000.00 for this case alone. The investigating officer concluded that "The benefits observed in this case has [sic] given us the opportunity to consider a review of our procedures and investigative options, in future cases involving CCTV and video footage where changes in illumination and other conditions are likely to be a factor. The use of the 360^TM Video Analytical [sic] methodology will be considered" [3] (source listed in Section 5).

In another example involving a smaller volume of video evidence, Leicestershire Police halved the length of their investigation of a case in 2013, through the use of AVA software. To quote the investigating officer: "This software helps eliminate wasted time viewing unrelated material, which can both speed up an investigation and save the force both man hours and money. [...] This 17 hour saving on time could free up an officer to conduct other enquiries for both this and other investigations, presenting more of a community impact instead of sat looking at a screen in the office" [3] (source listed in Section 5).

4.2 Impacts on society
Public benefits arising from the reduction of a factor associated with increased crime rates or legal costs: AVA software indirectly reduces crime rates. The significant reduction of the length of police investigations enables speedy prosecution, or simply a prosecution to happen, which (depending on resources available) may not have happened if staff were not available for the long periods of time required to analyse the videos. That, in turn reduces the likelihood of suspects being released on bail, while waiting for the Police to complete the lengthy manual review of the full video footage. By reducing the likelihood of suspects being released on bail and subsequently re-offending or absconding while on bail, AVA software indirectly contributes to the reduction crime rates and legal costs. It also releases police to undertake other work (as stated by the investigating officer at Leicestershire Police [3] (source listed in Section 5)).

4.3 Impacts on practitioners
Improved productivity and effectiveness of investigators: The automation provided by AVA software increases the productivity of investigators and the accuracy of their work, and boosts the confidence of investigators that no video evidence has been missed, as remarked by the investigator from Leicestershire Police [3] (source listed in Section 5).

4.4 Health impacts
Mitigation of potential physical or psychological harm: AVA software shortens significantly the length of time that investigators view video material. At best, such material is very tedious to watch; at worst, it may contain long static but very unpleasant imagery which, itself, can be harmful to the mental health of the investigator. The reduction of video viewing time also lowers the usage time of Display Screen Equipment which can cause injury or further tiredness for the user. Hence, by shortening viewing time, AVA software mitigates potential physical or psychological trauma.

4.5 Impacts on business
Creation of a spin-out company: Adaptive Video Analytics Technologies Ltd (Company number: 07451691; Registration date: 25 Nov 2010) was created during the assessment period, for the commercial exploitation of Spectral 360^®. The start-up company is currently conducting further development of its products, moving towards full scale commercialisation. It recently secured £79k from TSB and Dstl (details are given in Section 3.2). It currently employs a Technical Director, a Senior Researcher, a Researcher and a Software Developer. It has business premises at the Staffordshire University Business Village, in Stafford.

Influence on management decisions about technology choices: AVA software has influenced the strategic thinking of managers about the adoption of new technology within Serco Group plc (custodial services - HMP Dovegate), a leading private-sector provider of custodial services around the world. Steered by research-informed consultancy delivered by CIISS, Serco management installed closed-circuit television (CCTV) equipment at HMP Dovegate as a first step (feasibility study) towards adopting CCTV and AVA software across prisons administered by the Group.

Sources to corroborate the impact

[1] Staffordshire Police: Head of Staffordshire Police's Crime Support, Staffordshire Police. (Can corroborate the impacts relating to Staffordshire Police, detailed in Section 4.)

[2] Leicestershire Police: CCTV Coordinator, East Midlands Special Operations Unit, Leicestershire Police. (Can corroborate the impacts relating to Leicestershire Police, detailed in Section 4.)

[3] Factual statements, provided separately by Staffordshire Police and by Leicestershire Police, corroborating the claimed impact, can be supplied by Staffordshire University on request.

[4] Serco Group plc (HMP Dovegate). (Can corroborate the impacts relating to Serco Group plc, detailed in Section 4.)