Summary of the impact

Research undertaken at UEA developed revolutionary algorithms for making pictures look better. These algorithms were subsequently engineered into prize winning desktop and embedded applications, resulting in the creation of the spinout company, Imsense Ltd., in 2006.

In July 2010, Imsense Ltd. was acquired by [text removed for publication] and the Imsense technology has now been incorporated into [text removed for publication] product pipeline.

Underpinning research

The human visual system is quite remarkable: we see the details in the shadow and the clouds in the sky simultaneously even though the clouds might be thousands of times brighter. Yet, like digital images, the visual system does not code the world directly. Indeed there are at most, 100 brightness levels coded in the visual cortex. How the eye achieves this `dynamic' range compression (DRC) is not known but mimicking this ability is essential to achieve the best looking photographs.

Prior art in DRC assumed that the dynamic range compression could be solved by either adjusting brightness or contrast locally in an image. Finlayson proposed that differently lit parts of the same scene conveyed the same underlying information from the same physical surfaces but that this information was, effectively, `coded' in different units [1]. Standardising the units naturally required both local contrast and local brightness adjustments. Significantly, images recoded in standard units have much smaller dynamic ranges and also look good with hitherto invisible detail now apparent in images.

In common with other DRC approaches, the `local', spatially varying computations implied by the Imsense method raised two important questions [2]. First, how could the degree of `locality' be determined (evidently, some images need more DRC than others). Finlayson observed that the local filtering which was required could, equivalently, be cast as a problem of differentiating an image, modulating the derivatives and then reintegrating. In addition, Finlayson observed that very large brightness edge—such as shadows—are uncommon and yet they dominate any procedure that reintegrates the derivative field. In [3], the derivative field is reintegrated in a way that is robust to the unexpected large edges by using the tool of regularisation theory. Crucially, the degree of locality required `falls out' from the underlying regularisation theory.

Secondly, how could the required spatial processing be implemented in a way that avoided introducing spatial artefacts—such as haloing—into images (as are commonly found in competing algorithms)? Robustness to spatial artefacts is achieved by modelling the DRC computation as a provably optimal spatially varying look up table operation [4].

Complementing the robust reintegration work, a novel approach to finding differently illuminated regions (e.g. finding shadows) in an image was developed by Finlayson and Fredembach [5]. Building on previous commercially successful algorithms developed by Finlayson and Hordley [6], Imsense Ltd. also developed leading algorithms for white-balance leading to a patent by Finlayson and Trezzi.

Key Research Personnel:
Professor Graham Finlayson (UEA from 1999 to present)
Dr Barry Theobald (UEA Lecturer 2005 to present)
Dr Steven Hordley (UEA Postdoc and Lecturer, 1999 to 2006)
Dr Duan Jiang (UEA Fellow and Imsense employee, 2007)
Dr Francesc Tous (UEA summer intern 2004, 2005 and Imsense employee 2008 to 2010)
Dr Clement Fredembach (UEA PhD student 2003 to 2007)
Dr Elisabetta Trezzi (UEA PhD student 2003 to 2007)

References to the research

(UEA authors in bold)

Primary Publications

This paper, as well as reviewing the field, extended the state of the art in global tone curve adjustment (22 citations)

This paper develops a computational framework with respect to which the majority of white point estimation algorithms can be cast (338 citations)

Large dynamic range images typically have two or more illuminants (e.g. sun and shadow) and so it is useful — using the algorithm developed in this reference — to segment an image into regions lit by different lights (16 citations)

According to the ISI Web of Knowledge, Pattern Recognition and IEEE PAMI are respectively the 2^nd and 47^th ranked journals (from the 463 listed) in the area of Computer Science (with impact factors of 4.9 and 2.3 respectively). ICCV is the top conference in computer vision and has an acceptance rate of less than 20%. All citations from Google Scholar in September 2013.

Patents

[1] G.D. Finlayson
"Image Signal Processing"
WO/2004/051569 (2004) (copy of patent held on file at UEA)

[3] G.D. Finlayson
"Method and System for Generating Accented Image Data"
WO/2011/023969 (priority date 2010) (copy of patent held on file at UEA)

[4] G.D. Finlayson
"Method and System for Generating Enhanced Images"
WO/2011/101662 (priority date 2010) (copy of patent held on file at UEA)

Details of the impact

Imsense Ltd.: business and staff

The Intellectual Property which formed the core of the patent portfolio underpinning Imsense Ltd. pipelined from Finlayson's `Colour Lab' in the School of Computing Sciences at UEA. This portfolio included patents covering dynamic range compression [1], fast algorithms [3,4] and white balance. Imsense Ltd. was initially registered in 2006. It was supported by Seedcorn funding of £200K from the ICENI Seedcorn fund (www.icenifund.com) and raised venture capital from Braveheart Investment Group plc (supporting statement [A]) and IQ capital (supporting statement [B]) in two rounds of investment in 2008 and 2010. A significant portion of the investment into Imsense Ltd. came from COIN (supporting statement [C]): a regional `co-investment fund'. Like ICENI, this is an `evergreen' fund whose remit is to invest in companies in the East-Anglian region, with any returns reinvested into future regional projects.

At the end of 2010, Imsense Ltd. employed eight full time equivalent members of staff. The company was initially based in Norwich before moving to Cambridge in 2009. `Silicon Fen' as it is known, is the network of Cambridge-based engineering and science-based companies which thrive through the large number of co-located high-tech spin-outs and their interactions.

Imsense Ltd. worked with local businesses such as N++ and Argon design. N++ helped Imsense Ltd. develop its stand-alone PC software `imphoto' (see supporting statement [D]), whilst Argon Design helped assess the feasibility of implementing the Imsense technology on dedicated hardware — a key requirement sought by many of Imsense Ltd.'s potential customers (see supporting statement [E]).

Imsense Technology

Imsense Ltd. produced and sold software which had a wide impact in the photographic world. Its popular [text removed for publication] App imphoto attracted 100,000 downloads. Its Eye-Fidelity software was an integral part of the OnOne Phototune software (see supporting press release [F]) and Imsense stand-alone PC software was also available for purchase on-line. In January 2010 the Imsense software was a winner of an International Imaging Industry award (in the VISION 2020 competition) for its Eye-Fidelity Dynamic Range Correction technology, particularly as applied to real time video content processing (see supporting statement [G]).

The Imsense processing software was well regarded by photographers, including the internationally renowned photographer and author Michael Freeman, who has stated publicly:

"I've just revisited some images I'd previously processed using high-end photo-processing software. With imphoto I can get a more natural result in a fraction of the time"

(see supporting statement [H])

Acquisition of Imsense Ltd. by [text removed for publication]

In 2010 Imsense Ltd. was acquired by [text removed for publication] and the Imsense technology has now been incorporated into [text removed for publication] product pipeline (see supporting statement [I]) As a result of the acquisition, investments made in Imsense Ltd. returned a significant profit. The acquisition was also significant for the Imsense Ltd. employees. For example, the Engineering Team have now been working at [text removed for publication] for over two years.

On-going Collaboration

Following the Imsense Ltd. acquisition, [text removed for publication] established a four-year programme of collaboration with Prof Finlayson and the School of Computing Sciences at UEA. This collaboration has resulted in additional IPR, which is now being incorporated into new products.

Sources to corroborate the impact

Venture Capital Funding for Imsense Ltd.

[A] Supporting statement from Braveheart Investment Group plc — a major investor in Imsense Ltd. and observer on the management board
(letter held on file at UEA)

[B] Supporting statement from IQ Capital Partners — a major investor in Imsense Ltd. and member of the management board
(letter held on file at UEA)

[C] Supporting statement from `COIN' — a regional co-investment fund — confirming investment in Imsense Ltd.
(letter held on file at UEA)

Development of Imsense Products

[D] Supporting statement from N++ Ltd. confirming the provision of engineering support for the development of the imphoto stand-alone software
(letter held on file at UEA)

[E] Statement providing details of how Argon Design helped Imsense Ltd. develop `on chip' versions of their software processing technology
(letter held on file at UEA)

External Recognition for Imsense Products

[F] Press release from OnOne announcing the incorporation of Imsense Ltd.'s Eye Fidelity software engine into their Photo tune image processing software

(Downloaded from http://www.ononesoftware.com/press/release.html?r=2009-10-27-1 on 6/6/13 and held on file at UEA)

[G] Press release from the International Imaging Industry Association announcing a VISION 2020 award for Imsense Ltd.'s Eye Fidelity video processing embedded software

(Downloaded from http://www.prweb.com/releases/Imaging/Computer_Vision/prweb3188924.htm on 6/613 and held on file at UEA)

[H] Supporting statement by photographer Michael Freeman regarding his use of imphoto (letter held on file at UEA)

Acquisition of Imsense Ltd. by [text removed for publication]

[I] Supporting statement from [text removed for publication] confirming the acquisition of Imsense Ltd. in 2010 (letter held on file at UEA)