Summary of the impact

Methodologies for shape analysis developed by the Shape and Object Data Analysis group at The University of Nottingham (UoN) have underpinned important applications resulting in a range of benefits for companies and organisations, including in human movement capture and fingerprint modelling.

Firstly, the economic benefits of the methodologies developed at Nottingham to capture human movement data without a calibration trial have been used by a commercial software company, Charnwood Dynamics Ltd, and have saved time for its users and increased portability. Secondly, by incorporating research methods into practice, practitioners have improved standard processes, which have resulted in efficiency savings. Organisations which have benefitted from the research methods include the German Federal Police, where the methodology has been used in modelling growth in adolescent fingerprints, resulting in lower error rates and a reduction in false matches.

Underpinning research

Computer Vision has become a key enabling technology across a range of industrial and medical applications, including forensics, manufacturing, transport and disease diagnosis. A recent Frost & Sullivan analysis reports the global market for Machine Vision (the automated application of Computer Vision techniques) as $4.5 billion in 2012, projected to reach $6.75 billion by 2016 - tinyurl.com/o9f4jm7. A core requirement in many of these applications is the ability to recognise shapes; statistical shape analysis provides an important tool for achieving accurate shape recognition for difficult problems and under demanding operating conditions.

Pioneering research undertaken at UoN by Dr Huiling Le (UoN, 1991 to date, Associate Professor and Reader in Probability, School of Mathematical Sciences) provided the first detailed structure of Euclidean shape spaces [A1-A3]. Expanding on earlier work on the differential geometric structure of the shape and size-and-shape spaces [Le & Kendall, Annals of Statistics, 1993], Le produced detailed methodology for mean shape and mean size-and-shape estimation [A2] in 1995 and derivation of uniqueness conditions for the Fréchet mean [A2, A3] between 1995 and 2001. This research is part of a large body of work in shape theory and shape analysis conducted over the past 20 years by Le with Professor Ian Dryden, Professor Andrew Wood and colleagues (UoN, Dryden 2000-2010 and 2012 to date, Wood 1999 to date, both Professors of Statistics, School of Mathematical Sciences), that extends to the analysis of more general manifold valued data, e.g. [A4] and recent support from [A7, A8]. Practical results from these insights have been codified into an open source statistical package [A5] by Dryden, allowing researchers and practitioners access to these and other shape analysis methods in applications that require, and hence benefit from, accurate shape analysis. The software package, `shapes', was first made publicly available in 2003 as part of an EPSRC-funded grant [A6]. This has since been regularly updated by Dryden.

A specific example of a key research insight developed at Nottingham is the work on mean size- and-shapes by Le. Size-and-shape analysis is carried out where objects are compared with rotation and translation invariance, but not scale invariance. In 1995, Le [A2] gave definitive details of estimation of the mean size-and-shape and outlined various important properties including uniqueness of the mean. A fundamental question of practical interest is whether the population and sample mean size-and-shapes are unique; Le [A2] gave conditions for uniqueness which can be readily checked, and for shape and size-and-shape estimation this requires knowledge of the sectional curvature of the size-and-shape space, which was originally derived by Le & Kendall in 1993. A practical implementation of the sample mean size-and-shape estimate is available in Dryden's `shapes' package [A5], which is substantially based upon the research undertaken by the Nottingham group, as well as exploiting early theory developed elsewhere.

References to the research

The three publications that best indicate the quality of the research are indicated *

Grants:

[A6] EPSRC grant GR/R55757/0 `Identifying structure from shape and image data' PI: Dryden. Co- Is: Le and Wood. £157,196, 2001-2004.

[A7] EPSRC grant EP/K022547/1 `Statistical Analysis of Manifold-Valued Data' PI: Wood. Co-Is: Le, Dryden and Preston. £611,045, 2013-2016.

[A8] Royal Society Wolfson Research Merit Award `Object data analysis, with applications to medical images and molecular shapes' PI: Dryden. £60,000, 2012-2017.

Details of the impact

The impacts from the research developments at Nottingham in shape analysis have been made possible via a number of routes, including making the `shapes' software [A5] freely available and the description of the work in textbooks, such as Dryden and Mardia's book on Statistical Shape Analysis (1998, Wiley) and Kendall et al. [A1] in 1999, which have sold approximately 1800 and 760 copies respectively (as of 14/08/2013). These have helped to create links with users in industry and the police, who having become aware of the research and software via these links have subsequently made contact with Nottingham directly or via other academic collaborators.

The `shapes' package [A5] in particular has been one of the more popular downloaded packages on the CRAN (Comprehensive R Archive Network) website. Data from the University of California Los Angeles mirror on 30/09/2010 (http://neolab.stat.ucla.edu/cranstats/), one of the few from the 92 mirror sites in 43 countries/regions (http://cran.r-project.org/mirmon_report.html) that provides download data, reported that out of 2531 contributed packages the `shapes' package was the 112th most downloaded (520 downloads) from the 116th most separate IP addresses (379 IPs), i.e. in the top 5% of downloaded contributed R packages at that time (the most downloaded package from the mirror had 935 downloads, less than twice that of the `shapes' package). Sites at which the `shapes' package was publicised throughout the period 2003-2013 include locally at www.maths.nottingham.ac.uk/~ild/shapes and the SUNY Stony Brook Morphometrics website http://life.bio.sunysb.edu/morph/.

Commercial software

One specific illustration of the impact of the research on size-and-shape analysis [A2] is through the work of Dr Joel Mitchelson who works for Charnwood Dynamics Ltd and the start-up company Ogglebox Sensory Computing Ltd. Charnwood Dynamics has an established 3D movement analysis brand, Codamotion (www.codamotion.com), which is used in many settings, including clinical analysis, mobile gait labs, biomechanics, sports, orthotics and prosthetics, ergonomics, virtual reality and visualisation (tinyurl.com/kr64xfw, page 12). The company has an average annual turnover of £713k (2006-12) and is one of 8 main global players in the market for marker-based 3D movement analysis for life sciences (clinical and research use).

Mitchelson first confirms in his letter [B1] that: "The published results on shape spaces from University of Nottingham were instrumental in proving the convergence of an algorithm for measurement of the mean size-and-shape of a moving cluster of 3D markers. The algorithm and proof have now been accepted for publication in the Journal of Biomechanics, and are implemented in the open source library, Open3DMotion [tinyurl.com/ldyhkkc]."

The publication mentioned here is [B2], which makes clear its reliance on Le's work. Mitchelson's work arose after he contacted Le for a copy of [A3]. This led to independent work by Mitchelson and his team which, in particular, uses the sectional curvature calculations for size-and-shape space from Le & Kendall 1993 in the condition for uniqueness of the mean [A2, Condition C] to develop further results for occluded data. Mitchelson states by way of clarification that:

"This library forms the basis of the calculation engine within Codamotion's commercial ODIN software (http://www.codamotion.com/the-odin-software-suite.html) [Introduced in October 2011]." He adds: "The benefit to Codamotion customers is that the method allows rigid motions of clusters of markers on a human body to be tracked without a calibration trial, which can save them time. It also allows small portable 3D movement analysis systems to be moved around to measure very large volumes, using reference markers used to transform measurements from a moving system to a static reference frame. This opens up new market opportunities for the company, particularly for analysis of sports movements in the field, and ergonomics in industrial environments. The published shape space results from University of Nottingham are important for giving confidence in the results obtained from these new products."

That impact from this has already been realised is made explicit by Mitchelson in an email dated 23 July 2013 (copy on file): "A beta version of the algorithm has been in the software for some months. We're already able to engage with customers and potential customers about this due to the solid mathematical foundation, which is a result of the size-and-shape spaces work."

Fingerprinting

A further illustration is through the work of Dr Thomas Hotz (Ilmenau University of Technology) with the Bundeskriminalamt (Federal Criminal Police Office of Germany), where Nottingham research has led to efficiency savings after the methods were introduced into standard operating procedure within the Force. Hotz describes the profound impact of Nottingham research in [B3]:

"The aim of the study was to understand and predict the impact growth has on fingerprints of adolescents [B4, B5]. The difficulty automated fingerprint identification systems face when confronted with fingerprints of adolescents was that the points of interest used in matching algorithms move during growth, so that either tolerances in matching have to be increased, resulting in a worsened overall performance, or the fingerprint can no longer reliably be found in records after some years of growth. Understanding growth firstly is a matter of understanding whether it occurs isotropically, i.e. uniformly in all directions. If that were the case the shapes would not change during growth. We thus used the software package "shapes" developed by that [Nottingham] group [A5] as well as the methods described by the book co-authored by Ian Dryden in order to measure the anisotropy of growth. We found it to occur essentially isotropically, reducing the task of predicting its effects to the prediction of a single number, the growth factor, which simplified matters dramatically."

In specific experiments, [B4], error rates on a test set of 462 fingerprints were halved by scaling; this result was confirmed on the Bundeskriminalamt's database of 3.25 million right index fingerprints, where nine failures to retrieve a juvenile fingerprint out of 48 such identification attempts could be avoided by rescaling. Hotz notes the rescaling "...effect had not been understood, the European Union asked for a study to be conducted in this direction [Official J European Union, L131, 52, Regulation 390/2009, Annex 2, Article 2], and decided against the use of fingerprints of children under the age of 12 in visa applications [in May 2009]." This clearly demonstrates the importance of this problem and moreover the importance of the `shapes' package in this context. Further, Hotz notes: "It is worth mentioning that, roughly at the same time, the U.S. Department of Justice also had a study on the topic conducted which however appears to have failed to determine the effect of growth on fingerprints, and to produce a useful means for predicting it. This study did not use any shape analysis to look at anisotropy first, as they probably did not know of these techniques."

Overall, Hotz summarises by saying: "... I believe that without the Nottingham Group's research, making it available through software and textbooks, spreading their knowledge through further publications and conferences, this study could not have been conducted."

Thus Hotz's acquaintance with shape analysis, heavily influenced by personal contact with and publications of the members of the Nottingham group, has been key in solving the problem at hand. As Hotz notes in his letter, the results of his study were disseminated in 2011 at conferences involving persons from academia, industry and public office, e.g. from the biometrics industry such as Morpho (www.morpho.com), representatives of security forces such as the Metropolitan Police Service, and from the European Commission Joint Research Centre.

Other impacts of the work

There are other beneficiaries of the `shapes' package and, more broadly, of the underlying research undertaken at Nottingham. These include an impact as a teaching aid by introducing geometric morphometrics to biologists via an on-line workbook [B6], a face-shape study in patients with epilepsy [B7], use in radar signal processing, and applications in car headlight shape design [B8]. It is thus likely that the strong interest from the groups mentioned here and others will ensure impact from the Nottingham research will grow in reach and significance yet further.

Sources to corroborate the impact

[B1] Letter from Ogglebox Sensory Computing, Charnwood Dynamics Ltd, Leicester detailing work on human movement modelling software (copy on file).

[B2] Mitchelson, J.R. (2013). MOSHFIT: Algorithms for occlusion-tolerant mean shape and rigid motion from 3D movement data. Journal of Biomechanics, 46 (13), 2326-2329. http://www.jbiomech.com/article/S0021-9290(13)00262-5/abstract (copy also on file)

[B3] Letter from Ilmenau University of Technology detailing work on fingerprint modelling of adolescents with the German Federal Police (copy on file).

[B4] Gottschlich, C. et al. (2011). Modeling the growth of fingerprints improves matching for adolescents. IEEE Transactions on Information Forensics and Security, 6 (3), 1165-1169. (copy on file or through DOI: 10.1109/TIFS.2011.2143406).

[B5] Hotz, T. et al. (2011). Statistical Analyses of Fingerprint Growth. BIOSIG 2011 - Proceedings, Lecture Notes in Informatics, P-191, 11-20. (copy on file or through http://subs.emis.de/LNI/Proceedings/Proceedings191/11.pdf).

[B6] Zelditch, M.L., Swiderski, D.L. and Sheets, D.H. (2012). Geometric Morphometrics for Biologists, Second Edition. On-line companion materials. ISBN: 9780123869036 http://booksite.elsevier.com/9780123869036/ (includes full pdf of workbook)

[B7] Chinthapalli, K. et al. (2012). Atypical face-shape and genomic structural variants in epilepsy. Brain: A Journal of Neurology, 135(10), 3101-3114. DOI: 10.1093/brain/aws232. (copy also on file)

[B8] Ishihara, S. and Ishihara, K., Morphometrics and Kansei Engineering, in Proceedings of 10th QMOD Conference. Quality Management and Organizational Development. Our Dreams of Excellence (Editors: Dahlgaard-Park, S. and Dahlgaard, J.), 18-20 June, 2007 in Helsingborg, Sweden, Issue 026, No 142. www.ep.liu.se/ecp/026/142/ecp0726142.pdf (copy also on file)