Video Analytic Methods and the Design of Advanced Technologies

Submitting Institution

King's College London

Unit of Assessment

Business and Management Studies

Summary Impact Type


Research Subject Area(s)

Mathematical Sciences: Applied Mathematics
Information and Computing Sciences: Artificial Intelligence and Image Processing
Studies In Human Society: Policy and Administration

Download original


Summary of the impact

Members of the Work, Interaction and Technology (WIT) Research Centre, King's College London have had a significant impact on the ways in which a number of global corporations and other major organisations design, deploy and evaluate advanced systems. They have developed innovative video research methods that have provided critical resources for organisations, including Hitachi, Xerox, BT, Microsoft, and London Underground, to undertake fine-grained analysis of work, communication and technology in complex organisational environments. Their methods and approach have formed the foundation to a range of more applied `interventions' in areas that include health care, transport, education, markets and the cultural industries.

Underpinning research

Drawing on ethnomethodology and conversation analysis, Heath, Luff, Hindmarsh and vom Lehn have developed methods and techniques to enable the fine-grained analysis of communication and social interaction in everyday settings (1). Using advances in audio-visual systems, these methods provide the opportunity to systematically explore how participants produce their actions in collaboration with others in the ordinary everyday settings in which they arise (2.3). They drive analytic attention towards the tacit skills and resources that participants rely upon in the accomplishment of everyday tasks and activities including for example the competencies that people rely upon in using tools and technologies. The methods involve techniques for data collection, for the transcription of talk, nonverbal behaviour and technology use, and for the detailed analysis of social interaction in work and organisational environments. The methods have also allowed for the development of a distinctive approach to experimental research primarily used to evaluate prototype technologies. By providing new ways of presenting social science research both to academics and `practitioners', this approach has also served to enhance more traditional forms of ethnographic research.

The methods have been used in an extensive range of empirical studies of complex organisational settings both in the UK and abroad. This research has usually been undertaken in close collaboration with partners in both the private and public sectors and includes studies of control centres, operating theatres, schools, medical consultations, auctions, design practices, and museums and galleries (4,5,6,7,8,9,10,11,12,13). The studies have made a significant empirical, methodological and conceptual contribution to the burgeoning corpus of research in the social sciences concerned with the analysis of human interaction and communication, work and organisations, technology and system design. They have had an important bearing on contemporary developments and debates in the social sciences including those within Ethnomethodology and Conversation Analysis, Communications, Work and Organisation Studies Computer Supported Cooperative Work (CSCW)and Human Computer Interaction (HCI) (1,2).

Through a series of collaborative projects funded by EU IST programmes, the UK research councils and various public and private sector organisations in the UK and abroad, Heath, Luff, Hindmarsh and vom Lehn have applied these methods to inform the design, development and deployment of advanced technologies and organisational systems (projects securing funding of over £25m). These projects include: (i) collaboration with European transport operators including the London Underground, the RATP Paris and STIB Brussels (12,10). Drawing from a detailed analysis of operators' practices we informed the design, development and deployment of image recognition systems for safety and security management in urban rail networks (EU IST Prismatica), (ii) collaboration with leading museums and galleries in the UK and abroad (Exploratorium, National Gallery, Science Museum, Tate Britain and Modern) as well as design companies and system developers. Analyses of natural occurring visitor behaviour identified design criteria and requirements for interactive exhibits and mobile resources (NSF, EU IST, EPSRC, AHRC, ESRC) (7,8), (iii)collaboration with the leading industrial research laboratories including BT Research Laboratories, Xerox PARC, and NTT Communications Science Laboratories (Japan). Studies of workplace activities informed the design and configuration of advanced systems to support remote real-time collaboration between remote personnel (9,10,13).

References to the research

Hard copies of references are available on request.

1. Heath, C.C. Hindmarsh, J. & P. Luff (2010) Video in Qualitative Research: Analysing Social Interaction in Everyday Life London: Sage. (Submitted as a REF 2 output)


2. Heath, C.C. and P. Luff (2000) Technology in Action Cambridge: Cambridge University Press.


3. Heath, C.C. (2013) Dynamics of Auction: Social Interaction and the Sale of Fine Art and Antiques. Cambridge: Cambridge University Press. (Submitted as a REF 2 output)


4. Heath, C.C. & D. vom Lehn. (2008-2012) Assessing Eye Sight and Ocular Health: The Practical work of Optometrists ESRC (RES-062-23-1391) (with City University, the Institute of Optometrists, various optometric service provides and practices).

5. Hindmarsh, J. (2007-2011) hapTEL (Haptic Technology Enhanced Learning). ESRC/EPSRC Technology Enhanced Learning initiative (with KCL Dental Institute, University of Reading).

6. Heath, C.C. & P. Luff. (2006-2009) Pervasive Computing and Market Trading EPSRC WINES Programme (with the Universities of Sussex, Southampton, UCL, & Christie's London & New York, Peter Wilson Fine Art and various other auction houses in UK and abroad).

7. Heath, C.C. & D. vom Lehn. (2004-2008) Enhancing interpretation: new techniques and technologies for the arts and decorative arts. AHRC with the National Gallery, V&A, National Portrait Gallery, Tates (Britain & Modern) and others.

8. Heath, C.C. & D. vom Lehn. (2004-2007) Communicating Science through novel exhibits and exhibitions. ESRC Science in Society Programme. (with the Science Museum, @Bristol, Natural History Museum, Land Design and others).

9. Luff, P. & C.C. Heath (2005-2008) PaperWorks. EU Framework Future & Emerging Technologies Research Programme (with Arjowiggins, Anoto, Acreo, Pearson, University of Malmo and ETH Zurich,).

10. Heath, C.C. & P. Luff (2004-2008) Partners PalCom: Palpable Computing. EU IST VI Research Programme. (Siemens, Whitestein Technologies, Alexandra Institute, the Universities of Aarhus, Siena, Lund, Malmoe, and others).

11. Heath, C.C. & P. Luff(2001-2004) Paper ++ EU IST Disappearing Computer Programme Heath and Luff KCL (with Hewlett Packard, Anitra, ETH Zurich and ArjoWiggins, Anoto, Acreo).

12. Heath, C.C. (2000-2003). PRISMATICA (Pro-active Integrated Systems for Security Management by Technological, Institutional and Communication Assistance) EU IST V Programme (DG VII) (with RATP, London Underground, STIB, Inrets, Matra, ATM, TIS, and others).

13. Heath, C. C. & J. Hindmarsh (2001-2004) SHAPE Hybrid Assemblies for Physical Environments. EU Disappearing Computer Programme (with the Universities of Nottingham, Limerick, the KTH Stockholm & various museums and galleries).

Overall, £25 million has been secured from research councils and industry with in excess of £3 million in funding secured for King's college from these grants.

Details of the impact

King's methods, findings and concepts have had a significant impact on the design and development of advanced technologies and organisational systems. Below we exemplify this impact under two headings: Design and Deployment of New Technologies and the Application of Video-Based Methods in Industrial and Service Sector Organisations.

A. The Design and Deployment of New Technologies
The impact of King's research and deployment of new technologies is typified by examples in two areas.

(i)Advanced Mediated Communication Technologies: Leading industrial research laboratories, including Xerox, BT and Thales, have used our methods and research to inform the design and development of advanced `media spaces' and `virtual environments'. Two particular examples serve to illustrate this impact of King's research. (a) NTT (Nippon Telegraph and Telephone Corporation). Over a period of four years, King's research shaped the development of a high bandwidth, high fidelity, `blended video environment' by NTT. This environment provides remote participants with visible and audible access to each other and the ability to share and work with physical objects such as paper documents as well as a range of electronic resources. Our methods, analysis and findings changed the way NTT approached research and design, replacing or enhancing (as appropriate) existing quantitative methods with qualitative methods (the ethno-methodological approach), as noted by a senior researcher at NTT Laboratories. (Source A). As a result of King's research, NTT understood `the sequential nature of the interaction process', which provided `great insight into further development and improvement of our t-Room (advanced media space).' (Source A). Thus, King's research provided the user requirements for these highly sophisticated technologies, and formed the basis for the design, development and evaluation of successive prototypes. (ii) London Underground Limited (LUL) King's research has shaped command and control in large distributed urban transportation systems in London, Paris and Brussels, as well as being used by Thales UK to provide the requirements for, and assess, prototype systems to support real-time collaboration between operation centres and mobile personnel in the emergency services. King's research had conceptual impact on London Underground, allowing it to `understand more clearly the impact on a range of operational matters of the use of new technologies.' This shaped decisions and development of intelligent CCTV systems to enhance safety and security management', which, in turn, `ultimately had an impact on technology spending decisions.' (Source B). King's research shaped London Underground's development of image recognition systems for security management.

ii. Augmented Paper Technologies: Industrial and service organisations in healthcare, transportation control, design practice, and markets have used King's research that identified ways in which paper documents still provide critical resources for production and coordination of many workplace activities. This is despite the wide-spread deployment of digital systems. This research funded by industry and the EU IST programme (Future and Emerging Technologies) provided the foundation for a series of collaborative research projects led by the King's Work, Interaction and Technology team. Partners in these projects included leading industrial research laboratories, system companies, computer science departments, and organisations providing resources for `end users' (Arjowiggins, Anoto, Acreo,Hewlett Packard, Paper++, Superinks, PaperWorks, Pearsons and the ETH Zurich). In contrast to the idea of the `paperless office', these projects were concerned with the design and development of systems that created ways of interweaving paper documents and digital resources. Through a series of fine-grained studies, we developed specific requirements for systems to augment conventional pen and paper, requirements that in turn demanded technical innovations in inks, printing, software architectures and hardware devices.

These requirements also provided criteria and methods for assessing successive prototype solutions and in advancing our understanding of the relationship between material and digital documents. They also informed the development of a deployment strategy for the systems being developed and their potential exploitation within various application areas including healthcare, education and markets. This impact can be illustrated with the following example:

Arjowiggins/Hewlett Packard King's research changed thinking at Arjowiggins, a major business in the paper industry. Founded on research in Physics, Chemistry and Maths, King's introduced the company to ethnographic research, which gave the company `an unexpected and very rewarding insight and rapport with our major customers such as Hewlett Packard', according to their former Exploratory Research Manager. (Source C). This transformation of research had profound further impact. On the basis of the findings with King's, Arjowiggins was able to `back this up with lucrative patents.' (Source C). The King's research also provided `insight into our product portfolio and allowed us to develop a range of innovative products.' (Source C). Thus, both Arjowiggins and their customers, such as Hewlett Packard, gained transformative benefits from the King's research.

B. Application of video-based methods in industrial and service sector organisations.
The impact of King's research and deployment of new technologies is typified by examples in two areas. King's research has pioneered the use of video-based methods to inform the design, development and assessment of advanced technologies. This has benefitted industrial and service sector organisations, including Hitachi, Xerox, Microsoft, Thales, Hewlett Packard, the National Gallery, the Science Museum and the Tate Galleries in London, altering thinking and providing them with benchmarks against which to measure their performance and improve it. Two examples illustrate this. (i) Hitachi The Design Division of Hitachi (Tokyo & Maidenhead) Hitachi is creating a significant number of engineering depots in the UK for the maintenance of high-speed trains. The King's team changed thinking, introducing a new approach to research and provided specialised methods training and field manuals to organisation analysts from Hitachi, who conducted wide-ranging analysis of work and collaboration in the depots. King's enabled Hitachi to `acquire the deeper insight (sic)'. (Source D). As a result the Hitachi analysts were able to propose initiatives for new practices within the depots, and to propose improved technological support for maintenance activities. (ii) Microsoft Microsoft is a global computing corporation, with a research division based in Cambridge, which is at the forefront of evidence-based studies of every day interaction with computers. King's research findings have `provided the measure against which we judge any and all work practice studies.' (Source E). Thus, King's research makes a continuing difference to Microsoft, enabling it to check and revise its own research and development.

Sources to corroborate the impact

A. Senior Researcher, Media Information Laboratory, NTT Communication Science Laboratories, NTT (Nippon Telegraph and Telephone Corporation),Communication Science Laboratories, Kyoto, Japan [confirms impact on development of NTT's t-room system]

B. Development Manager, Customer Service Strategy, Strategy and Service Development, London Underground HQ, London [confirms impact on intelligent CCTV systems]

C. Former Exploratory Research Manager, ArjoWiggins Fine Papers Ltd, Chateau-Thierry, France/Beaconsfield [confirms work with Hewlett Packard, informing design, development and assessment of advanced technologies]

D. Senior Researcher, User Experience Research Department, Design Division, Hitachi Research Laboratories, Tokyo, Japan [confirms impact on train maintenance & control]

E. Principal Researchers, Microsoft Research, Cambridge, UK [confirms Microsoft's use of research as benchmark against which all work practices studies are measured]