Articulate Instruments - visualising speech

Submitting Institution

Queen Margaret University Edinburgh

Unit of Assessment

Modern Languages and Linguistics

Summary Impact Type

Technological

Research Subject Area(s)

Information and Computing Sciences: Artificial Intelligence and Image Processing, Computer Software, Information Systems


Download original

PDF

Summary of the impact

Articulate Instruments Ltd. was founded in 2003 as a research, design, manufacturing and consulting company for users of phonetic instrumentation. It invents, designs, markets and supports instrumental technologies for normative and clinical speech science and for the diagnosis and treatment of speech disorders. Products include electronic systems, headsets, software, and methodologies, underpinned by QMU research. Clinical use of relevant products as medical devices requires "CE marking" to prove on-going safety and support, first achieved in 2004.

Impact relates primarily to the company's on-going financial health and its non-academic customer base. In its first 10 years, turnover averaged ~£120k, with over 200 customers internationally, of whom more than 50 were non-academic.

Underpinning research

The underpinning research includes basic speech science, electrical and software engineering, articulatory phonetics and informatics. Over two decades, we have researched novel and unique combinations of phonetic instrumentation and methodology for both clinical and non-clinical topics. There has been a focus on enhancing reliability, usability and flexibility, with a view that its use should be extended to non-academic users, particularly in clinical contexts. Below, names that are CAPITALISED indicate staff with outputs included in this REF UOA.

  • WRENCH spent two years developing a MS-Windows version of Electropalatography (EPG) at QMU (2001-2003) then left to set up Articulate Instruments Ltd in September 2003 as a spin-out company. This was part of QMU's strategy to ensure the long term viability of EPG as a "clinical device" (see below). He had been a post-doctoral researcher at QMU from 1995 to 2003 on the QMU/Edinburgh University Speech Production Facility (ESPF) and other clinically-oriented speech technology projects. Strategic internal funding (1996-1997) and the EPSRC-funded project on speech recognition using articulatory data (1998-2001) were used to research and develop implementations of EPG in MS-Windows, replacing the obsolescent DOS-Windows version. EPG became the core of both a clinical system (Articulate Assistant) and then a multi-channel research system (Articulate Assistant Advanced, "AAA"). The company has been an independent entity, with WRENCH as its sole direct employee/director since 2003. However, close ties have been retained with QMU through collaborative research and shared strategic goals with the former Speech Science Research Centre (SSRC), now Clinical Audiology, Speech and Language Research Centre (CASL, 2011-2013).
  • Professor Bill Hardcastle was appointed by QMU (1990-2009). This initiated more than 20 years of continuous research and development into articulatory instrumentation. From 1996, Hardcastle secured the funding to develop the MS-Windows version of EPG . His work with Dr Steve Isard of the Centre for Speech Technology Research (CSTR) at the University of Edinburgh, on a Carsten's EMA AG-100 system, formed the heart of an integrated EPG-EMA system, producing the Mocha-TIMIT corpus which underpins impact in speech technology.
  • Dr Nigel Hewlett (retired 2010, with a further period of ESRC-funded research 2012-2013) initiated ultrasound-based articulatory research in 2001. QMU-internal SRIF-funding let us research ultrasound data capture and analysis, probe stabilisation and integration with EPG.
  • SCOBBIE secured more SRIF-funding (2005) to commission a high-speed system from Articulate Instruments Ltd, based around Ultrasonix hardware. The resulting two ultrasound tongue imaging "UTI" systems work on entirely different principles. The video system is conventional while the Ultrasonix is a high-speed system incorporating a number of innovations: audio-visual synchronisation hardware at its core, storage of raw scan data (one scan/frame), compatibility with EPG and video camera input (eg for lip analysis). SCOBBIE's joint research with Wrench is cited here as underpinning research in terms of impact.
  • An EPSRC grant, in collaboration with Professor Alice Turk (Edinburgh) and Professor Steve Renals (CSTR) with SCOBBIE and LICKLEY, recently set up the world's first dual-EMA system based on Carstens AG-500, also combining it with EPG. QMU commissioned an EMA-analysis module to integrate into AAA software from Articulate Instruments Ltd.
  • QMU staff (SCOBBIE, LICKLEY) and post-doctoral fellows and PIs (LAWSON, ZHARKOVA, CLELAND, S.SCHAEFFLER) have continued to secure funding to undertake basic and applied articulatory research, resulting in continuing research and development of the uses of the AAA multi-channel modular system. This methodological, descriptive and theoretical linguistic research is disseminated to academic audiences, with elements being exploited by Articulate Instruments Ltd. to improve software and generate financial impact. During the more recent development of these technologies, QMU staff have identified linguistic research goals and contributed to developments in areas such as spine averaging, headsets, clinical support functions and image processing (in addition to numerous external innovations by the company itself which are not being cited here as underpinning research).
  • In 2012, as part of ULTRAX (EPSRC-funded project to SCOBBIE and CLELAND, and Renals and Richmond at CSTR) MRI image analysis was included within AAA as an add-on module.

References to the research

Bold authors were QMU staff undertaking underpinning research. WRENCH'S research 2003-13 is underpinning since he was "category C" for the entire period (cf. QMU's RAE2008 Linguistics return). He is now on staff again (2013), with a professorial role. Evidence of Quality: [1 4 5 6] are peer reviewed; [5 6] stem from EPSRC-funded projects.

[1] Jones, W and Hardcastle, W J (1995) New developments in EPG3 Software. European Journal of Language and Communication Disorders, 30 (2) 183-192. http://eresearch.qmu.ac.uk/2962/

 
 

[2] Scobbie, JM, Wrench, AA and van der Linden, M (2008) Head-Probe stabilisation in ultrasound tongue imaging using a headset to permit natural head movement. Proceedings of the 8th ISSP. 373-376. http://eresearch.qmu.ac.uk/1099/

[3] Wrench, AA and Scobbie, JM (2008) High-speed cineloop ultrasound vs. video Ultrasound Tongue Imaging: Comparison of front and back lingual gesture location and relative timing. Proceedings of the 8th ISSP. 57-60. http://eresearch.qmu.ac.uk/2012/

[4] Wrench, AA and Scobbie, JM (2011) Very high frame rate ultrasound tongue imaging. Proceedings of the 9th ISSP. 155-162. http://eresearch.qmu.ac.uk/2505/

[5] Cleland, J, Wrench, AA, Scobbie, JM and Semple, S (2011) Comparing articulatory images: An MRI / Ultrasound Tongue Image database. Proceedings of the 9th ISSP. 163-170. http://eresearch.qmu.ac.uk/2477/

[6] Geng, CC, Turk, A, Scobbie, JM, Macmartin, C, Hoole, P, Richmond, K, Wrench, AA, Pouplier, M, Bard, EG, Campbell, Z, Dickie, C, Dubourg, E, Hardcastle, WJ, Kainada, E, King, S, Lickley, R, Nakai, S, Renals, S, White, K and Wiegand, R (2013) Recording speech articulation in dialogue: Evaluating a synchronized double Electromagnetic Articulography setup. Journal of Phonetics 41(6) 421-431. http://eresearch.qmu.ac.uk/3205/

 
 
 
 

Details of the impact

1. Context

The spin out of Articulate Instruments Ltd. from QMU in September 2003 was an impact-oriented strategic move to facilitate the eventual uptake of EPG in healthcare. The insight was that independence from QMU as an HEI was necessary to obtain and retain "CE marking" of products (especially EPG). Without CE marking, EPG cannot be sold, used or supported as a clinical device in the European Union, i.e. for non-research, clinical diagnosis and therapy in the NHS. Long term, impact requires that EPG is not just be a research tool but a certified medical device. CE marking was obtained in 2004. Prior to this process, sales of EPG were only for research purposes (albeit including clinical research), not clinical treatment, which had precluded many routes to impact. The main evidence for impact, along with levels of uptake of products in the census period clinically and internationally, is the ongoing financial health of the spin-out company.

2. Baseline figures

EPG is a well-known technology in academic phonetics. It has existed in a number of formats for several decades. Older sales figures of our unique systems will give some baseline context to current impact. The older system EPG2 (1983-1992) gave rise to the sale of 72 systems (17 to hospitals and clinics for clinical linguistic research). EPG3 (1993-1998) had 52 sales (12 to hospitals etc. for clinical research). From 1998-2003, just 23 EPG3 were sold. The Personal Training Unit (PTU) was a standalone product aimed at therapy use, of which around 30 were sold in the UK and Ireland from its launch in 1998 till 2003. These sales covered basic costs and were handled from Reading University, then QMU.

Articulate Instruments took over in 2003. From 2003-2008 there were 126 customers, 46 (37%) in the UK. Thirty of these were non-academic: 21 in the UK, 9 in Europe and Canada. Turnover was approximately £130k per year. Initially, most customers were buying EPG systems but from 2008 onwards UTI (Ultrasound Tongue Imaging) products have been added and sales have increased. An increased level of sales was seen when Articulate Instruments Ltd. was formed. This enhanced level of impact has been sustained over the census period with further new products added.

Some products, mainly the new EPG "Articulate" palate (see below), were designed by WRENCH alone. EPG palates exist in tandem with EPG analysis systems. We will note EPG palate sales separately below, which are largely of value to other companies. The new palate sales are an impact of underpinning QMU research into EPG systems and of WRENCH's palate research.

Number and Distribution of Sales, Turnover, and Profitability 2008-2013

Figures are from audited accounts and internal financial returns of the company. Average turnover was ~£109k in the census period. The company is profitable and sustainable and is supported through sales of goods and services. There have been 98 new customers — mostly universities — across 21 countries in addition to the UK (Fig 1) since January 1, 2008. There have been 27 non-academic customers — mostly hospitals — either from the UK (14) or Europe (13). The products are not licensed for clinical use in the USA at this time for reasons of cost of the licensing process.

Year end (July 31st) Turnover (£)
2008 (ie from 01.08.07) 125,856
2009 126,729
2010 55,863
2011 77,885
2012 105,491
2013 (unaudited) 160,381
total 652,205

Table 1. Financial summary of turnover

Figure 1. Number (and %) of UK customers and overseas customers by
        continent (n=98)
Figure 1. Number (and %) of UK customers and overseas customers by continent (n=98)

4. Effect on the UK economy via expenditure

The company's receipts have supported WRENCH since 2003 and through the impact census period (i.e. he received no salary from QMU). The company also supports SMEs through contracts to supply parts and services. This flow-through to the wider economy differentially favours high-technology SMEs, such as a local Musselburgh company ZOT, and freelance software programmers. The customisable ultrasonic system underlying the high-speed system is manufactured by Ultrasonix from Vancouver, Canada. Thus, the main related businesses are:

  • ZOT for circuit boards, electronics and containment boxes for EPG.
  • ZOT for shaped and prepared aluminium components for UTI headset.
  • Local freelance software programmer.
  • EPG palate manufacturers, supported directly by their own customers.

5. EPG palates

These are a necessary part of EPG use — they are custom-built from a personal palate mould and are unique for each speaker using EPG. In addition to the traditional "Reading" palate, designed at Reading University, there is a new "Articulate" design arising from research by WRENCH. We could cite sales of all palates for use with EPG as part of the financial impact, since the underpinning research was into a new generation of EPG, and all EPG use requires palates of one type or the other. From 2008-2013, most were manufactured in Frenchay Hospital, Bristol, by Incidental, Reading, or by individual hospitals or dental clinics, e.g. in Sweden or Japan. We estimate that over 80 of the Reading palates were made during the census period annually, at ~£400 each (~£200k total). Since the 2007 launch, 85 new Articulate palates have been made (~£34k). These figures are additional to the company turnover — customers get palates made by a local dentist and purchase them direct from one of the dental technician companies mentioned or use their own staff time to manufacture them. Articulate Instruments Ltd. merely supplies printed circuits for the Articulate palate (~£8k income in total, included above).

6. Synergistic research/impact activity, clinical activity

The company has contributed in financial terms (£20k) and in staff-time (approx. 3 months) as formal contributions to the ULTRAX EPSRC-funded project. The company has also contributed time and resources to implementing alterations to its software to reflect the developing research needs of its beta-testers in CASL Research Centre. The commissioned work for the ESPF dual-EMA facility contributed an EMA module to the AAA software and synchronisation components. Articulate Instruments and CASL collaborate both for research purposes and to achieve impact. The latter is facilitated by and embodied in activities such as clinician workshops, beta-testing, consultancy, physical collocation, seminars, technical support and applied research.

We secured funding as part of ULTRAX and a QMU PhD bursary to do clinical UTI research. Eight children were treated in the census period, providing invaluable applied research experience, case data for clinician workshops, and ideas for clinically-relevant improvements to AAA. Results were positive: Parent A said of her son's treatment: "using the visual seemed to help [him] position his tongue." Parent B said: "I was initially sceptical — especially when I saw the helmet! His confidence has increased and he is now able to communicate clearly, without anxiety."

7. Leadership and expansion of use of UTI

UTI was used by around a half-dozen laboratories across the world in 2001. QMU and Articulate Instruments have been involved in this community since the 2nd by-invitation-only international Ultrafest at UBC, Vancouver (2004). We know of no laboratories in the UK or Europe using UTI before QMU. The combination of underpinning research, falling costs and the promise of consistent reliable support from a specialist software/hardware company has helped in the expansion of UTI. We believe critical elements in this expansion are the leadership, commercial independence, longevity, and specialist support offered by Articulate Instruments Ltd. The company announced in 2012 that it would co-organise Ultrafest VI in Edinburgh and attracted 88 non-QMU registrations. An associated clinical workshop attracted 17 registrations. In the UK, video-UTI equipment with AAA is now being used in universities in Glasgow, Manchester and Newcastle. AAA software is available in Edinburgh, London, Aberdeen and Strathclyde. High-speed Ultrasonix systems, with associated AAA hardware and software, have been ordered or purchased by universities in Germany, Brazil, England and Italy, with more hardware in the USA. Of the 50 phonetic laboratories we know of worldwide in 2013 using UTI, 25 are using Articulate Instruments hardware and software. Even more institutions - 39 - use the headset.

Company A states that QMU's ultrasound research in collaboration with Articulate Instruments Ltd "has been commercially useful for our company ... The phonetic research done at QMU has solved problems with acoustic synchronization and demonstrates the benefits of high speed ultrasound for linguistic analysis. Our first sale for phonetics research was to QMU, but we have now sold 7 systems of that type of scanner to phonetics research labs in Europe, the UK, and North America, who are following their lead. We look forward to continuing our partnership with QMU and expanding our portfolio into this unique research area." (our emphasis, quoted from a letter on file at QMU, cf. the private list of named corroborators.)

Sources to corroborate the impact

  1. Articulate Instruments Ltd. http://www.articulateinstruments.com/ and at Company's House
  2. Ultrasonix Medical Corp., Canada — http://www.ultrasonix.com/research/clinical/linguistic
  3. Parent A & B testimonials http://www.ultrax-speech.org/links/client-testimonials (ULTRAX)
  4. The Scotsman Newspaper http://www.scotsman.com/news/health/future-face-of-speech- therapy-mapped-out-by-scottish-scientists-1-1821035