Articulate Instruments - visualising speech
Submitting Institution
Queen Margaret University EdinburghUnit of Assessment
Modern Languages and LinguisticsSummary Impact Type
TechnologicalResearch Subject Area(s)
Information and Computing Sciences: Artificial Intelligence and Image Processing, Computer Software, Information Systems
Summary of the impact
Articulate Instruments Ltd. was founded in 2003 as a research, design,
manufacturing and consulting company for users of phonetic
instrumentation. It invents, designs, markets and supports instrumental
technologies for normative and clinical speech science and for the
diagnosis and treatment of speech disorders. Products include electronic
systems, headsets, software, and methodologies, underpinned by QMU
research. Clinical use of relevant products as medical devices requires
"CE marking" to prove on-going safety and support, first achieved in
2004.
Impact relates primarily to the company's on-going financial
health and its non-academic customer base. In its first 10
years, turnover averaged ~£120k, with over 200 customers internationally,
of whom more than 50 were non-academic.
Underpinning research
The underpinning research includes basic speech science, electrical and
software engineering, articulatory phonetics and informatics. Over two
decades, we have researched novel and unique combinations of phonetic
instrumentation and methodology for both clinical and non-clinical topics.
There has been a focus on enhancing reliability, usability and
flexibility, with a view that its use should be extended to non-academic
users, particularly in clinical contexts. Below, names that are
CAPITALISED indicate staff with outputs included in this REF UOA.
- WRENCH spent two years developing a MS-Windows version of
Electropalatography (EPG) at QMU (2001-2003) then left to set up
Articulate Instruments Ltd in September 2003 as a spin-out company.
This was part of QMU's strategy to ensure the long term viability of EPG
as a "clinical device" (see below). He had been a post-doctoral
researcher at QMU from 1995 to 2003 on the QMU/Edinburgh University
Speech Production Facility (ESPF) and other clinically-oriented speech
technology projects. Strategic internal funding (1996-1997) and the
EPSRC-funded project on speech recognition using articulatory data
(1998-2001) were used to research and develop implementations of EPG in
MS-Windows, replacing the obsolescent DOS-Windows version. EPG became
the core of both a clinical system (Articulate Assistant) and then a
multi-channel research system (Articulate Assistant Advanced, "AAA").
The company has been an independent entity, with WRENCH as its sole
direct employee/director since 2003. However, close ties have been
retained with QMU through collaborative research and shared strategic
goals with the former Speech Science Research Centre (SSRC), now
Clinical Audiology, Speech and Language Research Centre (CASL,
2011-2013).
- Professor Bill Hardcastle was appointed by QMU (1990-2009). This
initiated more than 20 years of continuous research and development into
articulatory instrumentation. From 1996, Hardcastle secured the funding
to develop the MS-Windows version of EPG . His work with Dr Steve Isard
of the Centre for Speech Technology Research (CSTR) at the University of
Edinburgh, on a Carsten's EMA AG-100 system, formed the heart of an
integrated EPG-EMA system, producing the Mocha-TIMIT corpus which
underpins impact in speech technology.
- Dr Nigel Hewlett (retired 2010, with a further period of ESRC-funded
research 2012-2013) initiated ultrasound-based articulatory research in
2001. QMU-internal SRIF-funding let us research ultrasound data capture
and analysis, probe stabilisation and integration with EPG.
- SCOBBIE secured more SRIF-funding (2005) to commission a high-speed
system from Articulate Instruments Ltd, based around Ultrasonix
hardware. The resulting two ultrasound tongue imaging "UTI" systems work
on entirely different principles. The video system is conventional while
the Ultrasonix is a high-speed system incorporating a number of
innovations: audio-visual synchronisation hardware at its core, storage
of raw scan data (one scan/frame), compatibility with EPG and video
camera input (eg for lip analysis). SCOBBIE's joint research with Wrench
is cited here as underpinning research in terms of impact.
- An EPSRC grant, in collaboration with Professor Alice Turk (Edinburgh)
and Professor Steve Renals (CSTR) with SCOBBIE and LICKLEY, recently set
up the world's first dual-EMA system based on Carstens AG-500, also
combining it with EPG. QMU commissioned an EMA-analysis module to
integrate into AAA software from Articulate Instruments Ltd.
- QMU staff (SCOBBIE, LICKLEY) and post-doctoral fellows and PIs
(LAWSON, ZHARKOVA, CLELAND, S.SCHAEFFLER) have continued to secure
funding to undertake basic and applied articulatory research, resulting
in continuing research and development of the uses of the AAA
multi-channel modular system. This methodological, descriptive and
theoretical linguistic research is disseminated to academic audiences,
with elements being exploited by Articulate Instruments Ltd. to improve
software and generate financial impact. During the more recent
development of these technologies, QMU staff have identified linguistic
research goals and contributed to developments in areas such as spine
averaging, headsets, clinical support functions and image processing (in
addition to numerous external innovations by the company itself which
are not being cited here as underpinning research).
- In 2012, as part of ULTRAX (EPSRC-funded project to SCOBBIE and
CLELAND, and Renals and Richmond at CSTR) MRI image analysis was
included within AAA as an add-on module.
References to the research
Bold authors were QMU staff undertaking underpinning research.
WRENCH'S research 2003-13 is underpinning since he was "category C" for
the entire period (cf. QMU's RAE2008 Linguistics return). He is now on
staff again (2013), with a professorial role. Evidence of Quality: [1 4 5
6] are peer reviewed; [5 6] stem from EPSRC-funded projects.
[1] Jones, W and Hardcastle, W J (1995) New developments in EPG3
Software. European Journal of Language and Communication Disorders,
30 (2) 183-192. http://eresearch.qmu.ac.uk/2962/
[2] Scobbie, JM, Wrench, AA and van der Linden, M (2008)
Head-Probe stabilisation in ultrasound tongue imaging using a headset to
permit natural head movement. Proceedings of the 8th ISSP.
373-376. http://eresearch.qmu.ac.uk/1099/
[3] Wrench, AA and Scobbie, JM (2008) High-speed cineloop
ultrasound vs. video Ultrasound Tongue Imaging: Comparison of front and
back lingual gesture location and relative timing. Proceedings of the
8th ISSP. 57-60. http://eresearch.qmu.ac.uk/2012/
[5] Cleland, J, Wrench, AA, Scobbie, JM and
Semple, S (2011) Comparing articulatory images: An MRI / Ultrasound Tongue
Image database. Proceedings of the 9th ISSP. 163-170. http://eresearch.qmu.ac.uk/2477/
[6] Geng, CC, Turk, A, Scobbie, JM, Macmartin, C, Hoole, P,
Richmond, K, Wrench, AA, Pouplier, M, Bard, EG, Campbell, Z,
Dickie, C, Dubourg, E, Hardcastle, WJ, Kainada, E, King, S, Lickley,
R, Nakai, S, Renals, S, White, K and Wiegand, R (2013) Recording speech
articulation in dialogue: Evaluating a synchronized double Electromagnetic
Articulography setup. Journal of Phonetics 41(6) 421-431.
http://eresearch.qmu.ac.uk/3205/
Details of the impact
1. Context
The spin out of Articulate Instruments Ltd. from QMU in September 2003
was an impact-oriented strategic move to facilitate the eventual uptake of
EPG in healthcare. The insight was that independence from QMU as an HEI
was necessary to obtain and retain "CE marking" of products (especially
EPG). Without CE marking, EPG cannot be sold, used or supported as a
clinical device in the European Union, i.e. for non-research, clinical
diagnosis and therapy in the NHS. Long term, impact requires that EPG is
not just be a research tool but a certified medical device. CE marking was
obtained in 2004. Prior to this process, sales of EPG were only for
research purposes (albeit including clinical research), not clinical
treatment, which had precluded many routes to impact. The main evidence
for impact, along with levels of uptake of products in the census period
clinically and internationally, is the ongoing financial health of the
spin-out company.
2. Baseline figures
EPG is a well-known technology in academic phonetics. It has existed in a
number of formats for several decades. Older sales figures of our unique
systems will give some baseline context to current impact. The older
system EPG2 (1983-1992) gave rise to the sale of 72 systems (17 to
hospitals and clinics for clinical linguistic research). EPG3 (1993-1998)
had 52 sales (12 to hospitals etc. for clinical research). From 1998-2003,
just 23 EPG3 were sold. The Personal Training Unit (PTU) was a standalone
product aimed at therapy use, of which around 30 were sold in the UK and
Ireland from its launch in 1998 till 2003. These sales covered basic costs
and were handled from Reading University, then QMU.
Articulate Instruments took over in 2003. From 2003-2008 there were 126
customers, 46 (37%) in the UK. Thirty of these were non-academic: 21 in
the UK, 9 in Europe and Canada. Turnover was approximately £130k per year.
Initially, most customers were buying EPG systems but from 2008 onwards
UTI (Ultrasound Tongue Imaging) products have been added and sales have
increased. An increased level of sales was seen when Articulate
Instruments Ltd. was formed. This enhanced level of impact has been
sustained over the census period with further new products added.
Some products, mainly the new EPG "Articulate" palate (see below), were
designed by WRENCH alone. EPG palates exist in tandem with EPG analysis
systems. We will note EPG palate sales separately below, which are largely
of value to other companies. The new palate sales are an impact of
underpinning QMU research into EPG systems and of WRENCH's palate
research.
Number and Distribution of Sales, Turnover, and Profitability
2008-2013
Figures are from audited accounts and internal financial returns of the
company. Average turnover was ~£109k in the census period. The company is
profitable and sustainable and is supported through sales of goods and
services. There have been 98 new customers — mostly universities — across
21 countries in addition to the UK (Fig 1) since January 1, 2008. There
have been 27 non-academic customers — mostly hospitals — either from the
UK (14) or Europe (13). The products are not licensed for clinical use in
the USA at this time for reasons of cost of the licensing process.
Year end (July 31st) |
Turnover (£) |
2008 (ie from 01.08.07) |
125,856 |
2009 |
126,729 |
2010 |
55,863 |
2011 |
77,885 |
2012 |
105,491 |
2013 (unaudited) |
160,381 |
total |
652,205 |
Table 1. Financial summary of turnover
4. Effect on the UK economy via expenditure
The company's receipts have supported WRENCH since 2003 and through the
impact census period (i.e. he received no salary from QMU). The company
also supports SMEs through contracts to supply parts and services. This
flow-through to the wider economy differentially favours high-technology
SMEs, such as a local Musselburgh company ZOT, and freelance software
programmers. The customisable ultrasonic system underlying the high-speed
system is manufactured by Ultrasonix from Vancouver, Canada. Thus, the
main related businesses are:
- ZOT for circuit boards, electronics and containment boxes for EPG.
- ZOT for shaped and prepared aluminium components for UTI headset.
- Local freelance software programmer.
- EPG palate manufacturers, supported directly by their own customers.
5. EPG palates
These are a necessary part of EPG use — they are custom-built from a
personal palate mould and are unique for each speaker using EPG. In
addition to the traditional "Reading" palate, designed at Reading
University, there is a new "Articulate" design arising from research by
WRENCH. We could cite sales of all palates for use with EPG as part of the
financial impact, since the underpinning research was into a new
generation of EPG, and all EPG use requires palates of one type or the
other. From 2008-2013, most were manufactured in Frenchay Hospital,
Bristol, by Incidental, Reading, or by individual hospitals or dental
clinics, e.g. in Sweden or Japan. We estimate that over 80 of the Reading
palates were made during the census period annually, at ~£400 each (~£200k
total). Since the 2007 launch, 85 new Articulate palates have been made
(~£34k). These figures are additional to the company turnover — customers
get palates made by a local dentist and purchase them direct from one of
the dental technician companies mentioned or use their own staff time to
manufacture them. Articulate Instruments Ltd. merely supplies printed
circuits for the Articulate palate (~£8k income in total, included above).
6. Synergistic research/impact activity, clinical activity
The company has contributed in financial terms (£20k) and in staff-time
(approx. 3 months) as formal contributions to the ULTRAX EPSRC-funded
project. The company has also contributed time and resources to
implementing alterations to its software to reflect the developing
research needs of its beta-testers in CASL Research Centre. The
commissioned work for the ESPF dual-EMA facility contributed an EMA
module to the AAA software and synchronisation components. Articulate
Instruments and CASL collaborate both for research purposes and to achieve
impact. The latter is facilitated by and embodied in activities such as
clinician workshops, beta-testing, consultancy, physical collocation,
seminars, technical support and applied research.
We secured funding as part of ULTRAX and a QMU PhD bursary to do clinical
UTI research. Eight children were treated in the census period, providing
invaluable applied research experience, case data for clinician workshops,
and ideas for clinically-relevant improvements to AAA. Results were
positive: Parent A said of her son's treatment: "using the
visual seemed to help [him] position his tongue." Parent B said: "I
was initially sceptical — especially when I saw the helmet! His
confidence has increased and he is now able to communicate clearly,
without anxiety."
7. Leadership and expansion of use of UTI
UTI was used by around a half-dozen laboratories across the world in
2001. QMU and Articulate Instruments have been involved in this community
since the 2nd by-invitation-only international Ultrafest at
UBC, Vancouver (2004). We know of no laboratories in the UK or Europe
using UTI before QMU. The combination of underpinning research, falling
costs and the promise of consistent reliable support from a specialist
software/hardware company has helped in the expansion of UTI. We believe
critical elements in this expansion are the leadership, commercial
independence, longevity, and specialist support offered by Articulate
Instruments Ltd. The company announced in 2012 that it would co-organise
Ultrafest VI in Edinburgh and attracted 88 non-QMU registrations. An
associated clinical workshop attracted 17 registrations. In the UK,
video-UTI equipment with AAA is now being used in universities in Glasgow,
Manchester and Newcastle. AAA software is available in Edinburgh, London,
Aberdeen and Strathclyde. High-speed Ultrasonix systems, with associated
AAA hardware and software, have been ordered or purchased by universities
in Germany, Brazil, England and Italy, with more hardware in the USA. Of
the 50 phonetic laboratories we know of worldwide in 2013 using UTI, 25
are using Articulate Instruments hardware and software. Even more
institutions - 39 - use the headset.
Company A states that QMU's ultrasound research in collaboration with
Articulate Instruments Ltd "has been commercially useful for our
company ... The phonetic research done at QMU has solved
problems with acoustic synchronization and demonstrates the benefits of
high speed ultrasound for linguistic analysis. Our first sale
for phonetics research was to QMU, but we have now sold 7 systems
of that type of scanner to phonetics research labs in Europe, the UK,
and North America, who are following their lead.
We look forward to continuing our partnership with QMU and expanding our
portfolio into this unique research area." (our emphasis, quoted
from a letter on file at QMU, cf. the private list of named
corroborators.)
Sources to corroborate the impact
- Articulate Instruments Ltd. http://www.articulateinstruments.com/
and at Company's House
- Ultrasonix Medical Corp., Canada — http://www.ultrasonix.com/research/clinical/linguistic
- Parent A & B testimonials http://www.ultrax-speech.org/links/client-testimonials
(ULTRAX)
- The Scotsman Newspaper http://www.scotsman.com/news/health/future-face-of-speech-
therapy-mapped-out-by-scottish-scientists-1-1821035