Augmentative and Alternate Communication (AAC)
Submitting InstitutionUniversity of Dundee
Unit of AssessmentComputer Science and Informatics
Summary Impact TypeTechnological
Research Subject Area(s)
Information and Computing Sciences: Artificial Intelligence and Image Processing, Information Systems
Psychology and Cognitive Sciences: Cognitive Sciences
Summary of the impact
Augmentative and Alternative Communication (AAC) systems and devices
assist people with little or no functional speech to communicate. Dundee
research on pragmatics and conversational modelling to improve
communication rates and support conversational flow has led to widespread
use in AAC of re-usable phatic conversation (e.g. greetings, responses,
farewells) and visual scenes (rapid access to conversational items). More
recently, the application of natural language generation and sensor-based
data-to-text technology has resulted in the automated generation of jokes
and narratives to assist non-speaking people to engage in lively
conversation. Symbolic and phonetic interfaces have been developed for
children and adults with congenital and language impairments.
Research on conversational-modelling for AAC was started by Arnott and
Newell in Dundee, working with Ph.D. students Alm and Waller. Alm
researched support for non-speaking people in informal dialogue through
the use of conversational pragmatics, and extended this work into
computer-based personal biography including storage and retrieval methods
to manage anecdotal material .
Post 1993, Alm's funded collaboration ("PicTalk" project: Leverhulme,
1994, £23,600) with Todman (Psychology, Dundee) and File (Abertay) further
developed the theme of conversational pragmatics in AAC, with Alm also
contributing further HCI development. The concept of intersecting
perspectives as a mechanism to enable natural incremental movement through
conversational topics was proposed by Todman and Alm  at this time.
Alm, Arnott and Murray (all Computing, Dundee) developed the use of
scripts and visual scenes in AAC ,,. Such use of scripts with the
graphical visual scenes supporting them were novel contributions to AAC.
Concepts from conversational pragmatics and script-based systems have
since influenced most commercial AAC products in the form of reusable
utterances and visual scenes.
Transatlantic collaboration between Todman, Alm (Dundee), File (Abertay),
Higginbotham (SUNY, USA) and Enkidu Research Inc. (USA) resulted in the
CONTACT project ("A whole utterance-based communication aid for people
without speech: An office workplace implementation", EPSRC GR/R51353/01,
£132,254 pre-FEC, 2001-2004), merging outcomes of their research to
produce a prototype AAC system for use in the office workplace.
Waller and Newell (both Computing, Dundee) established the need for AAC
to support a range of conversational modes, including predictable (chat,
needs-based talk), formulaic (script-based talk, jokes), repeatable
(conversational narrative) and novel (new vocabulary) conversation .
Collaboration with Ritchie (Computing, Aberdeen) and Pain (Informatics,
Edinburgh) resulted in a joke generating system, STANDUP , for children
with speech and language impairment .
Based on a social construct of conversation, the PROSE system supported
interactive storytelling, allowing users to control the narration of a
story instead of delivering a monologue . Waller, Black (Computing,
Dundee) and Reiter (Computing, Aberdeen) addressed the issue of automatic
experiential data input by using sensor-based data-to-text technology to
generate narrative texts for children with speech and language impairment
,. Environmental sensor data (location, people, objects, voice
recordings) were transformed into narrative text using data-to-text
technology . Working with Grove (City University) on narrative use with
people with speech and language impairments, symbolic narrative interface
scaffolds have been designed by Waller to allow disabled children to
narrate and embellish their experiences.
Key Dundee Computing Researchers with principal years of involvement
since 1993: Alm (1993-present), Arnott (1993-2005, 2010-present), Murray
(1994-2000), Newell (1993-2005), Waller (1993-present).
References to the research
(three most significant marked *)
 Computer-assisted conversation for non-vocal people using pre-stored
texts, N. Alm and J. L. Arnott, IEEE Transactions on Systems, Man and
Cybernetics: Part C, Vol.28, No.3, August 1998, pp. 318-328.
 *Modelling conversational pragmatics in communication aids, J. Todman
and N. Alm, Journal of Pragmatics, Vol.35, No.4, April 2003, pp. 523-538.
 A script-based AAC system for transactional interaction, R. Dye, N.
Alm, J. L. Arnott, G. Harper and A. I. Morrison, Natural Language
Engineering, Vol.4, No.1, March 1998, pp. 57-71.
 Towards a narrative-based communication system, A. Waller and A. F.
Newell, European Journal of Disorders of Communication, Vol.32, Issue S3,
1997, pp. 289-306. (Journal continued by: International Journal of
Language and Communication Disorders.)
 *Evaluating the STANDUP Pun Generating Software with Children with
Cerebral Palsy, A. Waller, R. Black, D. O'Mara, H. Pain, G. Ritchie and R.
Manurung, ACM Transactions on Accessible Computing (TACCESS), Vol.1, No.3,
Article 16, February 2009, pp.16:1-16:27.
 *Supporting personal narrative for children with complex
communication needs, R. Black, A. Waller, R. Turner and E. Reiter, ACM
Transactions on Computer-Human Interaction (TOCHI), Vol.19, No.2, Article
15, July 2012, pp.15:1-15:35.
 ALADIN: Advanced Language Device for Interaction", EU TIDE Project TP
1035, £224,360, 1994-1997.
 REACT: Real-time Communication Terminal", EU TIDE Project DE 4207,
 Facilitating language play in non-speaking children through
computer-supported joke construction, EPSRC GR/S15419/01, £364,155
pre-FEC, Dundee share £181,218 pre-FEC, 2003-2007.
 How was School Today ...? Supporting narrative for non-speaking
children, a feasibility study. EPSRC EP/F067151/1 Digital Economy
Feasibility Studies, £102,258, 2008-2009.
 How was School Today ...? in the Wild, EPSRC EP/H022570/1 Digital
Economy Research in the Wild, £156,721, 2010-2011.
Details of the impact
AAC research at Dundee is user-led with strong connections with clinical
practitioners, end-users and industry.
Waller's invited presentations to clinical practitioners, e.g. the 22nd
European Academy of Childhood Disability EACD2010 conference and the 2013
British Society of Paediatric Dentistry Conference, reflect impact on
clinical practice. We have teaching arrangements with special schools and
clinical training departments nationally and internationally. Waller and
her team teach on the communication modules for medical (first, second,
third levels) and dental (final year) students at Dundee University and
biannually on the national community dentistry training CPD course run by
NHS Education Scotland, training over 250 clinicians since 2006. We run
clinical workshops in conjunction with the UK national AAC charity,
Communication Matters, and have delivered clinical training to over 150
teachers and therapists since 2008. In a new initiative with Communication
Matters we have established a forum which brings together researchers,
practitioners and AAC companies in order to reduce the time taken to
transfer technology research into the commercial sphere — "collaboration
with Dundee's School of Computing has allowed us to develop and enhance
the channels through which research is articulated to clinicians, research
and design technologists employed within the communication technology
(AAC) sector and then end users." [R1]
The STANDUP and "How was School Today?" projects, funded by EPSRC and
Digital Economy, have been well received internationally and has resulted
in collaboration with leading special education schools worldwide [R2].
Our work has resulted in schools adopting a narrative approach to
intervention with speech impaired children and we have established a story
telling group in an adult care centre. Our collaboration with Capability
Scotland has contributed to their two schools being awarded the highest
rating in recent HMI inspection: "Your department's continuing work with
our schools in terms of augmentative and alternative communication
assisted us in achieving the glowing report from Education Scotland in
April 2013 which commended the leadership team at Stanmore House School
for their commitment to the use of up to date technology. Your input is
also recognised in our services where, for example, augmentative and
alternative communication is used to support interactive customer led
reviews and the development of personal support plans. By providing the
necessary tools and using a mix of high tech communication aids,
individuals are empowered and supported to take control and make informed
life choices." [R3]
Working in school has resulted in identifying the need to support
literacy learning for nonspeaking children. Supported by Capability
Scotland, the PhonicStick™ has been evaluated by clinical departments in
Manchester, Sweden, Ireland and South Africa. It provides a unique way for
nonspeaking individuals to `play' with sounds and blend them into words.
Capability Scotland funded this fundamental research and "recognise, as
you [Waller] do, that whilst enabling speech to communicate basic needs
and wants is important, true communication is far more complex and
involved. That is why the Phonic Stick symbolises more than anything else
the benefit of our long partnership. It goes beyond mere symbols or words
to enable the formation of language and we were therefore proud to sponsor
its development." [R3] The Dundee City Education Department are supporting
the product development of the PhonicStick™ as a teaching tool because of
its "potential to transform literacy opportunities for some of our most
disadvantaged young people". [R4]
Waller has established a unique AAC usability review consultancy
comprising adults who use AAC. Adults with severe physical and speech
impairments volunteer their expertise in using AAC to support the
development and evaluation of assistive technologies, undertaking
commissions from industry to support the engineering of new devices;
clients include top AAC developers. We have developed innovative methods
for involving people with severe speech and physical disabilities in the
design of assistive technology by adapting techniques to enable
stakeholders of differing ages and abilities to engage in design and have
developed recommendations for ISO design guidelines.
Close links with international AAC developers have resulted in effective
technology transfer. We work with leading AAC companies (e.g. DynaVox,
Toby Churchill Limited (TCL), Tobii Technology AB) to commercialise our
IP. A patent has been registered for the PhonicStick®, a
joystick-controlled phoneme-based word generation tool for children with
language impairments, and the IP licensed to TCL in 2012. Waller's work on
narrative, licensed to Don Johnston Inc. of Illinois in 1992, has provided
much of the theoretical underpinning to the Improv™ AAC system, released
in 2012. Don Johnston Inc. state that: "work at University of Dundee is
instrumental in pushing the boundaries of AAC by taking a conversational
discourse approach to supporting aided communication." [R5]
CONTACT project partner Enkidu Research Inc. merged with DynaVox in 2004.
Conversation modelling and pragmatics techniques developed at Dundee were
commercialised in the InterAACt language framework from DynaVox
Mayer-Johnson of Pittsburgh, the largest AAC company in the world with 395
full-time employees and net sales of over $97 million (fiscal year to June
2012). Speech generating (i.e. AAC) products represented approximately 84%
of these sales. "InterAACt is the exclusive language framework used on all
DynaVox devices. It allows individuals with significant communication
needs to successfully communicate, develop high-level language skills and
express themselves, in everyday activities." [R6]
DynaVox products are promoted in the international AAC marketplace
(DynaVox lists 25 countries) and are prescribed extensively by clinicians
(e.g. speech and language therapists) for their clients. The InterAACt
framework therefore reaches a wide client base of people who experience
improved AAC support during conversation with others. Other beneficiaries
are family members, carers and friends of the people who use InterAACt, as
they have more fulfilling conversations with the InterAACt user, and
clinicians and therapists who work with non-speaking and communication
impaired people. The AAC industry benefits through a significant advance
in augmentative communication technology coming to market. Concepts from
conversational pragmatics and script-based systems have influenced most
modern AAC systems in the form of re-usable utterances and visual scenes.
DynaVox Mayer-Johnson state that: "Dundee is one of a small number of
centers whose ideas have defined the field of AAC as we know it. Your
ideas have .... greatly improved the lives of tens of thousands of people.
The research done at Dundee has directly generated significant economic
value for many organizations in our industry. ... DynaVox's InterAACt
language framework makes extensive use of techniques developed at the
University of Dundee and which we license from you. This technology
provides our customers with the opportunity for greater communication
rates while retaining the crucial flexibility and context sensitivity
needed so they can truly say what they want to say." [R7]
The projects have also experienced national and international news
exposure [R8]. STANDUP has been downloaded more than 1000 times from its
project webpage. "How was School today...?" [R9] won the 2010 TES Schools
Award for Outstanding ICT Learning Initiative of the Year for project
partner Capability Scotland (Corseford School): the "How was School
today...?" project "signalled how assistive technology could really open
up the world of communication to those who use our [Capability Scotland's]
services. We were particularly delighted that it won the only Scottish
award in the 2010 Times Educational Supplement's Awards for the School and
Sources to corroborate the impact
[R1] Factual Statement from Communication Matters, the UK Chapter of the
International Society for Augmentative and Alternative Communication
(ISAAC), September 2012.
[R2] Factual Statement from Lead Speech-Language Therapists, The
Children's Institute: The Day Institute, Pittsburgh, PA, USA, February
[R3] Factual Statement from Chief Executive, Capability Scotland,
Edinburgh, October 2013.
[R4] Factual Statement from Head of Education (Secondary and Support for
Learners), Dundee City Education Department, May 2013.
[R5] Factual Statement from President, Don Johnston, Volo, IL, USA,
[R6] DynaVox website (http://uk.dynavoxtech.com/interaact/).
[R7] Factual Statement from DynaVox Mayer-Johnson, Pittsburgh, PA, USA,
[R8] "New software helps children speak". BBC TV news report, GMT Friday
5 June 2009. Shown throughout the day and on 10:00pm news. Corresponding
BBC web site report: http://news.bbc.co.uk/1/hi/health/8084422.stm.
Local copy (video file, mp4): http://ref2014.computing.dundee.ac.uk/aac/[R8]_BBC_News.mp4.
[R9] "How was my school day? Now disabled pupils can tell the story". The
Times, 5 June 2009. Local copy available at: http://ref2014.computing.dundee.ac.uk/aac/[R10]_The_Times.pdf
[R10] The Times Educational Supplement website: Winner of the
"Outstanding ICT Learning Initiative of the Year" (http://www.tes.co.uk/article.aspx?storyCode=6047969&s_cid=winners2010)