Summary of the impact

Since 2008, drawing on the findings of the EU-funded project Digital TV for ALL (2008-2011) and on his book Subtitling through Speech Recognition: Respeaking (2011), Dr Pablo Romero-Fresco has researched and promoted a new technique to guarantee that deaf and hard-of-hearing people around the world can have access to live programmes and public events through speech recognition-based subtitles. The impact achieved as a result of this research is three-fold:

1. The introduction of live subtitles in countries (such as Spain and South Africa) and contexts (art galleries, university classrooms) where they did not exist before.
2. The improvement of the quality of live subtitles and the collaboration with public bodies and policy-makers such as the United Nations and Ofcom in countries and contexts where live subtitles already exist.
3. The engagement with the viewers, increasing their awareness of live subtitling and improving their access to audiovisual media.

Underpinning research

Although the professional practice of audiovisual translation (subtitling and dubbing) and media accessibility (subtitling for the deaf, audiodescription for the blind) has existed since 1930 and 1980 respectively, academic research in these areas started as late as the 1990s, and it has only begun to thrive in the past ten years. Respeaking, a technique whereby a subtitler paraphrases the speech of a live programme to speech-recognition software in order to produce live subtitles for deaf and hard of hearing viewers, is considered as a new modality of media accessibility. It was first used by the BBC in 2001, but had not been the subject of academic research until 2008. Since 2008 Romero-Fresco (Lecturer 2008-2010, Senior Lecturer 2010-2012, Reader 2012-2013) has been conducting research to promote and improve the provision of live subtitles through respeaking for deaf and hard-of-hearing people around the world.

As project manager of the subtitling part of Digital TV for All (2008-2011), a project on media accessibility funded by the EU Commission, Romero-Fresco coordinated the work of researchers from the UK, Spain, Italy, Germany, Poland, Denmark and France on the quality of subtitling in Europe. Romero-Fresco was also the researcher in charge of conducting the DTV4ALL study in the UK, which provided the findings for this case study. Romero-Fresco's research contribution to the project dealt with live subtitling and resulted in the monograph Subtitling through Speech Recognition: Respeaking (2011), the first book ever written on the subject. The first part of the book set the terminology for the new respeaking discipline. In the case of the Spanish language, it coined the Spanish term for respeaking (rehablado), which was officially accepted by the Royal Spanish Academy of the Language in 2011 and has now become the standard term used by the industry, the media and the public to refer to this discipline. This book and the article "Respeaking in Translator Training Curricula" (2012) also outlined the first theoretical and practical training programme for respeakers, which has been used by universities, media companies and broadcasters around the world.

Drawing on the data obtained by Romero-Fresco in the UK as part of the DTV4ALL Project, his 2011 book also presented the first reception study ever made on live subtitling, providing data on the viewers' preferences, comprehension and physical perception of live subtitles. The viewers' preferences were obtained through a large survey conducted in collaboration with the Royal National Institute for the Deaf. The results of the survey showed the need to raise awareness about respeaking in the UK, since many viewers were not aware of the existence of live subtitles and others had unrealistic expectations about them because they were not familiar with the complexity involved in respeaking. The study on comprehension and perception of live subtitles in the UK applied, for the first time ever, eye-tracking technology to determine how viewers process subtitles in live programmes. The findings of this study suggest that the word-for-word display mode of live subtitles prevents viewers from achieving an optimum comprehension of the TV programmes.

Finally, the last part of Romero-Fresco's 2011 book introduces the first model with metrics to analyse the quality of live subtitles. The model accounts for the different parameters that affect the quality of live subtitles and is designed to provide feedback for the companies (about the accuracy of their subtitles), the respeakers (about what aspects they need improve in order to obtain better results) and the viewers (about how their comprehension may be affected by the lack of accuracy in the subtitles).

References to the research

Quality indicators:

Funded by the EU-Comission (€2,925,300), the DTV4ALL project was completed between 2008 and 2011.

The monograph Subtitling through Speech Recognition: Respeaking (2011) was published by St. Jerome, one of the leading publishers in the field of translation, and has been praised as a "prime example of solid research and scholarship", "brave and pioneering work" and "not only the first but probably the ultimate" publication on live subtitling.

Details of the impact

The introduction of respeaking

The training programme proposed by Romero-Fresco in his book has been adopted by the universities of Barcelona, Vigo (Spain), Vienna (Austria) and Antwerp (Belgium) to start postgraduate modules on respeaking. This training programme has also been adopted by media companies in different countries around the world. In 2009, the Spanish company Mundovisión set up a respeaking team using Romero-Fresco's training methodology to provide, for the first time ever, live subtitles through respeaking in Spain for the public channel TVE. Following two specialised workshops delivered by Romero-Fresco and based on the above-mentioned research, the Italian media companies Deluxe and Subt-ti started providing live subtitles through respeaking in 2011, as did the British company TVT, which is currently using the methodology outlined in the book to provide live subtitles for BBC World in the United States.

The impact on the viewers and the industry

Romero-Fresco's study on the viewers' reception of live subtitles in the UK, also included in the book, has had significant impact on both the viewers and professional practice in the field. The results of the survey on viewers' preferences, which showed the need to raise awareness about the existence of respeaking in the UK, persuaded the BBC to prepare in collaboration with Romero-Fresco a special item on live subtitling for its BBC2 programme See Hear¹. Broadcast on 3 February 2011 and seen by 508,833 viewers, the programme included an interview with Romero- Fresco where he explained the main findings of his research. Since then, it has been hosted on Youtube with a total of 20,267 views as of 15 July 2013. Some of the comments received from the viewers are "I will certainly be less impatient with live subtitles now that I know how they are made" and "This is a fascinating documentary, a great way of getting across this service to so many people who are hard of hearing and are not aware of live subtitles. Also, it will bring understanding to those whose hearing is so far unimpaired". In Spain, an interview with Romero-Fresco about his research was presented in a news item by Canal 9's programme Gente on 25 January 2011². This was the first news item about respeaking ever broadcast in Spain and was seen by 81,944 viewers.

The second part of the reception study on live subtitles, carried out as part of the DTV4ALL project and presented in Romero-Fresco's 2011 book, led to a significant change in professional practice that has reached not only broadcasters and subtitling companies, but also millions of viewers. The results of the eye-tracking and comprehension tests showed that the word-for-word display mode of live subtitles often used in Switzerland and France prevents viewers from achieving an optimum comprehension of TV programmes. This problem can however be avoided if the subtitles are displayed in blocks. Following the recommendations from this study, Swiss TXT (the leading Swiss subtitling company) changed their subtitling policy to introduce block subtitles in 2011. From then, the live programmes broadcast by the Swiss public channels SF1, SF2 and SF Info (in German), TSR1 and TSR2 (in French) and RSI 1 and 2 (in Italian), which are seen by an average of between 1,564,138 and 4,066,758 viewers every day, are subtitled in blocks. In 2012, a workshop given by Romero-Fresco in Geneva also persuaded the leading French subtitling company MFP to stop displaying their subtitles in scrolling mode. As a result, the live programmes on the main French public channels (France 2, France 3 and France 5), with shares of 3.6%-14.4% (between 837,000 and 3,100,000 viewers per day), are now subtitled in blocks.

The impact on the regulators

Romero-Fresco's findings about the quality of live subtitling were presented on 19 April 2011 in the inaugural meeting of the United Nation's first Focus Group on Audiovisual Media Accessibility, of which he is a member. In the UK, the main deaf associations (Action on Hearing Loss, Tag, Hearing Concern and Link) used these findings in 2012 to demand the public regulator Ofcom to tackle the issue of quality in live subtitling. Until then, Ofcom had only been concerned with the quantity of live subtitling, mainly by establishing quotas for the different British channels. Following the pressure by the user associations, Ofcom agreed to engage with the quality of live subtitles and asked Romero-Fresco for a model to assess the accuracy of live subtitles. After months of consultation with broadcasters and service providers, Ofcom have now issued the first official statement about quality in live subtitling³. The public regulator urges broadcasters to inform their practice with the academic research carried out by Romero-Fresco on respeaking and presents his NER model (introduced in his 2011 book) as an example of how to assess the quality of live subtitles.

In Spain, the public regulator AENOR adopted in May 2011 the NER model and included it in the official Spanish guidelines on subtitling⁴. Every subtitling company in Spain is required to use it for quality control and training purposes. The model has also been adopted by the leading public broadcasters in Italy (RAI), Germany (ARD), Belgium (VRT) and Switzerland (SRG SSR). The application of the model ensures that the subtitles produced by these broadcasters and companies have a minimum of 98% accuracy, thus providing the hearing-impaired viewers with optimum access to the TV programmes they are watching.

Respeaking in new contexts

The research carried out by Romero-Fresco has also enabled the provision of live subtitles in countries and contexts where they did not exist before, such as university classrooms in South Africa and live events, telephone conversations and secondary school classrooms in the UK.

Although 9 of the 11 official languages in South Africa are African, most universities deliver their courses in English. The majority of South African students are thus at a disadvantage, having to follow classes in their third, fourth and sometimes fifth language. A visit to North-West University (Vaal Triangle) in October 2011 enabled Romero-Fresco to provide respeaking training to the researchers in the Language Department. The aim was to use respeaking as a tool to subtitle classes live into English so that non-native students could have live written input as well as the audio input from the lecturers. The first experiments proved extremely successful and promising with a view to extending this technique to other universities around the country.

In the UK, until 2009 respeaking had only been used on TV. In collaboration with Romero-Fresco and using the initial findings of the DTV4ALL project, the subtitling company Stagetext piloted in March 2009 the first experience of respoken subtitles for a live event in the UK (a talk at the National Gallery). Since then, Stagetext has been providing regular access for deaf and hard-of- hearing people to 100 live events every year at museums and theatres all around the country. Finally, Romero-Fresco's research is also being applied in the UK by Bee Communications and Ai- Live. Bee Communications are using Romero-Fresco's training programme and the NER model for the pioneering application of respeaking to telephone conversations. The goal is to enable people with hearing loss to read on their phone screens what their interlocutor is saying. In the UK, this has normally been done by a remote typist, but never by a respeaker. As for Ai-Live, they are working with Romero-Fresco and using both his training programme and the NER model to introduce respeaking in UK secondary schools. The aim in this case is to use respeaking to provide a live transcription of the teacher's words in the classroom for the benefit of students with hearing loss or learning difficulties.

Ongoing

With regard to ongoing developments, Romero-Fresco is now working as a consultant in the EU- funded project SAVAS, which aims to develop speech recognition software in languages that still do not have this technology. The project will be using the NER model to develop new software in Portuguese and in minority languages such as Basque and Galician, in order to ensure that the hearing-impaired viewers from these countries and regions can have access to live programmes and public events through respeaking.

Sources to corroborate the impact

Testimonial evidence:

1. Operations Manager at the British company TVT.
2. Head of the Italian company Sub-ti.
3. Head of the Swiss company Swiss TXT.
4. Head of French company MFP.
5. Head of the British company Stagetext.

Other sources of evidence:

¹ BBC2 programme on respeaking (See Hear): http://www.youtube.com/watch?v=u2K9-JPIPjg

² Canal 9's programme on respeaking in Spanish (Gente):
http://www.youtube.com/watch?v=zctEC3VOwng

³ Ofcom's official report on the quality of live subtitling:
http://stakeholders.ofcom.org.uk/binaries/consultations/subtitling/summary/subtitling.pdf

⁴ Official Spanish guidelines on subtitling published by AENOR:
http://www.aenor.es/aenor/normas/normas/fichanorma.asp?tipo=N&codigo=N0049426

⁵ Ai-media media release confirming ongoing use of the NER model:
http://www.ai-media.tv/news/quality_standard_captioning