Summary of the impact

Over a decade's research by Introna and Hayes has investigated the biases inherent in automated systems. As part of this research they showed that the design and use of plagiarism detection systems (PDS), used by Higher Education Institutions (HEIs) worldwide, may be unfair due to their embedded values and assumptions. A series of reports, workshops and supporting resources, using these insights, have transformed writing support and teaching practices at no less than 32 HEIs nationally and internationally, affecting lecturers, support staff and student union representatives. As a result, at least 10 HEIs have developed less punitive policy frameworks and taken a more developmental approach, leading to a much fairer treatment of plagiarism cases.

Underpinning research

The underpinning research of this impact case is part of an ongoing research project started more than a decade ago. The essence of this project was to demonstrate that technologies are not neutral machines/tools but are relatively `frozen cultures'. They embody, in their design, certain beliefs, values and interests. Furthermore, technology `use' is not a neutral taking-up of tools but rather a cultural and political issue in which some parties' interests prevail (often at the expense of others). The early part of the programme focused on search engine technology, which demonstrated that internet search engines, and their algorithms, systematically favour certain types of sites and content over others. The project also considered the rise in automatic facial recognition systems and also demonstrated that the design and operation of automatic teller machines (ATMs) systematically excluded the interests of certain users — i.e. that they assumed that the user is able-bodied. This larger programme established the theoretical framework and methodology to study PDS, the focus of this impact case.

The rapid rise in the use of PDS such as Turnitin in HEIs offered the opportunity to examine the political and ethical implications of PDS and how their algorithms work to detect what was assumed to be plagiarism. Work undertaken by Professor Lucas Introna and Dr Niall Hayes at LUMS, funded by the Higher Education Funding Council for England (HEFCE), and has been published and highly cited in top journals. These include Ethics and Behaviour (2005) and Information and Organisation (2011). This research demonstrated that the algorithms that underpin PDS did not detect plagiarism as such but only detected certain types of copied text.

The fundamental problem with PDS is that certain types of copying are detected and other types are not detected. Students who retain a sufficient number of consecutive characters will be detected by the system's algorithm. Often these students will be those who are copying words from a source in the system's database but are constructing their own ideas through a complex patchwork of copied words. There is little intention to cheat in such cases. When there is an intention to cheat, students with limited linguistic ability in English will be readily detected. However, students who have the linguistic ability to copy ideas but who break up the number of consecutive characters copied, for example every fourth word, will not be detected, even though they have based their ideas and their text on a number of sources. Thus the algorithm does not necessarily identify those who set out to cheat. It detects those who retain a sufficient number of consecutive characters for the algorithm to identify it as copied text. Of the ones who are cheating, PDS disproportionately identifies those who lack linguistic ability (non-native speakers) over those who are writing in their first language.

Furthermore, the research found that there were many culturally specific educational reasons why there might be copied text in a student's work. It demonstrated that the use of the technology discriminates against students who come from educational contexts in which there were different expectations of what academic writing involves, i.e. international students. This has had a significant impact on the way HEIs deal with cases of plagiarism, especially in relation to international students.

As part of the project the research team produced a website, based on the plagiarism project, entitled Student Diversity and Academic Writing (SDAW) to disseminate information to a global audience. This detailed their empirical work in Greece, India and China, the aims of the project and initial findings. The Director of Services and Research at the UK Council for International Student Affairs (UKCISA) endorses the website, stating that it `is one of the resources to which I refer enquirers on this topic. It is referenced by the Teaching International Students databank on the Higher Education Academy website (a project which UKCISA co-led).' The team also organised and held a conference entitled `International Students, Academic Writing and Plagiarism' which brought together key stakeholders in this area to debate the topic. Videos and slides from the presentations were made available on the SDAW website.

References to the research

The research has been published in the following books and international, peer reviewed journals:

Grant:

£215,000 HEFCE grant, `Plagiarism, Computers and Values' (Dr Niall Hayes, Professor Lucas Introna and Dr Edgar Whitley, LSE), awarded in 2004. Project activities began in January 2005 and concluded in December 2007. Additional funding for a transferability phase for further work was awarded in 2006, and this work took place during 2008.

Details of the impact

Globally, HEIs have become increasingly concerned with the prevalence of plagiarism in academic writing, and in particular in assessments. Plagiarism was perceived to be more common among international students. One seemingly simple solution to the problem was to check all assessments electronically, using PDS. It was assumed that these systems would subject all students to the same rigorous process of checking, i.e. that they would be fair. The research demonstrated that this was not the case. It revealed that, because of the assumptions embedded in the algorithms, these systems discriminate against certain students. These systems should therefore be used with caution and only as part of a comprehensive framework to deal with plagiarism. A THE article suggested, based on the research, that, `In fact, the reason such students are branded as cheats is that universities have flawed ideas about plagiarism. Plagiarism is not a simple phenomenon. It is not a straightforward choice between cheating and not cheating. A number of complex conditions shape the writing practices of students.' These insights were disseminated through 20 workshops and presentations at HEIs and at HE policy forums in the UK, Europe, China and the USA.

Local impact:

Sessions, incorporating the research, have been held for several years, training academic staff at Lancaster University on the Associate Teacher Programme (ATP) and Certificate of Academic Practice (CAP), courses accredited by the Higher Education Academy. Also, student academic writing practices have been influenced through workshops and working closely with faculty student learning advisors, based on the recommendations from this research.

Impact on policy:

The work has been influential in shaping policy at various national and international HEIs such as Nottingham Trent University, University of Northumbria, University of Sydney and Simon
Fraser University. Southern Illinois University, for example, actively drew upon it to formulate their institutional policy for dealing with plagiarism. They encourage staff to appreciate `that students from non-Western cultures may have different concepts of authorship and little or no training in how to use sources and therefore may need extra help in avoiding plagiarism' and that they should `expect some `patchwriting' (developmental plagiarism) that is unintended, and allow time for revision of patchwritten texts'. These are recommendations that come directly from the Lancaster research. At the London School of Economics the research was described by members of a working group on the use of Turnitin for PhD theses as being `very helpful in forming a sensible policy for LSE in this area.'

Impact on training for staff and support for students:

The research has been adopted by various international HEIs to train their staff to be sensitive to the complex, culturally specific practices that are implicated when students write for academic purposes. At the University of Sunderland, for example, it was used to develop workshops for staff to show them the limits of PDS and how knowledge of these systems can be used support strategies for developing `deep' learning approaches. The work has been used nationally in university staff training workshops at The LSE, York, Leeds, Wolverhampton, Kingston and Bradford and also internationally at Penn State University, University of San Diego, Wisconsin, Syracuse, Sydney, Guelph and Simon Fraser University. An external advisor to the Plagiarism Advice Service, at the time of the project, stated that `The work by Hayes and Introna has changed how I think about student plagiarism ... The SDAW outcomes, used generically, have made a difference to how I designed and delivered workshops and presentations on teaching international students since 2008 (amounting to c. 50 events in c. 10 countries to c. 2000 people.' This advisor suggested that the findings could have ongoing impact in the form of a handbook for practitioner use. This is now under development.

The research has enabled student unions to provide more effective support for students who were accused of plagiarism. The students union of Concordia University, in Montreal, used the research as part of their `Academic Fairness Campaign' (in 2010/11) for a more nuanced way of dealing with students identified as plagiarists by detecting systems. Drawing on its findings, they suggested that universities `should endeavour to: Create teachable moments out of seemingly bad situations. If an international student, especially a new student, is suspected of plagiarism, do not automatically assume intent to be dishonest... Adopting an educative approach to plagiarism is preferable to one based solely on punishment.' This campaign is just one example of how this work has helped students to get fair treatment.

An Executive Member of the UK Council for Graduate Education and member of the project steering group, based at Royal Holloway, University of London, confirms that there was little research at the time that, `explored either how difficult it may be for students from overseas (China, India, Greece) to make sense of doing a UK Masters programme ... nor had previous work integrated academic writing, student educational biographies and use of Turnitin'. She invited the research team to speak at Bristol University (her former institution) and Royal Holloway, to share the project findings. A member of the National Advisory Board of the Academic Integrity Standards Project funded by the Australian Office for Learning and Teaching since 2010 corroborates the impact of Introna and Hayes' research and that it, `contributes to ongoing dialogues among university lecturers, program/course coordinators and university executives about how to better understand and use pedagogic means to address the underlying issues that contribute to instances of plagiarism. I and others in this field in Australia contribute regularly to national dialogues on these issues... In addition, I contribute to university dialogues with colleagues and faculty management, presenting seminars and workshops that draw on the work of Introna and Hayes to advise on pedagogic approaches ... for supporting international students to develop competence in academic writing.'

Overall the work at LUMS has been instrumental in shifting the debate on plagiarism from one of `detecting and punishing' to a more nuanced understanding of the complex cultural and technological conditions that shape writing practices. The research has been disseminated and debated worldwide by institutions, students and interested parties via social media and has underpinned further academic research in the field. In sum: the project has, through its innovative research and dissemination strategies, transformed writing support and teaching practices in at least 32 HEIs, nationally and internationally. The actual impact is most certainly significantly more since this impact case only focused on impact that was formally recorded.

Sources to corroborate the impact

Media:

1. The Independent, 11^th March 2012, `45,000 caught cheating at Britain's universities'. This article demonstrates the reach and significance of this research, stating that an approximate 45,000 students per year are deemed to have `cheated' in British universities and that this `trend is on the rise'.
2. Times Higher Education, 29^th January 2009, `A cheat, moi? That's unfair' - discusses the misconception that copied text is plagiarism, often re-affirmed by `flawed software'.

References in institutional policies and guidance:

3. The Association of Information Systems drew on this research to develop their `Code of Research Conduct'.
4. Universities Scotland used the research to develop their `Equality Toolkit'.
5. The Asian Pacific Forum for Educational Integrity lists the research as a resource.
6. A list is available on request of national and international HEIs that have incorporated the research findings into their policies, and education-related institutions that use the research as a resource for students and media (newspapers, blogs, etc.).

Testimonials:

7. Professor of Higher Education Management and Executive Member of the UK Council for Graduate Education, Royal Holloway, University of London and member of the project steering group — confirms the originality of the approach taken to researching plagiarism and her invitation to present the findings at Royal Holloway and Bristol University.
8. Evaluator of the SDAW project and Independent Consultant, Oxford Centre for Staff and Learning Development. Formerly External Advisor to the Plagiarism Advice Service, 2003- 2008 - corroborates that this research has changed how she consults on plagiarism globally, to c.2000 people.
9. Director of Services & Research, UK Council for International Student Affairs — corroborates that the project findings are used for UKCISA's national training programme.
10. Associate Professor of Educational Studies, Australian Catholic University and member of the National Advisory Board of the Academic Integrity Standards Project funded by the Australian Office for Learning and Teaching (2010-2013) - corroborates the claims of the research and the impact on Australian educational policy and discourse on plagiarism.