Summary of the impact

The impact of a research programme into quality assessment measures for publicly funded legal services has been the establishment of a peer review programme for all civil and criminal lawyers operating in Scotland, England and Wales. This programme has ensured that the quality of service provided by legal aid lawyers in Scotland is consistently high, with only 10% of providers failing routine reviews. Moreover, the errors that do emerge are primarily administrative failings rather than poor legal advice. The Scottish model has been the basis for pilot projects in the Netherlands, Finland and Moldova, and has been drawn on for a peer review programme for all Dutch notaries.

Underpinning research

Context: Value for money for taxpayers in public expenditure has been a key goal for the UK Treasury for over 20 years. Publicly funded legal services were late to come into the frame. Concerned by evidence that most legal aid firms in England and Wales (70%) did only a small proportion of the work (30% ) and were, therefore, by definition `dabblers' who were likely to be doing the work inefficiently, the English Legal Aid Board decided in 1993 to introduce optional contracting (franchising ) for providers. In 2000, compulsory contracts were imposed and now competitive tendering for contracts based on price is planned. These initiatives (justified on value for money grounds) have all required a robust quality assessment mechanism as an essential component in the reforms. Similarly, in 2003 the Scottish Legal Aid Board (SLAB) introduced quality assurance measures of its public defenders as part of the justification for the expansion of their services.

Key research findings: Alan Paterson was first commissioned (along with Professor Avrom Sherr and Richard Moorhead from Liverpool University) by the English Legal Aid Board to provide a report [1], on assessing and developing competence and quality in legal aid lawyers. At that time there was no reliable, verifiable model for such an assessment. The theoretical framework adopted and assessment of work done in other disciplines and jurisdictions was provided by Professor Paterson and demonstrated for the first time in the legal world (1) the potential for file auditing methods for assessing quality, (2) that performance was a continuum (at a time when quality in a professional context was seen as binary phenomenon), and (3) the difficulties in identifying reliable proxies for quality in legal services.

In 1998 the original research team were again commissioned by the Legal Services Commission (LSC) to evaluate the quality of work done by lawyers and the `not for profit' sector, who held the new legal contracts for civil work that had been allocated by the LSC. The research [3], examined a range of quality measures including peer review, model clients, client satisfaction surveys and outcomes, and tested them against each other on a substantial scale for the first time in a legal context. The fieldwork and analysis established the reliability and validity of peer review (with appropriate criteria, marking frameworks and training of assessors) and showed (again for the first time) that it was the best available means for assessing the quality of legal work. In 2003, Paterson's work for the Scottish Legal Aid Board (SLAB) demonstrated that file based peer review was a viable quality measurement process for Scottish public defenders [4]. In 2005, his work was used to justify and underpin the introduction of peer review for civil legal aid practitioners in Scotland. In 2011, his monitoring research work was again instrumental in the decision by SLAB and the Law Society to expand civil and childrens' peer review in a risk based direction.

Key researchers: For all of these reforms, the legal aid authorities relied on research conducted by a small team of legal researchers in the UK, one of whom was Professor Paterson, who has held a Chair in Law at the University of Strathclyde from 1984 until the present day.

References to the research

Details of the impact

Process from research to impact

The 1994 and 2001 reports were commissioned by the legal aid authorities because a robust quality assessment mechanism was required in order to implement franchising, contracting and competitive tendering on price. The Legal Services Commission (LSC) accepted the 2001 report's recommendations on peer review and implemented a three year rolling programme in 2003 using reviewers trained and monitored by the research team, of a sample of contract holders in all areas of civil and criminal work in England and Wales. This has continued to the present day under the aegis of Avrom Sherr of London University. Professor Paterson was formally appointed by the LSC as the designated cover for Professor Sherr in 2008.

Initially the dissemination of the quality assessment research by Professors Sherr, Moorhead (now UCL) and Paterson was through the publications listed in section 3. However, from presentations by Professors Paterson and Sherr at the 2001, 2005 and 2007 International Legal Aid Group (ILAG) conferences the leading policymakers of the most developed legal aid programmes globally became aware of the outcome of the peer review research. As a result, Professor Paterson, already Chair of ILAG and the research adviser to SLAB, was asked to devise a peer review scheme based on the teams' research to assess the quality of the legal work done by the recently established Public Defence Solicitors Service (PDSO) in early 2003. Shortly thereafter he became the adviser to a partnership of SLAB, the Law Society and the Scottish Executive which established peer review for all civil legal aid practitioners in Scotland based on the research set out in section 2 (Source 1). Prof Paterson has been responsible for the training and monitoring of the reviewers between January 2008 and the present day. Lindsay Montgomery, the CEO of SLAB stated in 2013 that "the development of our quality regimes for legal aid lawyers in Scotland was heavily influenced by the research work published by Professor Paterson (in conjunction with Professor Sherr) demonstrating the efficacy of peer review as a tool for quality enhancement in the legal profession and legal aid" (Source 4).

Strathclyde contribution to impact: The research contributions of Paterson (University of Strathclyde) and Sherr (University of London — Institute of Advanced Legal Studies) are approximately equal with respect to peer review. Paterson took the initial lead on measuring quality, and shared the responsibility for the research into, and eventual format of the peer review, equally with Sherr. From 2003 onwards Moorhead (University of Liverpool) played a major role in the implementation of the research in England and Wales.

Types of Impact: The immediate impact of Paterson's research has been the establishment of a peer review programme for civil and criminal lawyers across the UK. The secondary impacts arising from this peer review process are improvement in the quality of service provided, which in turn has transferred to the recipients of Legal Aid. The UK peer review scheme has now been implemented in several European countries with Paterson playing a key role in the training of these reviewers.

Establishment of peer review system in the UK:

From 1st Jan 2008 to July 2009 400 civil firms in Scotland with around 800 practitioners and 4,000 files were assessed by peer review. An 18 month fundamental review of the civil peer review programme, conducted by SLAB and the Law Society drawing on the monitoring work done by Professor Paterson in relation to civil and children's peer review concluded that peer review was reliable and value for money even in times of austerity.

In implementation of the review and in keeping with risk based quality assurance the decision was taken in 2011 to extend the cycle to 6 years in which every civil firm and practitioner's files would be assessed, to focus particularly on poorer performing firms and to review a much larger range of files for practitioners working in areas of law with vulnerable clients (child law, mental health and immigration), together with the existing review of all civil practitioners. Since the start of the third cycle reviews of 140 firms, 320 practitioners and 2,240 files had been conducted by January 2013. In 2011 despite the cuts to legal aid, the Scottish Government, SLAB and the Law Society agreed to introduce a peer review programme based on Professor Paterson's research to assess the work of all 550 criminal law firms and 1500 criminal legal aid practitioners in Scotland over a six year cycle (Source 2). By January 2013 80 firms, 230 practitioners and 2,000 files had been assessed.

In England and Wales the peer review programme conducted reviews of approximately 1,000 firms between 2008 and 2012. The process in England and Wales is on a sample basis rather than the case in Scotland where every civil firm and practitioner is reviewed in cycles.

Quality of service provision:

The purpose of the quality assurance programme for legal aid providers is not to covertly reduce the supply base, but to demonstrate the quality floor that exists in the profession and to gradually raise overall standards. The programme has established that errors in legal advice, professional negligence or professional misconduct are relatively uncommon in Scotland. The evidence also suggests that the programme is raising standards. In the first cycle 10% of files failed the initial review compared with 9% in the second. However, in the second cycle a tougher standard was imposed to pass and in the third cycle the threshold has been raised again. The proportion of special reviews (triggered by serious concerns) reduced from 2% of firms in the first cycle to 0.5% of firms in the second and the number of firms taken to final review decreased from 3% to 2%. Further evidence of quality improvement stems from the fact that practitioners and files received a higher proportion of distinction grades in the second cycle (15.9% of practitioners and 11.6% of files as compared with 13.7% of practitioners and 10.2% of files in the earlier cycle).

Benefits to recipients of Legal Aid:

The 350,000 persons a year who apply for or receive legal assistance each year in Scotland, England and Wales are benefitting not just from the raised standards, but from by a reduction in delays, an improvement in client communications and a greater scrutiny of at risk files.

International reach and significance:

Dissemination at ILAG conferences in 2005 and 2007 led to requests for Paterson to demonstrate the Scots peer review model in Canada, Chile, Finland, Hong Kong, Moldova, the Netherlands, New Zealand and Northern Ireland. Delegations from several of these jurisdictions have visited Scotland to see the programme in action.

Paterson was invited by the Dutch legal aid authorities to demonstrate peer review in Amsterdam and to train reviewers in 2008 to initiate a pilot programme using peer review in mental health cases there in 2009. In the meantime, the Dutch notaries made peer review compulsory in 2009 in part drawing on demonstrations of the Scots programme. This was followed by further training visits and a request in 2012 to assist with the establishment of a pilot programme in social welfare in the Netherlands.

Inspired by a visit to Scotland to meet Paterson in 2007, the representatives of the Ministry and The Finnish Bar Association agreed to start examining different options for the development of quality evaluation of legal aid in Finland. At the request of the Finnish Ministry of Justice, Paterson demonstrated the peer review system in 2009 and in 2011 he provided training for 15 Finnish reviewers for a peer review pilot programme there. In April 2013, the Head of Legal Aid in the Finnish Ministry of Justice wrote, `The expertise of Professor Paterson has been of key importance during the entire development process of the peer review system in Finland. Thanks to Professor Paterson's inspiring and professional training, the peer reviewers attending the training were able to truly understand and appreciate the importance and use of peer review as a quality evaluation method. The positive experiences gained in England and Scotland convinced them of the potential of peer review in getting valuable information on the quality standard" (Source 3).

In 2011, Paterson visited Moldova twice to train ten reviewers for a criminal legal aid pilot peer review project there.

In summary: Scotland's legal aid lawyers are in effect being re-validated well before the doctors in the UK, or the criminal advocates in England and Wales, putting them ahead of anything that exists in other legal aid countries

Sources to corroborate the impact

1. http://www.lawscot.org.uk/members/legal-aid--access-to-justice/civil-quality-assurance confirms that the Law Society of Scotland carries out reviews of all Civil Legal Aid lawyers. The Manual also confirms that the programme was based on research by Paterson, Sherr and Moorhead.
2. http://www.lawscot.org.uk/members/legal-aid--access-to-justice/criminal-quality-assurance supports the claim that from December 2010 the Quality Review was extended to criminal legal aid practitioners.
3. Statement from Head of Legal Aid and Civil Enforcement Unit in Finland will support the claim(s) that Paterson has assisted with the introduction of peer review in Finland and training of peer reviewers.
4. Statement from CEO of the Scottish Legal Aid Board will support the claim(s) that the peer review programme in Scotland is based on the original work by Paterson, Sherr and Moorhead and of Paterson's continuing role in the programme.