Summary of the impact

Faced with pressures on the UK Employment Tribunal (ET) system, policymakers have turned to alternative dispute resolution (ADR) as a way of easing the strain. However, there is little robust evidence of a statistically significant cost-saving impact from ADR. The evaluation of Judicial Mediation (JM) in ETs was the first to use robust statistical evaluation techniques. The Ministry of Justice commissioned study found that JM did not provide good value for money. The results have been debated widely amongst policymakers, practitioners and across various media; impacting the activities, attitudes, awareness and practice of those involved in ADR within the UK.

Underpinning research

This large mixed-methods study was made up of Economists with experience of quantitative evaluation methodologies; Sociologists with experience of qualitative data methodologies; Industrial Relations experts and Lawyers. The research took a robust approach to evaluation, comparing outcomes for those undergoing JM against the outcomes from a control group that gave a credible estimate of the counterfactual, to see if there was a (statistically) significant difference. Monetary values were then assigned to the associated costs and benefits to see if any difference was large enough to overcome the expense of providing JM. The judicial mediation pilot was carried out in the areas of Newcastle, Birmingham and Central London for cases starting between June 2006 and March 2007. The study prioritized all cases that had at least one of the six discrimination jurisdictions and during the 10 months of the pilot an estimated 868 cases satisfied these eligibility criteria.

In order to capture the value added of mediation, the outcomes from a sample of 116 mediated cases (who had obviously expressed interest in JM) were compared to an otherwise identical sample of 80 unmediated cases (who had also expressed an interest in JM in principle, but were not mediated). This approach was used to overcome some of the problems associated with selection on unobservables; where cases more willing to consider early resolution would select into JM and falsely inflate any estimated impacts. To further facilitate comparison of mediated with otherwise identical unmediated cases (who had expressed an interest), propensity score matching (PS match) was then used to re-weight the two samples, on observable characteristics.

Satisfaction and resolution rates were then compared between the matched mediated and unmediated samples, and monetary values were estimated using ETS costings, together with figures from the Survey of Employment Tribunal Applications and ASHE. Findings from the qualitative and univariate quantitative aspects of the study have been useful in providing insights into the processes of ADR and users perceptions of such. However, the headline conclusion from the multivariate quantitative (matching) analysis was that we could find no statistically significant impact on either resolution rates and/or satisfaction rates amongst parties who had undergone judicial mediation, relative to those who had expressed an interest in mediation, but had not undergone the process. The cost-benefit analysis suggested that even with substantial impacts, financial costs would outweigh financial benefits. In fact, the research suggested that rates of resolution within the pilot areas had suffered more generally, possibly as members of the judiciary were distracted from their usual case-loads to provide mediation.

References to the research

Details of the impact

Findings from the Evaluation of JM in ETs (2010) have been debated widely amongst practitioners and policymakers, not least because the Employment Tribunal Service (ETS) ignored the findings of the study and rolled out JM nationally. The debate this generated goes to the heart of the relationship between academia and government, showing how independent evidence produced by academics has impact by forcing government to account for it's actions in the arena of public opinion.

In the practitioner publication Equal Opportunities Review No. 200 (01/05/2010) the results of the evaluation were reported and the assumption was that JM would not be rolled out. However, this was not the case and subsequent to publication the editor (Sue Johnstone) contacted Prof. Peter Urwin (who had led the research) to clarify the situation. There then ensued a detailed correspondence between the MoJ, EOR and staff at the University of Westminster. During this time there was extensive discussion across practitioner journals and other forums, for instance in Equal Opportunities Review; in Tribunal a publication of the Judicial Studies Board (overseen by the Lord Chief Justice), by policy-makers within the Ministry of Justice and other practitioner publications such as The Law Society Gazette — with some commentators supporting the roll-out of JM and some alluding to the apparent contradiction with the research findings.

For instance, a subsequent Equal Opportunities Review article (01/10/2010) reported the response of Ministry of Justice policymakers who defended the roll-out by suggesting that, after "thorough consideration of the interesting and important findings of the University of Westminster evaluation report....the Tribunals Service decided that judicial mediation is sufficiently different from other alternative dispute resolution to warrant inclusion in the toolbox for employment dispute resolution". This contrasted with the reported comment from Prof. Urwin that, whilst, "he agreed that there is evidence that various forms of alternative dispute resolution provide benefits to both claimants and employers...this does not constitute evidence of value for money ....and the evidence is that judicial mediation does not deliver value for money". Soon after, the research was a central focus of a special issue of the Winter 2010 issue of Tribunal, with Sir Henry Brooke discussing the study in his Introduction and David Latham (President of Employment Tribunals, England and Wales) publishing a separate article defending roll-out of JM on the grounds that users were `enthusiastic'.

There have been various debates about the pros and cons of JM in other media, with the general tone of debate being positive. The findings from the Evaluation of JM have given support to those in the public domain who adopt a more critical and analytical perspective; for instance those writing in the April 2010 issue of The Law Society Gazette and the Human Law Mediation website (2010).

Policymakers have increasingly turned to ADR as one of the tools to be deployed in coping with problems faced by adversarial systems of justice (for instance, White Paper, Department for Constitutional Affairs, 2004; Gibbons, 2007). The (mainly qualitative) evidence that continues to inform much of the discussion on various forms of ADR suggests that it is popular and provides benefits to both claimants and employers, but this does not constitute evidence of `value added'. Some commentators have rightly argued that many of the benefits of ADR are intangible and therefore hard to quantify, but the findings from evaluation of judicial mediation suggest policymakers should be more cautious in their approach and that one must have evidence of significant impact before even intangible benefits can be attributed to an intervention such as JM.

Ultimately, this project has impacted upon the activities, attitudes, awareness and practice of those involved in ADR within the UK. It has contributed to the process of accountability with respect to the allocation of public funds, and also (through subsequent dissemination) the wider debates on what constitutes evidence for UK policy-making.

Sources to corroborate the impact

Brooke, H. (2010), Re-thinking what we're trying to achieve, Tribunals, Winter Edition, (a publication of the Judicial Studies Board)

Equal Opportunities Review (2010), Judicial Mediation not to be extended, May

Equal Opportunities Review (2010), Judicial Mediation in Discrimination Cases, October

Latham, D. (2010), An Enthusiastic Response from Users, Tribunals, Winter Edition.

Rayner, J. (2010), Judicial mediation in Employment Tribunal cases falls short, The Law Society Gazette, April 22^nd.

Patten, J. (2010), Employment Judicial Mediation- In What Circumstances Should you Refuse It? Human Law Mediation Tuesday January 12^th
http://www.human-law.co.uk/Blog/2010/01/Employment-Judicial-Mediation-In-What-Circumstances-Should-you-Refuse-It/