Summary of the impact

Salford Business School researchers were commissioned by PA Sport, the sports division of the Press Association, the Football Association Premier League (FA), and FootballDataCo, which handles the rights to football data for the FA, to develop the quantitative analysis and models for an objective index of football player performance. The official player ratings system of the English Premiership, Championship and the Scottish Premiership and first of its kind:

• The EA Sports Player Performance Index (previously ACTIM) analyses player contributions to match outcomes;
• The Index informs squad selection, supports the refinement of team performance and Index statistics are presented worldwide, in print, online, and on television, and;
• The index is used in football gaming software and the English Premiership's official application for the Fantasy Football League, engaging fans across the world with quantitative analysis and generating statistics driven debate through connecting with their passion.

Underpinning research

The key researchers and positions they held at the institution at the time of the research are as follows: Dr Phil Scarf, Professor of Applied Statistics (from 1989), Dr Ian McHale, Reader in Statistics (from 2003), Dr Rose Baker, Professor of Statistics (from 1990), Dr David Forrest, Professor of Economics (from 1986). The Centre for Sports Business at the University of Salford promotes research and consultancy in sports analytics, sport finance, sport law and the economics of sport. Globally there are few academics specialising in research focused on sports analytics. Scarf et al build quantitative models for the prediction and explanation of match outcomes, analysis of sports strategies, and the management of player and team performance at the interface between business, statistics and economics. The impact described in this case study is underpinned by the following research:

• 2005: Scarf and Shi's research into match outcome modelling demonstrates the fundamental idea behind the quantitative model for the index. While focused on a different sport and a different model from that used in the PPI, the principles used (data collection, generalised linear model fitting, model diagnostics and performance) form the heart of the PPI methodology. [1]
• Two aspects of the models developed in McHale and Forrest underpin those used in the PPI. First is the fitting of generalised linear models to player performance data. Second is the idea to use, in the construction of a rating system, those performance related factors that influence tournament and match outcomes to the greatest extent. [2]
• 2006: Baker and Scarf explore computational aspects of the fitting of complex models of match outcome, a key underpinning component of the PPI. Sporting contests in which the same two teams compete regularly are studied. Strong and weak symmetry requirements for possible models are identified. The need to compute the exact likelihood function and the presence of missing values make model fitting non-trivial. [3]
• 2007: Forrest and McHale employ a large data set on professional men's tennis matches and a new econometric approach to the estimation of the relationship between returns and odds. It illustrates a key skill available in the team at the time of the development of the PPI: econometric model fitting to large data sets in sport. [4]
• 2007: McHale and Davies focus on performance measurement. The principal idea is to use results and inferred strengths to objectively determine ratings and hence rankings, a key idea in the PPI. This approach is demonstrated in the context of international football rating. [5]
• 2007: The models developed in McHale and Scarf were very similar to those developed for the PPI. While the paper and the PPI were developed concurrently, this research demonstrates that the models developed for the PPI were scientifically rigorous, an important factor in later reviews of the PPI that were carried out. [6]
• 2012: While McHale et al. appeared much later, publication of an exact description of the models in the PPI was embargoed for 5 years from the point of development. The paper describes the PPI model and procedure in detail, and demonstrates the quality of the underpinning research. The paper is co-authored with David Folker, CEO at Football-DataCo, a key decision-maker in the commissioning of the original project to develop the PPI. The paper describes how in 2003, Scarf and McHale presented to the PL, FL, FDC and PA ideas for the construction of an index, and were commissioned as consultants in 2004. The clients required the new index to be based on rigorous quantitative reasoning and to be used by PA to raise the profile of the (then) ACTIM brand. The index would:
• Have an objective, quantitative basis so that subjective opinions are not used when ranking players, or judging which player activity is more valuable than another;
• Compare players from different positions;
• Be transparent so that "lay-analysts" can understand how points are awarded;
• Use a scale familiar to the general public, and;
• Include goals scored (and later, at an update in 2006, assists and clean-sheets) as a direct component of the final index. [7]

The most challenging aspects of the development process for the Salford team were: managing expectations; deciding on desirable properties of the developing index, and communicating the statistical nuances of the index (e.g. short term variation). The final index would be a combination of individual component indices each with its own properties, construction method, and emphasis, including: winning performance; match contributions (e.g. passes, tackles, dribbles, blocks); match appearances; goals scored; assists, and; clean-sheets. The component indices were built using two seasons of PL data (2002/3 and 2003/4). PA collected these data using the Football Live system, real-time information on actions and match events that provided "ball-by-ball" or "kick-by-kick" match reports on internet and mobile platforms, supplemented by post-match video analysis that captured events too fast to log in real time. The data-capture process describes match events and associates these with a named player, so that the data contain information on match histories of player contributions. At present (2013), data collection is carried out by Opta www.optasports.com".

References to the research

Key outputs

Key grants

8. PASport A player performance index, £52,875 PScarf (50%), I McHale (50), 2004.

9. PASport Updates for the PPI, £18,800 PScarf (50%), I McHale (50), 2005-6.

10. EPSRC Optimum Strategy in Sport, £68,816, EP/F005792/1. Scarf (50%), McHale (50%), 2007.

Details of the impact

Context: Rating individuals in team sports is a complex task. Metrics for comparing players with the same specialism abound, but comparing players with different roles is difficult. It has been attempted in baseball, but football is more difficult because players have different roles that can themselves vary between games and within games. Despite these difficulties, the potential usefulness of a single rating system was clear: teams can identify undervalued players; player selection debate can be enhanced; such an index can be sold commercially. FIFA World rankings form part of the criteria for awarding work permits for players outside the European Union in the English Premier; and rankings can influence tournament outcomes. The development of the Index, the EA Sports Player Performance Index (PPI), the official player ratings system of the English Premiership, Championship and the Scottish Premiership, the first of its kind to rate individual players in team sport on a common scale, followed the convergence of three factors:

• The availability of detailed data on player contributions;
• Underpinning research at University of Salford; and
• The commercial imperative of the FA Premier League (PL).

The Index only includes actions that can be measured objectively, with the intention of removing opinion bias and only working with proven statistical metrics that become more precise as the season progresses. The Index forms part of a wider package of sports news information that is sold commercially around the world by PA acting as an agent for Football DataCo (FDC).

The Index is used in football gaming software and the English Premiership's official application for the Fantasy Football League, engaging fans across the world with quantitative analysis and generating statistics driven debate by connecting with their passion:

• On completion of its development, the index became part of ACTIM statistics and was sold worldwide by PA to media organisations, and marketed by PL, notably through Fantasy Football. Free to play and with over 2.5 million players, Fantasy Premier League is the biggest fantasy football game in the world, demonstrating the use of high quality, theoretical research in a living application of relevance to people who do not normally get the opportunity to engage with objective performance measurement and statistics;

The Index has brought economic benefit to English Football through its syndication and commercial benefit internationally:

• EA SPORTS became Official Sports Technology Partner of the Premier League in 2010 when the Index was renamed. PL Director of Sales & Marketing, said: "We are extremely pleased that EA SPORTS has had the foresight to enter into this exciting new partnership that will connect with Barclays Premier League fans in new and inventive ways. Fans have a thirst for statistics and information and that is something that the Premier League, working with EA SPORTS, will provide to a level not seen before in professional football."
• EA SPORTS uses the Player Performance Index to extend the authenticity and depth of its football franchises, sponsoring the PPI to improve its brand image and to associate objectivity and rigour with the rating system used in its games, including the EA SPORTS FIFA series of computer games, one of the highest selling games internationally. An index based on the PPI methodology is used in this game for the dynamic modification of player form. Beginning with FIFA11's new Rating, Skill and Morale System, FIFA MANAGER used the new system to "rate its 40,000 players and to make the differences between the players more obvious".
• The functionality of the Manager Hub was extended in subsequent FIFA series including FIFA13's Manager Hub to: "Analyse the match by looking at match or player specific stats to help make managerial decisions. Tap on any player to get full statistical analysis...See a player's performance stats and even their energy to help you make...substitutions." Since its release in October 2012, FIFA13 was named as the most successful game of 2012. Winning Sports Game of the Year — D.I.C.E Awards 2012, FIFA13 sold over 4.5 million copies worldwide within five days of release. It counted 800,000 simultaneous players online, an all- time record. More than 1.2 million copies were sold in the UK within 48 hours, and 66 million online games have already been played — representing 600 million minutes of online play. 14.5 million copies had been sold by May 2013.The Executive VP of EA Sports said "Every minute of every day millions of fans around the world are experiencing what the critics have already said FIFA 13 is the biggest and best sports game we have ever created."
• The EA Sports Player Performance Index (PPI) was the first of its kind, others have followed (e.g. the FIFA Castrol Index), motivated by the commercial benefits of engaging with fans', managers' and sports analysts' appetite for statistical analysis.
• The index is discussed on television e.g. Barclays Premier League Review (Sky Sports 1, Friday 25.10.2013) reviewed the best goalkeepers, presenting the top 5 and their index points.
• McHale and Scarf have promoted the science behind the index to non-academic audiences, in national newspapers (The Telegraph, The Scotsman), local newspapers and online, internationally (Bloomberg Press) discussing McHale's predictions for the winner of the 2010 World Cup. McHale appeared on national television in October 2010 (BBC 1 Football Focus); in May 2012 he discussed rating footballers using statistics (BBC Radio London and Radio Manchester; in June 2012 he discussed forecasts for European Championships on BBC R5.

Sources to corroborate the impact

a) Letter from the General Manager of the Football Data Co. "When we originally talked to Ian McHale and Phil Scarf on this project we knew what we wanted to achieve but not how it could be achieved. We had a lot of collected data but no idea of how this could be developed into an index. All we knew is that we wanted all emotive decision making to be removed and replaced with sustainable, rigorous and impartial science. What the project delivered and continues to deliver, albeit in modified form, is the concept of using data to generate public interest. From this interest comes commercial application via the creation of a new sponsorship category relating to performance."

b) Letter from Commercial Director, Press Association Sport: "The Index was adopted by the Premier League and Football League as the Official Player Rating System. Since its launch the clients have been happy with its performance."