Summary of the impact

Bournemouth University (BU) research has been instrumental in an industry-wide shift in public relations (PR) evaluation practice. Accurate PR evaluation allows organisations to maximise use of resources and target efforts efficiently. The once widely-used Advertising Value Equivalence (AVE) — sometimes referred to as `equivalent' or `equivalency' — measures PR activity in terms of financial equivalence in advertising space. Watson's (BU 2007 to present) research has exposed AVE usage as methodologically faulty. It has been highly influential in a policy change by the UK's Chartered Institute of Public Relations (CIPR) in 2010 banning AVE data from its awards. This simultaneously set a standard for industry practice. Watson also shaped international PR sector policy through participation in the development of the `Barcelona Principles', advising against AVE in favour of objective-driven, evidence-based evaluation.

Underpinning research

Watson has conducted much research into practitioner attitudes towards PR evaluation (P3 & P4) and in a joint paper with Gregory (P1) identifies AVE as a continuing PR industry problem. Evidence from practitioner surveys, industry award schemes and articles shows extensive reliance on this faulty metric.

AVE values news coverage at equivalent cost of the space or airtime in advertising and is criticised by Watson (P2) as a false comparison for a number of reasons:

- AVE does not take into account whether the coverage is positive or negative. For example, a piece of negative news coverage could be given a high financial equivalent value.

- AVE will value the size of an entire article, even if the organisation is only mentioned once alongside a number of their other competitors.

- Advertisement content is controlled by the advertiser and can, for example, include any number of key brand messages or logos. News coverage is controlled by the news editor, subject to an unbiased editorial process, and therefore not necessarily what the organisation is choosing to say.

- AVE suggests all news releases are attempts at advertising, demeaning the reputation of public relations.

- AVE measures a form of output not outcomes, such as shifts in perception, changes of opinion or decisions to take action.

Watson argues that evaluation should take a more holistic approach, related to the campaign objectives. He claims there is no single system that can be applied because every organisation and campaign has different requirements and priorities (P2).

Watson examined different methods of evaluating public relations activity as an alternative to AVE (P2). He looked at how best to gather and interpret information, a range of structures and approaches that can be applied, the most efficient way to devise a media evaluation system and objective setting. An Excel-based spread sheet connected to publication P2 guides organisations in evaluating their media coverage. Some of the factors included are:

- Tonality of message (positive/negative/neutral);

- Relevance of target audience or media outlet;

- Opportunities to see/hear (relating to readership or ratings);

- Whether the news coverage is only about the one organisation (solus) or whether it is one of many;

- Use of spokespeople; identification of journalists;

- Effectiveness of public relations strategies and tactics.

More recently, Watson's research focus has moved to evidence-based evaluation of PR and communication activities, especially the application of the business concept Return on Investment (ROI) to measure effectiveness (P4).

In parallel to his research into public relations measurement and evaluation, Watson has examined the historic evolution of AVE, to better understand why it has become such a feature of the industry. This has resulted in several publications (P5 and P6) identifying AVE's origin with press agents and publicists in the US during the 1920s and 1930s. Its usage became widespread from the 1960s onwards.

References to the research

* P2 was much revised from the first edition published in 2005. Some 30% of its content was new and expanded material. The revision was undertaken after Watson joined BU in January 2007 and was published in August that year. A third edition will be published shortly (2013).

Details of the impact

The UK PR sector has a workforce of 61,000. Accurate evaluation of their PR activity allows organisations to apply their resources as effectively as possible. Watson's research and academic advice has significantly contributed to a shift in industry evaluation practice away from AVE to an evidence-based, objective-driven approach. In addition, it has led to increased budget allocations for measurement and evaluation (GAP study, 2012) (R1).

Watson helped establish the European Summit on Measurement, held in Berlin in 2009 and it was at the second Summit in Barcelona in 2010 when the seven `principles' for measurement of PR and communication activity were established. Barcelona Principle 3 states: "AVEs are not the value of Public Relations" (R2).

Watson's involvement in the Barcelona Principles and Principle 3 in particular, is evidenced by the Chief Executive of the US-based Institute for Public Relations (IPR). Referencing P1 and P2 specifically, he states Watson "has been a vigorous participant in professional debate and discussion to help bridge the gap between the university world and professional practice" (R1).

He continues: "His research papers have been valued highly by the IPR Measurement Commission for their practical advice for people working in the field. He has made an important contribution to the Commission's policy rejecting Advertising Value Equivalency (AVE) and to a major international policy statement in this regard (within a declaration known as the Barcelona Principles that has been adopted by public relations professional bodies around the world). More recently Watson's work on other public relations measurement issues, such as the concept of Return on Investment (ROI, see P4), has sparked intense industry discussion in North America" (R1 and see R3 for the IPR policy).

Impacts arising from the Principles began in 2010. The IPR concluded there is no evidence to suggest advertising and editorial space hold equivalent value (R4). In October that year Watson used his research evidence (P1 and P2) to present a case to CIPR's chief executive to ban the use of AVE in its annual awards. CIPR adopted this policy in late 2010 and it is still current (R4).

The Chief Executive of the CIPR confirmed the role Watson's research played in the decision to ban the use of AVE in its awards: "In this activity, we have been greatly assisted by the work undertaken on this subject by Professor Tom Watson of Bournemouth University. Two of his works particularly provided these policy developments with intellectual credibility: `Evaluating Public Relations', co-authored with Paul Noble, part of the CIPR PR in Practice Series and a joint paper with Professor Anne Gregory `Defining the gap between research and practice in public relations programme evaluation — towards a new research agenda'. Both works set out problems with AVE and encourage the profession to develop a more comprehensive, realistic and valuable approach to measurement and evaluation of public relations management" (R5).

The CIPR is the longest-standing (since 1948) and most respected UK PR professional organisation. The Excellence awards attract more than 700 entrants on average per year and have a significant influence on industry practice. The CIPR's stance against AVEs is set out in its Excellence Awards judging rules which state: "The CIPR's understanding of best practice does not include the use of Advertising Value Equivalent (AVEs) in any form and AVEs will not be scored." Entrants are informed that any use of AVE "will result in a zero score in the Award's measurement criteria" (R4).

To gauge the response to CIPR's policy move against AVEs, entries from the 2010 and 2011 awards (pre and post-AVE ban) in the five categories with most entries were investigated by BU (Fig. 1). A total of 85 entries were reviewed and showed the effect was immediate.

Fig. 1. Use of AVE as a measurement metric in CIPR awards. Source, CIPR.

	2010 Winners	2011 Entries	2011 Winners
Use of AVE as a measurement of PR campaign effectiveness	60%	32%	0%

# CIPR only retains winning entries, and disposes of other entries. It was not possible, therefore, to compare all 2010 and 2011 in the five categories.

Although judges had an instruction to reject AVE-supported entries, usage of AVE dropped by nearly half (46.7%) when the 2011 entries were compared with the 2010 winners.

Notably six of the world's twelve leading public relations award schemes have also barred the use of AVE as evidence in entries. Watson's research has made a significant contribution to this trend. Organisations include the Middle East Public Relations Association, the Public Relations Society of America, the Society for New Communications Research (US) and the Institute for Public Relations (US) (R6).

The awards themselves reflect best practice within the industry and Watson's research has been instrumental in changing award entry criteria. This forms a significant contribution to the industry trend away from using the AVE metric in favour of an objective-driven, evidence-based evaluation, as advocated by Watson.

Sources to corroborate the impact

R1. Letter Chief Executive, Institute of Public Relations, University of Florida, Gainesville, FL, USA, 26 September 2013. (Available on request).

R2. The Barcelona Principles: http://amecorg.com/2012/06/barcelona-declaration-of-measurement-principles/

R3. Institute for Public Relations policy (October 11, 2010):
http://www.instituteforpr.org/releases/ave-is-not-a-proxy-for-measuring-the-roi-of-public-relations/ [Accessed September 12, 2013]

R4. CIPR Chartered Institute of Public Relations, 2013. Judging. London: CIPR. Available from http://www.cipr.co.uk/content/events-awards/excellence-awards/how-the-judging-works [Accessed September 12, 2013]
* The URL evidences the CIPR's policy to bar AVE, which was adopted in late 2010.

R5. Letter from Chief Executive of the Chartered Institute of Public Relations, London, UK. 10 September 2013. (Available on request).

R6. Measurement Standard (November, 2012):
http://kdpaine.blogs.com/themeasurementstandard/2012/11/barcelona-principles-report-card-how-do-12-awards-programs-measure-up.html?utm_source=TMS+November+2012c&utm_campaign=Nov+29+2012+TMS+email&utm_medium=email