Driving change in public relations evaluation
Submitting InstitutionBournemouth University
Unit of AssessmentCommunication, Cultural and Media Studies, Library and Information Management
Summary Impact TypeSocietal
Research Subject Area(s)
Studies In Creative Arts and Writing: Journalism and Professional Writing
Summary of the impact
Bournemouth University (BU) research has been instrumental in an
industry-wide shift in public relations (PR) evaluation practice. Accurate
PR evaluation allows organisations to maximise use of resources and target
efforts efficiently. The once widely-used Advertising Value Equivalence
(AVE) — sometimes referred to as `equivalent' or `equivalency' — measures
PR activity in terms of financial equivalence in advertising space.
Watson's (BU 2007 to present) research has exposed AVE usage as
methodologically faulty. It has been highly influential in a policy change
by the UK's Chartered Institute of Public Relations (CIPR) in 2010 banning
AVE data from its awards. This simultaneously set a standard for industry
practice. Watson also shaped international PR sector policy through
participation in the development of the `Barcelona Principles', advising
against AVE in favour of objective-driven, evidence-based evaluation.
Watson has conducted much research into practitioner attitudes towards PR
evaluation (P3 & P4) and in a joint paper with Gregory (P1) identifies
AVE as a continuing PR industry problem. Evidence from practitioner
surveys, industry award schemes and articles shows extensive reliance on
this faulty metric.
AVE values news coverage at equivalent cost of the space or airtime in
advertising and is criticised by Watson (P2) as a false comparison for a
number of reasons:
- AVE does not take into account whether the coverage is positive or
negative. For example, a piece of negative news coverage could be given a
high financial equivalent value.
- AVE will value the size of an entire article, even if the organisation
is only mentioned once alongside a number of their other competitors.
- Advertisement content is controlled by the advertiser and can, for
example, include any number of key brand messages or logos. News coverage
is controlled by the news editor, subject to an unbiased editorial
process, and therefore not necessarily what the organisation is choosing
- AVE suggests all news releases are attempts at advertising, demeaning
the reputation of public relations.
- AVE measures a form of output not outcomes, such as shifts in
perception, changes of opinion or decisions to take action.
Watson argues that evaluation should take a more holistic approach,
related to the campaign objectives. He claims there is no single system
that can be applied because every organisation and campaign has different
requirements and priorities (P2).
Watson examined different methods of evaluating public relations activity
as an alternative to AVE (P2). He looked at how best to gather and
interpret information, a range of structures and approaches that can be
applied, the most efficient way to devise a media evaluation system and
objective setting. An Excel-based spread sheet connected to publication P2
guides organisations in evaluating their media coverage. Some of the
factors included are:
- Tonality of message (positive/negative/neutral);
- Relevance of target audience or media outlet;
- Opportunities to see/hear (relating to readership or ratings);
- Whether the news coverage is only about the one organisation (solus) or
whether it is one of many;
- Use of spokespeople; identification of journalists;
- Effectiveness of public relations strategies and tactics.
More recently, Watson's research focus has moved to evidence-based
evaluation of PR and communication activities, especially the application
of the business concept Return on Investment (ROI) to measure
In parallel to his research into public relations measurement and
evaluation, Watson has examined the historic evolution of AVE, to better
understand why it has become such a feature of the industry. This has
resulted in several publications (P5 and P6) identifying AVE's origin with
press agents and publicists in the US during the 1920s and 1930s. Its
usage became widespread from the 1960s onwards.
References to the research
P1. Gregory, A. and Watson, T. (2008) Defining the gap between
research and practice in public relations programme evaluation — towards a
new research agenda. Journal of Marketing Communications, 14(5),
337-350. DOI: 10.1080/13527260701869098
P2. Watson, T. and Noble P. (2007) Evaluating Public Relations
(2nd ed.). London: Kogan Page.* [First edition 2005; also published in
Japanese, Korean and Russian].
P3. Watson, T. (2008) Public relations research priorities: A
delphi study. Journal of Communication Management, 12(2), 104-123.
P4. Watson, T. (2012) An initial investigation on the use of
`Return on Investment' in public relations practice. Public Relations
Review, 37(3), 314-317. DOI: 10.1016/j.pubrev.2011.06.001
P5. Watson, T. (2012) The evolution of public relations
measurement and evaluation. Public Relations Review, 38(3),
P6. Watson, T. (2013) Advertising value equivalence — PR's orphan
metric. Public Relations Review, 39(2), 139-146. DOI:
* P2 was much revised from the first edition published in 2005. Some 30%
of its content was new and expanded material. The revision was undertaken
after Watson joined BU in January 2007 and was published in August that
year. A third edition will be published shortly (2013).
Details of the impact
The UK PR sector has a workforce of 61,000. Accurate evaluation of their
PR activity allows organisations to apply their resources as effectively
as possible. Watson's research and academic advice has significantly
contributed to a shift in industry evaluation practice away from AVE to an
evidence-based, objective-driven approach. In addition, it has led to
increased budget allocations for measurement and evaluation (GAP study,
Watson helped establish the European Summit on Measurement, held in
Berlin in 2009 and it was at the second Summit in Barcelona in 2010 when
the seven `principles' for measurement of PR and communication activity
were established. Barcelona Principle 3 states: "AVEs are not the value of
Public Relations" (R2).
Watson's involvement in the Barcelona Principles and Principle 3 in
particular, is evidenced by the Chief Executive of the US-based Institute
for Public Relations (IPR). Referencing P1 and P2 specifically, he states
Watson "has been a vigorous participant in professional debate and
discussion to help bridge the gap between the university world and
professional practice" (R1).
He continues: "His research papers have been valued highly by the IPR
Measurement Commission for their practical advice for people working in
the field. He has made an important contribution to the Commission's
policy rejecting Advertising Value Equivalency (AVE) and to a major
international policy statement in this regard (within a declaration known
as the Barcelona Principles that has been adopted by public relations
professional bodies around the world). More recently Watson's work on
other public relations measurement issues, such as the concept of Return
on Investment (ROI, see P4), has sparked intense industry discussion in
North America" (R1 and see R3 for the IPR policy).
Impacts arising from the Principles began in 2010. The IPR concluded
there is no evidence to suggest advertising and editorial space hold
equivalent value (R4). In October that year Watson used his research
evidence (P1 and P2) to present a case to CIPR's chief executive to ban
the use of AVE in its annual awards. CIPR adopted this policy in late 2010
and it is still current (R4).
The Chief Executive of the CIPR confirmed the role Watson's research
played in the decision to ban the use of AVE in its awards: "In this
activity, we have been greatly assisted by the work undertaken on this
subject by Professor Tom Watson of Bournemouth University. Two of his
works particularly provided these policy developments with intellectual
credibility: `Evaluating Public Relations', co-authored with Paul Noble,
part of the CIPR PR in Practice Series and a joint paper with Professor
Anne Gregory `Defining the gap between research and practice in public
relations programme evaluation — towards a new research agenda'. Both
works set out problems with AVE and encourage the profession to develop a
more comprehensive, realistic and valuable approach to measurement and
evaluation of public relations management" (R5).
The CIPR is the longest-standing (since 1948) and most respected UK PR
professional organisation. The Excellence awards attract more than 700
entrants on average per year and have a significant influence on industry
practice. The CIPR's stance against AVEs is set out in its Excellence
Awards judging rules which state: "The CIPR's understanding of best
practice does not include the use of Advertising Value Equivalent (AVEs)
in any form and AVEs will not be scored." Entrants are informed that any
use of AVE "will result in a zero score in the Award's measurement
To gauge the response to CIPR's policy move against AVEs, entries from
the 2010 and 2011 awards (pre and post-AVE ban) in the five categories
with most entries were investigated by BU (Fig. 1). A total of 85 entries
were reviewed and showed the effect was immediate.
Fig. 1. Use of AVE as a measurement metric in CIPR awards. Source,
|Use of AVE as a measurement of PR campaign effectiveness
# CIPR only retains winning entries, and disposes of other entries. It
was not possible, therefore, to compare all 2010 and 2011 in the five
Although judges had an instruction to reject AVE-supported entries, usage
of AVE dropped by nearly half (46.7%) when the 2011 entries were compared
with the 2010 winners.
Notably six of the world's twelve leading public relations award schemes
have also barred the use of AVE as evidence in entries. Watson's research
has made a significant contribution to this trend. Organisations include
the Middle East Public Relations Association, the Public Relations Society
of America, the Society for New Communications Research (US) and the
Institute for Public Relations (US) (R6).
The awards themselves reflect best practice within the industry and
Watson's research has been instrumental in changing award entry criteria.
This forms a significant contribution to the industry trend away from
using the AVE metric in favour of an objective-driven, evidence-based
evaluation, as advocated by Watson.
Sources to corroborate the impact
R1. Letter Chief Executive, Institute of Public Relations,
University of Florida, Gainesville, FL, USA, 26 September 2013. (Available
R2. The Barcelona Principles: http://amecorg.com/2012/06/barcelona-declaration-of-measurement-principles/
R3. Institute for Public Relations policy (October 11, 2010):
[Accessed September 12, 2013]
R4. CIPR Chartered Institute of Public Relations, 2013. Judging.
London: CIPR. Available from http://www.cipr.co.uk/content/events-awards/excellence-awards/how-the-judging-works
[Accessed September 12, 2013]
* The URL evidences the CIPR's policy to bar AVE, which was adopted in
R5. Letter from Chief Executive of the Chartered Institute of
Public Relations, London, UK. 10 September 2013. (Available on request).
R6. Measurement Standard (November, 2012):