Summary of the impact

This case study describes the effects of a new performance management system, developed in China and adopted by organisations facing complex management challenges. To date, seven public and private sector organisations have successfully used the `3E' (Effectiveness, Efficacy, and Efficiency) system for tasks as varied as developing performance indicators, improving management communication and designing appraisal systems that work under conditions of rapid growth and change.

The participating organisations include the Chinese Academy of Sciences (CAS), Hunan University, Tonsan Adhesives Inc, Xingxiang Insurance, Sun-rivier local government, Liao-ling Plaze hotel and Hua-bao International Holdings. Their adoption of the 3E system has resulted in improved performance management within these organisations.

Underpinning research

This case study is based on an action research project carried out by Professor Wenbin Liu (joined KBS in 1995) and Professor John Mingers (joined KBS in 2003) between 2005 and 2010. Their collaboration drew on their expertise in soft systems methodology (Mingers) and data envelopment analysis (Liu) and was conducted at Kent Business School.

Data envelopment analysis (DEA) is a quantitative methodology for assessing the relative efficiency of a number of similar organisational units (3.1, 3.2). One of the key steps in performance assessment is to develop Key Performance Indicators (KPIs) for the assessed organizations.

This collaboration developed when the Chinese Academy of Sciences asked Liu to develop performance indicators and then use data envelopment analysis to measure the performance of their 116 science and technology units (which included 89 research institutes and three universities employing 50,000 people). One of the first challenges was to reach agreement with unit directors on the agreed key performance indicators (KPIs).

Mingers' expertise in soft systems methodology (SSM) was used to help generate this consensus. Soft systems methodology is a qualitative, interpretive methodology designed to deal with complex situations of uncertainty and disagreement. In this context, it was used to generate agreement over the primary purposes and activities of research centres. From this, a model of necessary activities was developed and key performance indicators were defined for each activity. This allowed the development of an appropriate data envelopment analysis model. This method was then used in developing KPIs for the Chinese Academy of Sciences International Research Institute Ranking Project. (3.3, 5.1).

Based on the work at the Chinese Academy of Sciences, a refined version of the combined methodology was developed around the concepts of effectiveness, efficacy, and efficiency and was called the `3E' methodology (3.4). This system emerged from a second intervention in the Hunan University Foreign Languages School, which wished to improve its research performance by developing KPIs and a performance management system. This proved to be a generic performance management methodology that was particularly suited to public sector organisations, which often incorporate a variety of missions, stakeholders, values and cultures. It provides an alternative to the much used balanced business scorecard and is more appropriate for complex organisations.

In the third intervention, the 3E methodology was applied to the private sector within Tonsan Adhesives, a hi-tech company struggling to manage its rapid growth. High level strategies were broken down into low level key business activities and a set of key performance indicators. These were used in subsequent performance appraisals. This study demonstrated that the 3Es system can also be applied successfully in the private sector (3.5).

The primary research contribution is the derivation of a generic methodology for improving performance measurement systems, which is both rigorous and yet can be tailored to each individual situation.

As this case study involves action research, the impact activity frequently precedes publication. Research outputs containing evaluations of the interventions also serve as evidence of impact and are referenced in both section 3 and 5 of this case study.

References to the research

Details of the impact

The 3Es performance management methodology is designed to help users manage rapid growth and change. It was developed within three participating organisations during the study, and since been adopted by four further public and private sector organisations in China.

4.1 Chinese Academy of Sciences

The Chinese Academy of Sciences is the major Chinese state research organisation. It manages over 100 research institutes with over 50,000 staff, covering a wide variety of disciplines and approaches. CASREC (the Management Innovation and Evaluation Research Centre) is responsible for evaluating Chinese research and benchmarking it against international standards.

In 2009, the 3Es method was used in a major academic impact ranking project, using 86 national research institutes across 26 countries. The ranking results were used as a reference in a recent Chinese Academy of Sciences strategic performance assessment by the Chinese government. Top ten rankings were compiled across seven research areas and one example follows (Molecular Biology and Genetics) (5.1):

1. National Institutes of Health, US (NIH)
2. National Centre for Scientific Research, France (CNRS)
3. National Laboratories, Research Councils UK (RCUK)
4. Max-Planck Society for the Advancement of Science, Germany (MPG)
5. National Institute for Health and Medical Research, France (INSERM)
6. Helmholtz Association of German Research Centre, Germany (HFG)
7. Department of Energy, US (DOE)
8. Science and Technology Agency, Japan (JSG)
9. Rikagaku Kenkyujo, Japan (RIKEN)
10. National Research Council, Italy (CNR)

As the Professor of Institute of Policy and Management of CASREC wrote in a statement supplied:

"The methodology developed by Kent Business School is essential tool in benchmarking scientific research in China against world-class standards and helped us successfully complete a major project in ranking academic impact of worldwide research institutes"(5.2).

4.2 Hunan University: Foreign Languages School

Hunan is one of the most important and oldest universities in China with around 13,000 students. The College of Foreign Languages has 2,000 students and 50 members of staff. Its objectives in developing a performance management system included providing clearer lines of communication and helping the College expand internationally.

At Hunan, the project took the University mission and used it to develop lower level actions and key performance indicators. These formed the foundation of the institution's performance management system, launched in 2009.

As the Vice Dean of the Hunan College of Foreign Languages wrote in a statement supplied:

"We used the 3E method to deploys our overseas training and teaching strategies and then to develop performance measurements for these staff as to build a quite useful performance management system. We found that this system can increase two-way communications between the staff and managers, and motivate them to work better. Some positive effectiveness of this system is now gradually seen, while the staff's satisfaction rate to the college management is increasing. For instance number of our collaborative universities (in North America and Europe) has much increased, referring to our aims. The Chinese Ministry of Education has granted us as one of the national bases for overseas training and teaching." (sic) (5.3)

4.3 Tonsan Adhesives

Tonsan Adhesives Inc is a Beijing-based high-tech enterprise that operates internationally and specialises in developing and producing engineering adhesives. It offers several hundred products across seven categories. Its objective in adopting the 3Es performance methodology was to aid strategic planning, during a period of rapid growth.

In this intervention, the research team developed the `Tonsan Performance Management System' for its senior management team during 2009.

The research process resulted in a fully-documented, six-step performance management cycle including the architecture of the 1) performance appraisal indicator system, 2) performance planning set up, 3) performance tracking and guidance, 4) performance appraisal, 5) appraisal result feedback and 6) the uses and requirements of the appraisal results.

By the end of February 2010, Tonsan's profit had increased by 50%, despite the economic crisis, and the company had expanded manufacturing operations to the Shanghai area. Executives believed that the new performance management system was a significant help in achieving this.

As the CEO of Tonsan, commented:

"The most valuable thing of the system is to make manager and staff all plan pro-actively how to achieve their targets step by step so that everyone is engaged. This provides the foundation for our amazing growth." (5.4)

4.4 Further Developments

Through these three interventions, the research team developed a generalised performance management system that especially suitable for organisations with varied or individual missions.

The team is now developing versions of the 3E performance management system for the following Chinese organisations including Xingxiang Insurance (since 2010), Sun-rivier local government (since 2012), Liao-ling Plaze hotel (since 2012 - Contact no.3)) and market leading flavours and fragrances manufacturer, Huabao International Holdings (since 2012).

Each organisation is using the 3E system to meet a variety of management challenges, including management and assessment of staff performance. The CEO of Huabao wrote in a statement:

`Haubao found the system and 3E performance indicators that they established very useful....The new system allows managers and staff to discuss how to achieve their targets during the planning process, and assess performance of staff fairly. This has much contributed to the fast growth of our company recently.' (5.5)

Sources to corroborate the impact

References 3.4, 3.5 and 3.6 also corroborate the impact of this action research project.

5.1 Appendix 4, supplementary data : F. Xu, X. X. Li, W. Meng, W. B. Liu, and J. Mingers (2013)' Ranking academic impact of world national research institutes—by the Chinese Academy of Sciences' Research Evaluation (DOI: 10.1093/reseval/rvt007)

5.2 Statement from the Chinese Academy of Sciences (corroborating contact No. 4)

5.3 Statement from the Hunan Foreign Languages School (corroborating contact No. 2)

5.4 Comment by the CEO of Tonsan Adhesives Ltd (corroborating contact No. 5)

5.5 Statement from HuaBao Ltd (corroborating contact No. 1)