Summary of the impact

Research conducted between 1997 and 2013 at the University of Edinburgh has examined qualifications reforms designed to make education and training systems more coherent, unified and flexible. This research has influenced policy developments in Scotland and internationally, and especially the development of National Qualifications Frameworks (NQFs). It has achieved three types of impact: i) analytical tools and conceptual frameworks developed in the research have informed and guided policy development; ii) it has encouraged greater realism in the aims and objectives of qualifications-led reforms; and iii) it has encouraged policy designs and implementation strategies based on a better understanding of the processes of qualifications-led change. The reach of the impact has extended to policy-makers, education providers, learners and other stakeholders in Scotland, the UK and elsewhere, especially in countries introducing or considering NQFs.

Underpinning research

The underpinning research consists of three overlapping strands:

• Research on the conceptualisation, implementation and impact of reforms to `unify' education and training systems, with a particular (and increasing) focus on qualification frameworks. This research exploited Scotland's pioneering role in many of these policy initiatives, but variously had a UK, European or international focus. It included projects funded by the ESRC (1996-98, 2000-03, 2010-11), EU-Leonardo (1995-97) and the Scottish Government (2004), as well as participation in international projects, eg International Labour Organization (ILO) (2009-10).
• Studies of aspects of qualifications such as unitisation, records of achievement, credit systems, baccalaureates, work-related awards and the articulation of sub-degree and degree-level qualifications, funded by the Scottish Qualifications Authority (SQA) and other Scottish and European policy organisations (eg the European Centre for Development of Vocational Training (CEDEFOP) and the German Institute for Vocational Training (BIBB)).
• Longitudinal analyses of youth transitions, including ESRC projects (1997-2000, 2003-06), which have tested hypotheses arising from the first strand above.

The research has been conducted primarily by Linda Croxford, Cathy Howieson and David Raffe, Senior Research Fellows and Professor respectively, employed in the University of Edinburgh's Centre for Educational Sociology continuously from 1993, and by Stephanie Allais (ESRC-funded postdoctoral fellow Jan-Dec 2010) and John Hart, CES Associate since 2003. It includes collaborations with the University of London Institute of Education, Glasgow Caledonian University and international organisations and projects as described in details of the impact below.

The key message of the research is the importance of the social and political aspects of qualifications and qualification systems, and the need to understand their design, implementation and impact in these terms. Reform approaches that treat qualifications and NQFs merely as technical instruments are inadequate. The research draws attention to the institutions and relationships which underpin qualifications: reforms need to maintain relations of trust and to respect the `institutional logics' of the educational, labour-market and social contexts which determine the ways in which qualifications are used and valued.

Research thus challenges inappropriate expectations about the capacity of qualifications reforms to transform an education system. It encourages more sophisticated models of change: an NQF may have radical aims but the process of introducing one needs to start from the existing system and proceed slowly, incrementally and iteratively. A comprehensive NQF may need to include different sub-frameworks in order to accommodate diverse types of learning and the varying dynamics of change in different sectors of education and training. The research also points to the need for `policy breadth', ie for qualifications policies to be part of a wider reform programme.

The research has shown that a `flexible' education and training system is neither easy to achieve in practice nor a panacea for problems of access, progression and relevance. A learning system can achieve greater permeability by designing bridges and pathways between programmes or sectors, but a fully flexible or seamless system in which qualifications provide a perfectly convertible currency is not attainable. `Parity of esteem' between different types of learning may, similarly, be an unrealistic and inappropriate aim of policy. A `unified system' of education and training, which brings different types and modes of learning into a single framework, needs to be conceived and designed as a means for coordinating diversity rather than for imposing uniformity. The research also shows how the role of qualification systems varies across national contexts; cross-national `policy borrowing' should be rejected in favour of `policy learning' approaches which recognise national distinctiveness.

References to the research

The quality of the research is evidenced by outputs in respected peer-reviewed journals, exemplified by the above, and by the sources of funding (such as ESRC, Scottish Government, EU) indicated in section 2.

Details of the impact

The beneficiaries have been policy-makers, providers, learners and other stakeholders in education and training in Scotland, the UK and elsewhere, especially in countries introducing or considering NQFs. Currently 142 countries have an NQF, are introducing one or are considering doing so. The research has:

• provided analytical tools and conceptual frameworks which have been used to inform policy strategy and development;
• encouraged greater realism in the aims and objectives of such reforms; and
• supported policy designs and implementation strategies based on a better understanding of the processes of qualifications-led change.

Analytical tools developed by the University of Edinburgh for analysing qualifications-led reforms have been widely cited in influential policy documents such as the ILO's Introductory Guide to NQFs [5.1], the most influential guide for policy-makers available throughout the REF period. These tools, and examples of citations, include:

• a typology of unified, linked and tracked education/training systems [5.1]
• the distinction between the intrinsic logic of a qualifications reform and the intrinsic logic of the system in which it is embedded [5.1]
• a typology of communications, reforming and transformational frameworks [5.1, 5.2, 5.3, 5.4, 5.5]
• the analysis of policy breadth, that is the coherence of qualifications reforms with other policies which may affect their implementation and impact [5.1, 5.3]
• an incremental, contextual model of NQF development [5.4, 5.6, 5.7]
• the distinction between outcomes-led and outcomes-referenced frameworks, which respectively incorporate radical and pragmatic approaches to the role of learning outcomes [5.4, 5.5]
• the distinction between policy learning and policy borrowing [5.2, 5.3].

These analytical tools accompany more substantive messages from the research, including the need to set realistic aims for qualifications reforms and to design and implement them in the light of the change processes they can stimulate. The impact of these messages is reflected in recent developments among NQFs; for example most European NQFs follow an incremental model of change, starting as modest communications frameworks but adopting more reforming aims over time and in specific sub-frameworks, are coordinated with other policy measures, and incorporate a pragmatic aproach to learning outcomes [5.4]. Individual countries, such as Turkey, have used the conceptual frameworks arising from the research in developing their frameworks [5.8]. The impact of the research is also reflected in the encouragement from international bodies (ILO, CEDEFOP and European Training Foundation) for rigorous monitoring and evaluation to assess the impact of reforms, and in their support for `policy learning' rather than `policy borrowing' approaches to cross-national diffusion. The impact is strengthened by the objectivity and independence of the team's research, especially as most information about NQFs is provided by qualifications bodies and agencies with a vested interest in their diffusion and perceived success.

Within Scotland the research showed that earlier reforms (eg Higher Still) had extended opportunities for all learners, but it also revealed their limitations. It identified the problems of applying a common template to diverse qualifications, the difficulties of constructing a progression `climbing frame' and the obstacles to achieving `parity of esteem'. New Scottish secondary-level qualifications designed during the REF period apply these lessons, for example by adopting desigh principles based on fitness-for-purpose and a higher priority for progression [5.9].

These impacts have been achieved, in part, through conventional written and oral dissemination. The research has informed guides and briefings for international policy-makers which have been influential during the REF period (see citations above). The Centre for Educational Sociology has published three Briefings on National Qualifications Frameworks since 2008, one of which was posted on a CEDEFOP discussion web site, and a fourth on credit systems. The researchers have given invited presentations at meetings and conferences on qualifications reforms organised by national and international policy organisations, including the European Training Foundation (2008), Scottish Credit and Qualifications Framework Partnership (2008), German Government (2008), ILO (2009, 2012), CEDEFOP (2009), European Commission (2010, 2011, 2012), BIBB (2010), SQA (2008, 2010), UK Qualifications Frameworks (2010), German Government/Asia- European Meeting (2012), Quality and Qualifications Ireland (2012) and the National Council on Education of Chile (2013).

The research has had a multiplier effect as University of Edinburgh staff have been invited to join international projects led by policy-making bodies themselves; this has helped both to disseminate the Centre for Educational Sociology knowledge base and to add to it. Allais and Raffe, with Michael Young (IoE), formed the core academic team and wrote the conceptual starter papers for the ILO's influential 16-country study of NQFs (2009-10); Allais wrote its final report. Raffe was international member of the National Qualifications Authority of Ireland's study of the implementation and impact of its NQF (2008-09), and participated in a CEDEFOP study of Changing Qualifications (2008-10). Howieson, with Raffe, conducted the Scottish component of a BIBB project on credit systems (2009-12).

The impact on Scottish qualifications is further attested by commissions as consultants for the Scottish Qualifications Authority (SQA). These have drawn on Centre for Educational Sociology research to inform SQA policy in areas including unitisation (2004), progression (2005), the design of baccalaureates (2007-8), work-related qualifications (2007) and upper-secondary qualifications (2009-10), in all cases drawing on earlier research by the team to inform policy developments during the REF period.

Other channels of impact include invitations to comment on drafts of international policy reports (eg OECD 2007, ETF 2010, 2011); Raffe's membership of the SQA's Qualifications Committee (2003 onwards); being interviewed as `experts' by other research projects on credit (2009), transnational qualifications frameworks (2010), the European Qualifications Framework (2011) and modularisation (2012); and discussing NQFs with visiting policy-makers and policy analysts from Australia, China, Japan, Russia, South Africa and Spain.

John Hart, Associate of the Centre for Educational Sociology and former SQA official, has supported impact through his Scottish networks and international consultancy work. During the current REF period this has included presentations or consultancy on NQFs in Albania, Croatia, Germany, India, Kazakhstan, Kosovo, Macedonia, Poland and Turkey, support for referencing the Dutch, Maltese, Scottish and Welsh Frameworks to the European Qualifications Framework, and support to NHS Education Scotland on competence-based frameworks. In several of these countries (especially India, Kosovo, Macedonia and Turkey) he has used the analytical tools in the development of policy papers and legislation which provide the basis for the introduction of NQFs.

Sources to corroborate the impact

Sources to corroborate the impact are indicated in square brackets in the text. Web pages have been archived at https://www.wiki.ed.ac.uk/display/REF2014REF3B/UoA+25

[5.1] Tuck, R. (2007) An Introductory Guide to NQFs: conceptual and practical issues for policy-makers. Geneva: International Labour Office. http://www.ilo.org/wcmsp5/groups/public/---ed_emp/---ifp_skills/documents/instructionalmaterial/wcms_103623.pdf

[5.2] Maguire, B. (2010) Issues arising from qualifications frameworks in Europe. Issues paper for conference on NQFs and the European Overarching Frameworks. Dublin: NQAI. http://www.nqai.ie/documents/IssuesarisingfromqualificationsframeworksinEurope-Final.pdf

[5.3] European Centre for Development of Vocational Training working paper (CEDEFOP) (2011). National Qualifications Frameworks developments in Europe. Thessaloniki.
http://www.cedefop.europa.eu/EN/Files/6112_en.pdf

[5.4] CEDEFOP working paper (2013) Analysis and overview of NQF developments in Europe. Thessaloniki. http://www.cedefop.europa.eu/EN/Files/6117_en.pdf

[5.5] European Parliament report (2012) State of play of the European Qualifications Framework Implementation. Brussels.
http://www.europarl.europa.eu/meetdocs/2009_2014/documents/cult/dv/esstudyeurqualifframewimplem/esstudyeurqualifframewimplemen.pdf

[5.6] Education Training Foundation report (2010) Trans-national Qualifications Frameworks. Turin: Education Training Foundation.
http://www.etf.europa.eu/webatt.nsf/0/720E67F5F1CC3E1DC125791A0038E688/$file/Transnational%20qualifications%20frameworks.pdf

[5.7] Education Training Foundation report (2011) Implementation arrangements for national qualifications frameworks and the role of stakeholders and institutions. Turin: Education Training Foundation.
http://www.etf.europa.eu/webatt.nsf/0/08EA60DCD9C98168C125791A002FE9D9/$file/NQF.pdf

[5.8] Factual statement of 21 February 2013 on behalf of Vocational Qualifications Authority of Turkey, inviting Raffe to give keynote to conference launching Turkish Qualifications Framework.

[5.9] Scottish Qualifications Authority (n.d.) New Qualifications: Principles and Guidelines.
http://www.sqa.org.uk/sqa/58431.3590.html