Summary of the impact

Lack of interoperability can leave educational material, e-portfolios and course information locked into proprietary systems. This not only limits freedom of choice, it also blocks collaboration. IEC's work in educational technology interoperability standards has addressed this issue internationally by leading on the development of open formats for educational material (IMS Content Packaging), online assessment material (IMS Question and Test Interoperability), e-portfolios (Leap2a) and course information (eXchanging Course Related Information). The adoption of these specifications means that students can move their e-portfolios, course information can be shared, educational content can be exchanged between VLEs, and large assessment infrastructure projects are enabled.

Underpinning research

IEC has conducted research in the interoperability field by building on two different roles: stewarding the development of national specifications by the Further and Higher Education community within the UK, and leading on the development of specifications in international bodies. The work has been largely funded by Jisc, either directly with support for specific projects (as was the case with XCRI), or through the general brief of the Centre for Educational Technology and Interoperability Specifications (Cetis). Cetis has been funded as an innovation support service throughout the period covered by the REF, with a particular focus on interoperability. This provided a sustained flow of funding with which to carry out a coordinated programme of research and development in educational technology interoperability. In this work the IEC consistently deployed a community led methodology, and developed extensive insight and expertise into how this should most effectively be managed, through a series of cycles of action and reflection. The methodology and its achievements was discussed by IEC member Wilson in the International Journal of IT Standards and Standardization Research [1]. Note that this paper is submitted as part of Wilson's outputs in UOA36, where the bulk of his research is better situated.

As part of its work as an innovation support centre Cetis was tasked by Jisc to represent UK Higher Education on international bodies such as IEEE and IMS Global Learning Inc. (IMS), whose members include Blackboard, ETS, IBM, Pearson and Elsevier. This provided a context within which the impact of the research carried out could be maximised.

The research focused around a number of specific projects, of which the four most significant are:

Research and development of national specifications

• XCRI (eXchanging Course Related Information). The need for XCRI [2] emerged from on-going Cetis work with representatives of higher education institutions, who identified a need for a format with which to exchange course related information. The IEC, through Cetis, set up and facilitated a group of university information services representatives who defined a format for the exchange of course information. IEC contributed its expertise in the formulation and design of interoperability specifications, in particular through the role of IEC member Scott Wilson who was the XCRI Technical Consultant and took a leading role in authoring the specification. Wilson conceived the strategy of developing XCRI as a conforming binding of EN 15982, Metadata for Learning Opportunities, and within this context it is referred to as BS 8581. The IEC also designed implemented and maintained an organisational home. We extended our work to make contributions to related international efforts, such as the CEN (Comité Européen de Normalisation), where XCRI informed the development of the Metadata for Learning Opportunities (MLO) standard. IEC also contributed by researching XCRI aggregation and validation software tools, which informed the design and further implementation of the specification.
• Leap2A. Leap2A [3] is another example of community led development, in which the role of IEC was similar to that taken in XCRI. This involved the development methodology, facilitation of a group that included the main vendors of e-portfolio systems on the market, and strong contributions to the authoring of the specification. A leading role was taken by IEC member Simon Grant. This included experience of earlier work on IMS E-portfolio, and an analysis of how this related to the Leap2a initiative. Again, IEC also provided the group's infrastructure and facilitation, see, for example, [4]. A critical contribution was our research in balancing the often competing requirements of various stakeholders in the process of agreeing which features to include or exclude while still delivering an effective specification.

Research and development of international specifications:

IMS Content Packaging. This specification [5] responds to the need of educational content providers to deliver collections of aggregated content to Virtual Learning Environments with confidence that it will display and function correctly. IEC led the working groups within the IMS consortium that specified the last two versions, 1.1.4 and 1.2., and IEC member Kraan was an editor of the specification. A particular contribution was in the area of content aggregation practice, and software development research. This contribution was recognised when Kraan received an Award for Specification Leadership from IMS at their Learning Impact conferences in 2008 and 2009. Most of the research was done in a series of projects that led to the development of the Reload content packaging tool, as well as a series of interoperability testing events. The research outcomes of both activities were used directly to inform the development of the specification. Some of these activities took place before 2008, but the Content Packaging specification has been elevated to the status of a de jure standard of the International Standards Organisation (ISO) in 2009 - 2012.

IMS Question and Test Interoperability. QTI [6] is the world's only open interoperability format for computer aided assessment. The latest version of the specification was in beta for a number of years, and the effort was in danger of collapse until IEC intervened in 2010. This effort was led by Wilbert Kraan, who was both an editor of the specification as well as a leading contributor to the information model, the specification overview, and the implementation guide. IEC's leadership and advocacy ensured continued research and development of the specification, and its eventual release in the spring of 2013. This contribution was recognised when Kraan received an Award for Specification Leadership from IMS at their Learning Impact conferences in 2011 and 2012. As part of this effort IEC also organised projects and events that enabled others to both develop prototype implementations of the QTI specification, and to test interoperability between these implementations. As with Content Packaging, the outcomes of this research informed the design of QTI as it was being written.

In all cases, IEC's research in data modelling, interface design, as well as practical research into the use of technology in UK Higher and Further Education proved crucial to the design and development of these specifications, as is demonstrated by the references below. This, approach, in turn, facilitated the adoption and subsequent impact of the specifications.

References to the research

Details of the impact

The impact of interoperability specifications occurs at the adoption stage, not at the stage of writing the specifications themselves. For that reason, IEC's interoperability research has adopted a strategy in the development of specifications that centres on adoption, as described in [1] above. This involves understanding the requirements of software vendors from the start, both to make sure their requirements are being considered, as well as to ensure they have a stake in the emerging specification. Another characteristic of this approach was to rigorously and iteratively ensure that the formulation of the specification was as simple as possible, while still remaining effective, in order to make implementation as easy as possible. Also important was the re-use of existing, widely used interoperability specifications as a basis for newer agreements, to further minimise implementation effort. Finally, a deep understanding of the needs of end-users has been required in order to make adoption by the vendors worthwhile, by generating demand. This approach achieved the following results:

Course Related Information (XCRI). In this case the involvement of conventional software vendors was less relevant, since XCRI's principal application is internal to the institution which adopts it. The end-user community is therefore largely the same as the implementer community, and it was they who drove both development and adoption. A key intervention of the IEC was to maximise the simplicity of the specification, which ensured that XCRI is easy to implement. This has led to high levels of impact, and our research indicates that over 100 institutions have adopted XCRI in some form, many of whom are included in the XCRI-CAP directory, maintained by the company InGenius Solutions [7]. A notable adopter is Skills Development Scotland, who have implemented a method for bulk import of courses into the National Learning Opportunities Database which depends on the use of XCRI [8]. Further details of adoption are available in the Case Studies presented on the XCRI Knowledge Base [9]. Our research estimates that Higher Education Institutions in the UK have saved £1.5m to £2.8m per year in course data rekey-ing, either in outright savings or else in deploying budgets for higher level marketing purposes. Looking at external use, the national XCRI aggregator (http://xxp.igsl.co.uk/app/xcridirectory) currently has feeds from eighty-two institutions. The XCRI specification is now also aligned to national and international standards. RDF vocabularies for XCRI are part of Linked Universities http://linkeduniversities.org/lu/index.php/vocabularies/. It has been also been used as a model for data in institutions, for example in the data sets for https://data.ox.ac.uk/ where it is augmented with things like geographical data for buildings, departments etc.

Leap2A. This specification involved vendors as well as end-users from the start. The methodology led by IEC resulted in a simple and effective specification, and also ensured that it was based on the widely used IETF Atom feed format, which both aids implementation and usage in tools that weren't designed specifically for Leap2A. The impact of the specification on the ePortfolio market has been considerable. As is shown in [7], many leading providers of ePortfolio tools have adopted Leap2A. The result is that the specification is integrated in nearly all e-portfolio installations in UK Further and Higher Education institutions (42% of all such institutions have an e-portfolio system), and many similar markets.

The IMS/ISO Content Packaging (CP). This standard followed software design patterns from large vendors such as Microsoft and Sun (now Oracle). More specialised e-learning vendors such as Blackboard and Moodle have been more active in the design of profiles of Content Packaging such as Common Cartridge and SCORM. IEC contributed substantial research into educational resource sharing practices from Jisc programmes in the UK, as well as technology research from interoperability events into the design of the specification. This ensured that the specification met the needs of UK Higher Education institutions. The high degree of impact of the specification can be seen from IMS Interoperability Conformance Certification Status, [11], which shows that almost all learning platform providers have adopted the specification. Content Packaging is also a component of the widely adopted SCORM application profile, and details of adoption of SCORM are available at [12]. The result is that between 90 and 99% of Higher and Further Education Institutions in the UK have Virtual Learning Environments that can import and export educational Kcontent or whole courses using some variant of IMS CP. The global picture is similar. Content Packaging was adopted as an international standard by ISO in 2009, further cementing the role of the specification within the educational technology landscape [13].

IMS Question and Test Interoperability (QTI). This specification involved some large assessment boards in the early design stages, although they participation dropped off as the project developed. IEC helped organise and fund a community of end-users who were able to develop implementations of the specification as it was being designed. This type of technology action research has proven crucial in the development of QTI as well as other specifications. QTI is not a simple specification, since the requirements on computer aided assessment are complex and varied, but it is built on a very widely implemented standard: HTML, the document language of the web. This, as well as QTI's position as the world's only open assessment interoperability specification, has led an enduring impact on the field. The QTI specification was always the most popular download from the IMS website, as was IEC's briefing on QTI on CETIS' website. According to Pearson "QTI is the most widely used and adopted assessment interoperability standard in the assessment industry" [14]

More recently, IEC has calculated that the specification has been instrumental in the design of a series of assessment infrastructure projects in six different nations worth a total of at least £250 million. As a result of US federal government investments in particular, most major US publishers are now standardising their own internal systems and item and test collections on IMS QTI, ensuring that the impact of the specification will continue for the foreseeable future.

Sources to corroborate the impact

[7] inGenious Solutions Ltd. (2011). XCRI eXchange Platform (XXP). Retrieved from:
http://xxp.igsl.co.uk/app/xcridirectory [Accessed July 8, 2013].

[8] Skills Development Scotland (2013) Bulk import to the National Learning Opportunities Database. Retrieved from http://www.providercentral.org.uk/OurServices/BulkImport/Bulk_import_facility.aspx [Accessed November 5th, 2013]

[9] XCRI Knowledge Base (2013). Case Studies. Retrieved from http://www.xcri.co.uk/h2-case-studies-article.html [Accessed July 8, 2013]

[10] leapspecs.org, 2013. Who is Using Leap2A. Leap2A eportfolio portability. Available at:
http://www.leapspecs.org/2A/who-is-using-leap2a [Accessed July 8, 2013]

[11] IMS Global Learning Consortium, 2013. IMS GLC: CC/LTI Conformance Directory. Available at: http://imsglobal.org/cc/statuschart.cfm [Accessed July 8, 2013]

[12] U.S. Government © Advanced Distributed Learning, 2012. Advanced Distributed Learning Initiative: SCORM Certification. Available at: http://www.adlnet.gov/scorm/scorm-certification [Accessed July 8, 2013]

[13] ISO/IEC 12785-1:2009, Information technology -- Learning, education, and training -- Content packaging -- Part 1: Information model. Retrieved from
http://www.iso.org/iso/home/store/catalogue_tc/catalogue_detail.htm?csnumber=51707

[14] Dolan, B., Strain-Seymour, E., Deokar, A., Ostler, W. (2010). Next-Generation Assessment Interoperability Standards, a white paper from Pearson. Retrieved from
http://www.pearsonassessments.com/NR/rdonlyres/80F6D77A-94E3-4225-9392-CB43E96616D0/0/AssessmentInteroperabilityStandards_FINAL_111710.pdf [Accessed November 5, 2013]

[15] Kraan, W., 2013. QTI 2.1 spec release helps spur over £250m of investment worldwide. Wilbert's work blog. Available at: http://blogs.cetis.ac.uk/wilbert/2013/05/03/qti-21-spec-release-helps-spur-over-250m-of-investment-worldwide/ [Accessed July 8, 2013].