Open ECBCheck

Open ECBCheck is an accreditation and quality improvement scheme for E-learning Programmes and institutions in international Capacity Building. This low cost framework supports capacity building organisations to measure how successful their e-learning programmes are and allows for continuous improvement through self-assessment, peer collaboration and benchmarking. The Open ECBCheck provides a quality label valid for three years, after this period programs and institutions can re-apply in order to re-obtain the label. Certified institutions and programs also commit to reporting improvements made after having received the certification.

Open ECBCheck has been developed through a participative process initiated by InWent. This interagency collaboration involves about 25 international capacity building agencies (among which 8 are UN agencies). ^[1]

Criteria

The Open ECBCheck is based on a self-assessment exercise followed by a peer-review performed by external agencies (see the certification process).

Both the self-assessment and the peer-review are guided by a set of criteria. The label has two different sets of criteria for the evaluation of a programme or an institution.

Set of Criteria for a Program / Course Certification

The program/course quality criteria are divided into seven main areas:

1. Information About and Organization of the E-Learning Programme: relates to the information provided about the program for (potential) learners and the general organisation of the programme.
2. Target Group Orientation: how the learners’ perspective is considered in the various phases of a program, from the development to the evaluation.
3. Quality of the Contents: relates to the quality of the material provided.
4. Didactical Design: focuses on the learning design and on the quality of the learning experience.
5. Media Design: relates to the accessibility and usability of the media provided to learners.
6. Technology - Equipment and Infrastructure: deals with the support that technology provides to the learning experience.
7. Evaluation and Review: comprises the factors that ensure the continuous improvement of a program / course.

Set of Criteria for an Institutional Certification

The institution quality criteria are organised into four main areas:

1. Education & Learning: deals with the quality of factors connected with the experience of the learners (resources available, support provided by the organisation).
2. Organizational Strategy and Innovation: relates to the way innovation is fostered and how the quality of elearning is embedded into the institutional strategy.
3. Organization Processes: concentrates on the quality and documentation related to elearning that the organisation created and provides to the audience.
4. Technology: Equipment & Infrastructure: focuses on the way technology supports the learning experience.

Methods

The Open ECBCheck label makes use of 5 central methods for quality evaluation and validation:

1. Benchmarking: Systematic and continuous process to evaluate products, services or processes of an organisation by comparing them with the recognized market leaders.
2. Benchlearning: Process of learning from the "best in the class", identifying the best practices available and integrating them in the organisation.
3. Self-assessment: Analysis of the strengths and weaknesses of a company or organisation to identify areas of improvement and determine one position in relation to the others.
4. Peer-Review: External evaluation of an organisation or program by experts that belong to a different organisation.
5. Qualitative Weighting and Summation: Method for evaluation based on a criteria catalogue. Each criterion is attributed a weight according to its importance. The criteria are afterwards rated and in the final step the product of rating and correspondent weighting indicates the final score.

Process

The Open ECBCheck process is structured into seven distinct steps in which a course or a programme is analysed and evaluated. ^[2] The steps involved in the Open ECBCheck process are:

0. Inquiry: An organisation interested in obtaining the certification either for the whole institution or for a specific program or course contacts the Open ECBCheck Administration Office (http://www.ecb-check.net/).

1. Application: The applying institute fills in a questionnaire providing basic information about the organisation itself and / or the specific program to be considered for certification.

2. Eligibility Check: The Open ECBCheck Administration Office evaluates the application and decides on its eligibility. If the institution or programme is not considered to be eligible, the reasons for this decision are provided. If the institution or programme is considered eligible, the process can be taken further.

3. Self-Assessment: In this phase the institution needs to conduct an evidence-based self-assessment of either the institution itself or the program. This assessment is conducted with the help of a toolkit (see documentation), which provides all the guidance needed in terms of evaluation of the criteria and proof needed. Once the self-assessment is completed, it can be submitted to the Open ECBCheck Administration Office. The self-assessment has two purposes:

• It is the foundation of the decision if a label is granted to the institution / program.
• It has a learning function as it provides the institution with a possibility for identifying areas of improvement.

4. Peer-Review: The peer-review is conducted by two different organisations, selected by the Administrative Office. The Administrative Office takes into account that there are not conflicting interests. Each reviewer has to evaluate the self-assessment provided by the organisation, on the basis of three criteria:

• Clarity of the information: Is the provided information clearly understandable?
• Comprehensiveness of the information: Has all the necessary information been provided to support the self-assessment rating?
• Validity of the information: Is the rating reasonable in consideration of the proof provided?

5. Peer-Review Report: The peer-reviewers issue a report, which covers three main areas:

• Summary of the self-assessment, which includes all the areas where the reviewer disagreed with the auto-evaluation made by the institution.
• Recommendation, whether the institute should or should not be awarded the certificate.
• Learning report which highlights the reason of the recommendation and contains suggestions for improvement.

The peer-review is submitted to the Administrative Office.

6. Awarding Body Decision: The awarding body will evaluate the anonymous peer-review report and decide whether the Open ECBCheck label will be granted. If the decision is negative, the Awarding Body informs the institute about the reason for this decision and provides the learning reports for possible improvement. It is possible to re-apply for the label after one year from the first application.

7. Open ECBCheck Label: If the decision of the Awarding Body is positive, the Open ECBCheck label will be granted to the institution / program.

8. Reporting on Results: The organisation receiving the certification commits to reporting on the achievements and improvements implemented in accordance with the learning report.

9. Re-application after 3 years: The label is valid for three years, after which there is a possibility to re-apply for the certificate.

Documentation

‎Criteria Overview

Self Assessment Tool

Open ECBCheck Manual