Open ECBCheck

Open ECBCheck is a low cost accreditation and quality improvement scheme for E-learning Programmes and institutions in international Capacity Building. It supports capacity building organizations to measure how successful their e-learning programmes are and allows for continuous improvement through self-assessment, peer collaboration and benchmarking. The Open ECBCheck provides a quality label valid for three years, after this period program and institution can re-apply in order to re-obtain the label. Certified institutions and programs also commit in reporting improvements made after having received the certification.

Open ECBCheck has been developed through a participative process initiated by InWent. This interagency collaboration involves about 25 international capacity building agencies (among which 8 UN agencies). ^[1]

Criteria

The Open ECBCheck is based on a self-assessment exercise followed by a peer-review performed by external agencies (see Process for more information). Both the self-assessment and the peer-review are guided by a set of criteria. The label has two different sets of criteria for the evaluation of a programme or an institution.

Set of Criteria for a Program / Course Certification:

The program/course quality criteria are divided into seven main areas:

1. Information About and Organization of the E-Learning Programme is related to the information provided about the program for (potential) learners and the general organization of the programme.
2. Target Group Orientation highlights the importance of the learners’ perspective in the various phases of a program, from the development to the evaluation.
3. Quality of the Contents is related to the quality of the material provided.
4. Didactical Design focuses on the learning design and on the quality of the learning experience.
5. Media Design is related to the accessibility and usability of the media provided to learners.
6. Technology: Equipment and Infrastructure is concerned with the support that technology provides to the learning experience.
7. Evaluation and Review comprises the factors that ensure the continuous improvement of a program / course.

Set of Criteria for an Institutional Certification

The institution quality criteria are organised into four main areas:

1. Education & Learning is concerned with the quality of factors connected with the experience of the learners (resources available, support provided by the organisation).
2. Organizational Strategy and Innovation is related to the way innovation is fostered and how the quality of elearning are embedded into the institutional strategy.
3. Organization Processes concentrates on the quality and documentation related to elearning that the organization created and provides to the audience.
4. Technology: Equipment & Infrastructure focuses on the way technology supports the learning experience.

Methods

The Open ECBCheck label makes use of 5 central methods for quality evaluation and validation:

1. Benchmarking: Systematic and continuous process to evaluate products, services or processes of an organization by comparing them with the recognized market leaders.
2. Benchlearning: Process of learning from the "best in the class", identifying the best practices available and integrating them in the organization.
3. Self-assessment: Analysis of the strengths and weaknesses of a company or organisation to identify areas of improvements and determine one position in relation to the others.
4. Peer-Review: External evaluation of an organization or program by expert that belong to a different organisation.
5. Qualitative Weighting and Summation: Method for evaluation based on criteria catalogue. Each criterion is attributed a weight according to its importance. The criteria are afterwards rated and in the final step the product of rating and correspondent weighting indicates the final score.

Process

The Open ECBCheck process is structured into seven distinct steps in which a course or a programme is analysed and evaluated. Applying programmes or institutions benefit from a clear guidance to structure their self-assessment and a comprehensive learning report, result of the peer-review process, which outlines their improvement potential.^[2] The steps involved in the Open ECBCheck process are:

0. Inquiry: Organization interested in obtaining the certification either for the whole institution or for a specific program or course contact the Open ECBCheck Administration Office (info[at]ecb-check.org).

1. Application: The applying institute fills in a questionnaire providing basic information about the organization itself and / or the specific program that is candidate for a certification.

2. Eligibility Check: The Open ECBCheck Administration Office evaluates the application and decides on the eligibility of the candidature. If the institution or programme is not considered to be eligible, the reasons for this decision are provided. If the institution or programme is considered eligible, the process can be taken further.

3. Self-Assessment: In this phase the institution needs to conduct an evidence-based self-assessment of either the institution itself or the program. This assessment is conducted with the help of a toolkit (see documentation), which provides all the guidance needed in terms of evaluation of the criteria and proofs needed. Once the self-assessment is completed, it can be submitted to the Open ECBCheck Administration Office. The self-assessment has to purposes:

• It is the foundation of the decision if a label will be granted to the institution / program.
• It has a learning function as it provides the institution with a possibility for identifying areas of improvements.

4. Peer-Review: The peer-review is conducted by two different organizations, selected by the Administrative Office, among the list of organizations members of the community. The Administrative Office takes into account that there are not conflicting interests. Each one of the reviewer has to evaluate the self-assessment provided by the organization, on the basis of three criteria:

• Clarity of the information: Is the provided information clearly understandable?
• Comprehensiveness of the information: Is all the necessary information been provided to support the self-assessment rating?
• Validity of the information: Is the rating reasonable in consideration of the proof provided?

5. Peer-Review Report: The peer-reviewers issue a report, which contains three main areas:

• Summary of the self-assessment, which includes all the areas where the reviewer disagreed with the auto-evaluation made by the institution.
• Recommendation, whether the institute should or should not be awarded with the certificate.
• Learning report which highlights the reason of the recommendation and contains suggestions for improvements.

The peer-review is submitted to the Administrative Office.

6. Awarding Body Decision: The awarding body will evaluate the anonymous peer-review report and decided whether the Open ECBCheck label will be granted. If the decision is negative, the Awarding Body informs the institute about reason for this decision and provides the learning reports for possible improvements. It is possible to re-apply for the label after one year from the first application.

7. Open ECBCheck Label: If the decision of the Awarding Body is positive, the Open ECBCheck label will be granted to the institution / program.

8. Reporting on Results: The Organization needs to report on achievements and advices based on the learning report.

9. Re-application after 3 years: The label is valid for three years, after which there is a possibility to re-apply for the certificate.

Documentation

‎Criteria Overview

Self-Assessment Tool for Courses

Self-Assessment Tool for Programmes

[http://www.efquel.org/openecbcheck/images/stories/docs/OpenECB-Check_manual.pdf Open ECBCheck Manual