Most Significant Change (MSC)

• It is participatory because many project stakeholders are involved both in deciding the sorts of change to be recorded and in analyzing the data.
• It is a form of monitoring because it occurs throughout the program cycle and provides information to help people manage the program.
• It contributes to evaluation because it provides data on impact and outcomes that can be used to help assess the performance of the program as whole.

Like monitoring, MSC provides ongoing data about program performance that assists program management. But MSC goes further than most conventional forms of monitoring in that it also focuses on outcomes and impact, involving people in making judgments about the relative merits of different outcomes in the form of MSC stories. In this way, MSC contributes to both monitoring and evaluation.

MSC also has the potential to influence organizational learning within an organization, and maybe even within its associated stakeholders. The horizontal dimension is between a group of participants engaged in discussing and selecting the most significant of a set of stories. Vertical dialogue involves exchanges of views between groups of participants at different levels. The vertical dimension is very important if the MSC process is to aid organizational learning throughout the organization. It depends on good documentation and communication of the results of one group’s discussion to the next.

The process pf MSC involves the collection of significant change (SC) stories emanating from the field level, and the systematic selection of the most significant of these stories by panels of designated stakeholders or staff.

The kernel of the MSC process is a question along the lines of:

• Looking back over the last month, what do you think was the most significant change in [particular domain of change]? or
• From among all these significant changes, what do you think was the most significant change of all?

For a successful MSC implementation, it is necessary to create and facilitate the following contexts:

• An organizational culture where it is acceptable to discuss things that go wrong as well as success
• Champions (For example, people who can promote the use of MSC) with good facilitation skills
• A willingness to try something different
• Time to run several cycles of the approach
• Infrastructure to enable regular feedback of the result to stakeholders
• Commitment by senior managers

Implementation steps of MSC technique:

1. Starting and raising interest
2. Defining the domains of change
3. Defining the reporting period
4. Collecting SC stories
5. Selecting the most significant of the stories
6. Feeding back the results of the selection process
7. Verification of stories
8. Quantification
9. Secondary analysis and meta-monitoring
10. Revising the system

Step By Step^[2]

Step 1: Starting and raising interest

• If you are the person attempting to initiate the adoption of MSC, it may help to use one of the following metaphors to explain it:
  • Newspaper

    Newspapers are structured into different subject areas in the same way that MSC uses domains.

  • School of fish

    MSC helps the individual fish to communicate with each other and to swim in the same direction, away from what is not good and towards what is good.

  • Holiday memories

    MSC helps teams of people focus on the memorable events and uses these events to help realign effort towards achieving more of the wonderful things and less of the terrible things.

  • Restaurant menu

    MSC does not present one version of what is happening but a series of glimpses of what a program is achieving. Stakeholders can select from these glimpses in much the same way as they would select food from a restaurant menu.

• Start small. It is a risky exercise to implement a huge and complicated MSC system without first piloting it on a smaller scale.
• Identify key people (champions) who are excited by MSC. These champions can:
  • Excite and motivate people
  • Answer questions about the technique
  • Facilitate selection of SC stories
  • Encourage people to collect SC stories
  • Ensure that feedback occurs
  • Ensure that the stories are collected and organized and sent to review meetings
  • Develop protocols to ensure confidentiality where necessary

Using domains of change helps organizations to group a large number of SC stories into more manageable lots, which can each be analyzed in turn. The “any other type of change” domain is a useful open category that allows participants to report significant changes that don’t fit into the named domains.

Between three and five domains is a manageable number. The limiting factor is how much time participants are willing to spend in discussing each domain.

Domain can be identified before SC stories are selected or afterwards by sorting SC stories into meaningful groups. This depends on the extent to which the organization wants to be open to new experiences rather than continuing to be guided by past experiences.

The frequency of collection of SC stories has varied from fortnightly to yearly. Each organization has to make its own decision about the most appropriate reporting period, balancing the costs and benefits involved, and taking into account the reporting gaps that any existing monitoring and evaluation systems may be ignoring.

Our experience suggests that organizations tend to start MSC with more regular reporting and decrease the frequency as the process continues.

Step 4: Collecting SC stories

How to capture SC stories:

• Fieldworkers write down unsolicited stories that they have heard
• By interviewing and note-taking
• During group discussion
• The beneficiary writes the story directly

Information to be documented should include:

• Information about who collected the story and when the evens occurred
• Description of the story itself – what happened
• Significance (to the storyteller) of events described in the story (key part of MSC)

Most MSC stories are a page or less in length, with some being up to two pages. Shorter MSC stories are quicker and easier to read, but they should not be so short that vital information is left out.

When a storyteller tells a story, the person collecting the story needs to explain how the story is to be used and to check that the storyteller is happy for the story to be used. The storyteller should also be asked whether they wish their name to accompany the story. Even when consent has been given, it is good practice to check with storyteller before placing any stories in external media such as newspapers.

Planning the selection process

• How many levels of selection will there be above the field staff who initially documents the SC stories? This usually depends on the number of layers of management that already exist within the organization.
• At each of these levels, how many separate selection processes will there be? This will depend on the number of separate offices at each level (based on location or specialization).
• In each of these levels, how many SC stories can be managed by the staff involved? It is unrealistic to expect staff to meet and work on the selection of SC stories for more than two hours at the most. If there are four domains of change to review, this means 10 minutes for each. Within each domain, aim to read through and discuss no more than 10 SC stories.
• Who should participate in each selection process? This aspect is covered in more derail below.
• How often should selection occur? Normally this choice would be dependent on the frequency with which SC are collected.

For each domain the group will select a story that they believe represents the most significant change of all. The selection process invariably begins with reading some or all of the stories wither out loud or individually. The key ingredients to story selection are:

• Everybody reads the stories
• The group holds an in-depth conversation about which stories should be chosen
• The group decides which stories are felt to be most significant
• The reasons for the group’s choice(s) are documented

The group must decide whether the criteria for selecting stories will be identified before or after reading stories. If MSC is being used to aid organizational learning , the selection criteria should not be decided in advance but should emerge through discussion of the reported changes.

There are several ways of reaching a decision about which stories to select:

• Majority rules

  Read the stories, make sure everyone understands them, and then vote by show of hands. The main risk is that a choice will be made without any substantial discussion.

• Iterative voting

  After the first vote, people discuss why they voted as they did. This is followed by a second and then a third vote, ideally with some movement towards consensus.

• Scoring

  Instead of voting, participants can rate the value of a SC story. The ratings for each of the stories are then aggregated and the story with the highest rating is selected as the most significant. The downside is the limited opportunity for dialogue, although explanations for rating can be given at the same time as the ratings.

• Pre-scoring and group vote

  The method is suitable for groups who are short of meeting time. Prior to the meeting, participants are asked to read SC stories and rate their significance. These ratings are summarized in a table and presented to the participants when they meet face-to-face. Participants discuss the score and cast their vote. The disadvantage is that all stories must be sent to participants some time before the meeting.

• Secret ballot

  Cast votes anonymously. Each person writes their choice of SC story on a slip of paper, and then the total votes are presented. This should be followed by an open discussion of the reasons for the choices. This process can be surprisingly useful, especially if there are power inequalities in the group, or if people are initially reluctant to cast their votes publicly.

Building capability for effective MSC^[3]

Job Aid