Most Significant Change (MSC)

The MSC process involves the collection of significant change (SC) stories emanating from a particular project and the systematic selection of the most significant of these stories by panels of designated stakeholders or staff. The process is developed around positive questioning such as Looking back over the (concerned period), what do you think was the most significant change in [particular domain of change]? or From among all these significant changes, what do you think was the most significant change of all?

The MSC can potentially influence organizational learning through its horizontal and vertical processes. The horizontal dimension is between a group of participants engaged in discussing and selecting the most significant of a set of stories while the vertical dialogue involves exchanges of views between groups of participants at different levels. The vertical dimension is particularly important if the MSC process is to assist organizational learning throughout the organization. It will depend on good documentation and communication of the results of one group’s discussion to the other.

The creation and/or facilitation of the following contexts are important for a successful MSC implementation:

• An organizational culture where it is acceptable to discuss both successes as well as things that go wrong;
• Champions capacitated to promote the use MSC, including having good facilitation skills
• Willingness to try something different
• Time to run several cycles of the approach
• Infrastructure to enable regular feedback of the results to stakeholders
• Senior management commitment

Implementation steps of MSC technique:

1. Starting and raising interest
2. Defining the domains of change
3. Defining the reporting period
4. Collecting SC stories
5. Selecting the most significant of the stories
6. Feeding back the results of the selection process
7. Verification of stories
8. Quantification
9. Secondary analysis and meta-monitoring
10. Revising the system

Step By Step^[2]

Step 1: Starting and raising interest

A. It may help to use one of the following metaphors to explain the MSC:

• Newspaper: Newspapers are structured into different subject areas in the same way that MSC uses domains.
• School of fish: MSC helps the individual fish to communicate with each other and to swim in the same direction, towards what is good and away from what is not.
• Holiday memories: MSC helps teams to focus on memorable events and to use these events to help realign efforts towards achieving more of the wonderful things and less of the terrible ones.
• Restaurant menu: MSC presents a series of glimpses of what a programme is achieving. The stakeholders can then select from these glimpses in much the same way as they would select food from a restaurant menu.

B. Start small. It is a risky exercise to implement a huge and complicated MSC system without first piloting it on a small scale.

C. Identify key people (champions) who are excited by MSC. These champions can:

• Excite and motivate people
• Answer questions about the technique
• Facilitate selection of SC stories
• Encourage people to collect SC stories
• Ensure that feedback occurs
• Ensure that the stories are collected and organized and sent to review meetings
• Develop protocols to ensure confidentiality where necessary

Using domains of change helps organizations to group a large number of SC stories into more manageable lots, which can each be analyzed in turn. The “any other type of change” domain is a useful open category that allows participants to report significant changes that don’t fit into the named domains. Between three and five domains is a manageable number. The limiting factor is how much time participants are willing to spend in discussing each domain.

A domain can be identified before SC stories are selected or afterwards by sorting SC stories into meaningful groups. This depends on the extent to which the organization wants to be open to new experiences rather than continuing to be guided by past experiences.

The frequency of collection of SC stories has varied from fortnightly to yearly. Each organization has to make its own decision about the most appropriate reporting period, balancing the costs and benefits involved, and taking into account the reporting gaps that any existing monitoring and evaluation systems may be ignoring.

Experiences suggest that organizations tend to start MSC with more regular reporting and decrease the frequency as the process continues.

Step 4: Collecting SC stories

A. How to capture SC stories:

• Fieldworkers write down unsolicited stories that they have heard
• By interviews and note-taking
• Group discussion sessions
• Beneficiaries write a story directly

B. The information to be documented should include:

• Information about who collected the story and when the evens occurred
• Description of the story – what happened
• Significance (to the storyteller) of the events described in the story which is the key part of MSC

C. How long should the stories be?

Most MSC stories are a page or less in length, with some being up to two pages. Shorter MSC stories are quicker and easier to read, but they should not be so short that vital information is left out.

D. Ethics of collecting stories

When a storyteller tells a story, the person collecting the story needs to explain how the story is to be used and to check whether the storyteller is in agreement with its use. The storyteller should also be asked whether they wish their name to accompany the story. Even when consent has been given, it is good practice to check with storytellers before placing any stories in media such as newspapers.

Step 5: Selecting the most significant of the stories

The MSC approach uses a hierarchy of selection process. People discuss SCs within their area and submit the most significant of these to the level above, which then selects the most significant of all the SCs submitted by the lower levels and passes this on to the next level. The diagram below illustrates this process.

A. Planning the selection process

• How many levels of selection will there be above the staff who initially documents the SC stories? This usually depends on the number of layers of management that exist within the organization.
• At each of these levels, how many separate selection processes will there be? This will depend on the number of separate offices at each level (based on location or specialization).
• In each of these levels, how many SC stories can be managed by the staff involved? It is unrealistic to expect staff to meet and work on the selection of SC stories for more than two hours at the most. If there are four domains of change to review, this means 10 minutes for each. Within each domain, aim to read through and discuss no more than 10 SC stories.
• Who should participate in each selection process? This aspect is covered in more derail below.
• How often should the selection occur? Normally this choice would be dependent on the frequency with which SC are collected.

For each domain the group will select a story that they believe represents the most significant change of all. The selection process invariably begins with reading some or all of the stories out loud or individually. The key ingredients to story selection are:

• Everybody reads the stories
• The group holds an in-depth conversation about which stories should be chosen
• The group decides which stories are felt to be most significant
• The reasons for the group’s choice(s) are documented

The group must decide whether the criteria for selecting stories will be identified before or after reading stories. If MSC is being used to aid organizational learning, the selection criteria should emerge through discussion of the reported changes and not be decided in advance.

There are several ways of reaching a decision about which stories to select:

• Majority rules

  Read the stories, make sure everyone understands them, and then vote by show of hands. The main risk is that a choice will be made without any substantial discussion.

• Iteractive voting

  After the first vote, people discuss why they voted as they did. This is followed by a second and then a third vote, ideally with some movement towards consensus.

• Scoring

  Instead of voting, participants can rate the value of a SC story. The ratings for each of the stories are then aggregated and the story with the highest rating is selected as the most significant. The downside is the limited opportunity for dialogue, although explanations for rating can be given at the same time as the ratings.

• Pre-scoring and group vote

  The method is suitable for groups who are short of meeting time. Prior to the meeting, participants are asked to read SC stories and rate their significance. These ratings are summarized in a table and presented to the participants when they meet face-to-face. Participants discuss the score and cast their vote. The disadvantage is that all stories must be sent to participants some time before the meeting.

• Secret ballot

  Cast votes anonymously. Each person writes their choice of SC story on a slip of paper, and then the total votes are presented. This should be followed by an open discussion of the reasons for the choices. This process can be surprisingly useful, especially if there are power inequalities in the group, or if people are initially reluctant to cast their votes publicly.

It is important to remember that in the MSC transparency is an important way of making subjectivity accountable. Therefore, it is very important to add the second step of capturing and discussing the reasons for choices.

The documentation attached to the most significant story should record:

• The reasons for selecting an SC story as the most significant
• The process used to select the story

Stories that are filtered out should not be thrown away. They should be kept on file so that they are accessible to others within the organization using the MSC, for as long as they continue to use it, and arguably even for a while after that. This is to enable some systematic content analysis of the full set of documented SC stories.

Step 6: Feeding back the results of the selection process

The results of a selection process must be fed back to those who provided the SC stories. At the very least, this feedback should include the following points:

• Explain which SC was selected as the most significant and why.
• Provide information on how the selection process was organized.
• In some cases participants provide more comprehensive feedback in the form of tables showing who gave which rating to what SC story.

Building capability for effective MSC^[3]

Job Aid