Most Significant Change (MSC) Technique
Excerpted from "The 'Most Significant Change' (MSC) Technique: A Guide to Its Use":
"The most significant change (MSC) technique is a form of participatory monitoring and evaluation. It is participatory because many project stakeholders are involved both in deciding the sorts of change to be recorded and in analysing the data. It is a form of monitoring because it occurs throughout the program cycle and provides information to help people manage the program. It contributes to evaluation because it provides data on impact and outcomes that can be used to help assess the performance of the program as a whole.
Essentially, the process involves the collection of significant change (SC) stories emanating from the field level, and the systematic selection of the most significant of these stories by panels of designated stakeholders or staff. The designated staff and stakeholders are initially involved by 'searching' for project impact. Once changes have been captured, various people sit down together, read the stories aloud and have regular and often in-depth discussions about the value of these reported changes. When the technique is implemented successfully, whole teams of people begin to focus their attention on program impact...
Overview of implementation steps
MSC is an emerging technique, and many adaptations have already been made that will be discussed throughout this Guide. Before getting into modifications, we present a comprehensive overview of what a 'full' implementation of MSC might look like. We have described this using ten steps.
The first step in MSC generally involves introducing a range of stakeholders to MSC and fostering interest and commitment to participate. The next step is to identify the domains of change to be monitored. This involves selected stakeholders identifying broad domains - for example, 'changes in people's lives' - that are not precisely defined like performance indicators, but are deliberately left loose, to be defined by the actual users. The third step is to decide how frequently to monitor changes taking place in these domains.
SC stories are collected from those most directly involved, such as participants and field staff. The stories are collected by asking a simple question such as: 'During the last month, in your opinion, what was the most significant change that took place for participants in the program?' It is initially up to respondents to allocate their stories to a domain category. In addition to this, respondents are encouraged to report why they consider a particular change to be the most significant one.
The stories are then analysed and filtered up through the levels of authority typically found within an organisation or program. Each level of the hierarchy reviews a series of stories sent to them by the level below and selects the single most significant account of change within each of the domains. Each group then sends the selected stories up to the next level of the program hierarchy, and the number of stories is whittled down through a systematic and transparent process. Every time stories are selected, the criteria used to select them are recorded and fed back to all interested stakeholders, so that each subsequent round of story collection and selection is informed by feedback from previous rounds. The organisation is effectively recording and adjusting the direction of its attention - and the criteria it uses for valuing the events it sees there.
After this process has been used for some time, such as a year, a document is produced with all stories selected at the uppermost organisational level over that period in each domain of change. The stories are accompanied by the reasons the stories were selected. The program funders are asked to assess the stories in this document and select those that best represent the sort of outcomes they wish to fund. They are also asked to document the reasons for their choice. This information is fed back to project managers.
The selected stories can then be verified by visiting the sites where the described events took place. The purpose of this is two-fold: to check that stories have been reported accurately and honestly, and to provide an opportunity to gather more detailed information about events seen as especially significant. If conducted some time after the event, a visit also offers a chance to see what has happened since the event was first documented.
The next step is quantification, which can take place at two stages. When an account of change is first described, it is possible to include quantitative information as well as qualitative information. It is also possible to quantify the extent to which the most significant changes identified in one location have taken place in other locations within a specific period. The next step after quantification is monitoring the monitoring system itself, which can include looking at who participated and how they affected the contents, and analysing how often different types of changes are reported. The final step is to revise the design of the MSC process to take into account what has been learned as a direct result of using it and from analysing its use."
Click here to download the full document as a PDF file.
"The most significant change (MSC) technique is a form of participatory monitoring and evaluation. It is participatory because many project stakeholders are involved both in deciding the sorts of change to be recorded and in analysing the data. It is a form of monitoring because it occurs throughout the program cycle and provides information to help people manage the program. It contributes to evaluation because it provides data on impact and outcomes that can be used to help assess the performance of the program as a whole.
Essentially, the process involves the collection of significant change (SC) stories emanating from the field level, and the systematic selection of the most significant of these stories by panels of designated stakeholders or staff. The designated staff and stakeholders are initially involved by 'searching' for project impact. Once changes have been captured, various people sit down together, read the stories aloud and have regular and often in-depth discussions about the value of these reported changes. When the technique is implemented successfully, whole teams of people begin to focus their attention on program impact...
Overview of implementation steps
MSC is an emerging technique, and many adaptations have already been made that will be discussed throughout this Guide. Before getting into modifications, we present a comprehensive overview of what a 'full' implementation of MSC might look like. We have described this using ten steps.
- How to start and raise interest
- Defining the domains of change
- Defining the reporting period
- Collecting SC stories
- Selecting the most significant of the stories
- Feeding back the results of the selection process
- Verification of stories
- Quantification
- Secondary analysis and meta-monitoring
- Revising the system
The first step in MSC generally involves introducing a range of stakeholders to MSC and fostering interest and commitment to participate. The next step is to identify the domains of change to be monitored. This involves selected stakeholders identifying broad domains - for example, 'changes in people's lives' - that are not precisely defined like performance indicators, but are deliberately left loose, to be defined by the actual users. The third step is to decide how frequently to monitor changes taking place in these domains.
SC stories are collected from those most directly involved, such as participants and field staff. The stories are collected by asking a simple question such as: 'During the last month, in your opinion, what was the most significant change that took place for participants in the program?' It is initially up to respondents to allocate their stories to a domain category. In addition to this, respondents are encouraged to report why they consider a particular change to be the most significant one.
The stories are then analysed and filtered up through the levels of authority typically found within an organisation or program. Each level of the hierarchy reviews a series of stories sent to them by the level below and selects the single most significant account of change within each of the domains. Each group then sends the selected stories up to the next level of the program hierarchy, and the number of stories is whittled down through a systematic and transparent process. Every time stories are selected, the criteria used to select them are recorded and fed back to all interested stakeholders, so that each subsequent round of story collection and selection is informed by feedback from previous rounds. The organisation is effectively recording and adjusting the direction of its attention - and the criteria it uses for valuing the events it sees there.
After this process has been used for some time, such as a year, a document is produced with all stories selected at the uppermost organisational level over that period in each domain of change. The stories are accompanied by the reasons the stories were selected. The program funders are asked to assess the stories in this document and select those that best represent the sort of outcomes they wish to fund. They are also asked to document the reasons for their choice. This information is fed back to project managers.
The selected stories can then be verified by visiting the sites where the described events took place. The purpose of this is two-fold: to check that stories have been reported accurately and honestly, and to provide an opportunity to gather more detailed information about events seen as especially significant. If conducted some time after the event, a visit also offers a chance to see what has happened since the event was first documented.
The next step is quantification, which can take place at two stages. When an account of change is first described, it is possible to include quantitative information as well as qualitative information. It is also possible to quantify the extent to which the most significant changes identified in one location have taken place in other locations within a specific period. The next step after quantification is monitoring the monitoring system itself, which can include looking at who participated and how they affected the contents, and analysing how often different types of changes are reported. The final step is to revise the design of the MSC process to take into account what has been learned as a direct result of using it and from analysing its use."
Click here to download the full document as a PDF file.
Source
MandE News Archives, "The 'Most Significant Change' (MSC) Technique: A Guide to Its Use" by Rick Davies and Jess Dart (2005).
Comments
- Log in to post comments