MAMA Global Monitoring and Evaluation Framework

The Mobile Alliance for Maternal Action (MAMA) partnership seeks to achieve scale, sustainability and impact "by creating a replicable model for reaching low-income mothers and household decision-makers by increasing the impact of current mHealth (health information by way of mobile telephones [see related summaries below]) programmes, providing technical assistance to new mHealth models, and improving methods of applying mobile technology to protect maternal health." The programme "is based on the premise that timely reminders and health information delivered via text message will help women remember and stay motivated to seek appropriate health services as their needs arise, as well as providing them with information regarding their babies' development."

This strategic tool, MAMA's monitoring and evaluation (M&E) plan, responds to a general consensus that "there is insufficient evidence to measure or quantify the impact of [mHeath and eHealth] strategies on improved health outcomes." By using this M&E framework to evaluate programming, MAMA performance can be measured regularly and evidence can be built for associating mHealth messages and greater access to health information with improved care-giving and care-seeking behaviour among women.  The MAMA M&E framework is designed for use by project implementers, including beneficiaries (pregnant women, new mothers, household decision-makers) and other key stakeholders (global MAMA partners, in-country partner organisations, country governments, mobile network operators, corporate advertisers, and future donors). They may use it as a functional guide to assist with expansion of the MAMA initiative programmes to other countries.

The framework includes performance metrics, indicator definition and reference sheets, and guidelines for data collection and programme evaluation. Indicators for routine process and output monitoring are based on the MAMA theory of change, through which small groups of international partners can work together to:

• Catalyse locally driven, sustainable country partnerships.
• Create a global learning agenda that builds capacity among mHealth organisations by drawing upon the country partnership experience to inform the larger mHealth community, and drawing best practices from the larger mHealth community to inform MAMA country programmes.
• Focus on the outcomes of improved care-seeking and preventive behaviours for pregnant women, new mothers, and their families in resource-poor settings.
• Develop a more robust set of resources and evidence base to support future mHealth initiatives.

M&E activities are performed in country by MAMA in-country implementing partners and their M&E team and concentrate on measuring change in the MAMA process and output indicators (monitoring) and examining the effects of certain processes in the formative state to improve design of the programme and focus on more long-term effects on outputs and health outcomes of the programme (evaluation). Impact evaluations are the responsibility of external evaluators and focus on comparisons between participants and non-participants to answer these questions:

1. Does the MAMA programme have a positive impact on a mother’s antenatal and postnatal care regimens?
2. For those women who adhere to MAMA recommendations, does it have a positive impact on their health and the health of their child?
3. What aspects of the messaging are most effective in prompting women who are enrolled to take action?

The tool includes key M&E questions, how to operationalise the framework, resources and a time line, and sample performance indicator reference sheets. Key M&E questions on page 10 of the document and key suggested indicators (Table 3 on pages 13-18) can be linked to a theory of change table and an activities table in this same document. Key suggested indicators include (but are not limited to):

• Percentage of women/households who followed healthy behaviours;
• Percentage of women who utilised follow-up services offered by the health system;
• Return on investment (upfront investment/operating cost);
• Average cost per user and assessment of target population's willingness to pay;
• Percentage of messages adapted for local use and easy understanding by user by MCH area/state of pregnancy, motherhood;
• Percentage of population enrolled who are women or household member subscribers (by socio-economic characteristics/geographic location and representation);
• Percentage of subscribers who opt out of the programme;
• Percentage of messages recalled/understood by population;
• Number of peer referrals outside the household;
• Percentage of subscribers who intend to adopt appropriate care-giving and care-seeking behaviour.