“Everything that can be counted does not necessarily count; everything that counts cannot necessarily be counted.” – Albert Einstein

What is MEL, and why does it matter?

MEL (Monitoring, Evaluation, and Learning) is a critical function of any organization, particularly in the nonprofit sector. The approach to MEL varies greatly from one organization to another, reflecting and driving organizational culture and values. Historically, the purpose of MEL has largely been to demonstrate and uphold transparency and accountability, particularly to donors, boards, governments, senior management, etc. As a result, MEL activities prioritize the prerogatives and perceptions of power holders, not of those directly impacted by the work. 

When external stakeholders are prioritized above all else, MEL does not generate the most valuable, reliable, robust, credible evidence of change and impact. In addition, the “L” of MEL is often missing from organizational systems and processes, despite knowing that evidence-based learning and adaptation is a necessary part of ensuring effectiveness and impact. For these and other reasons, evidence has shown that upward-accountability-focused MEL is antithetical to the mission and vision of civil society organizations, particularly those focused on beneficiary-led initiatives and interventions.

More recently, several connected themes have emerged to advance how we think about learning and accountability. First is the recognition, particularly by rights-focused organizations, that people served by development and humanitarian organizations must have a say in program design, implementation, adaptations, and desired impact. While this has been influenced by rights-based programming, numerous scandals have highlighted high costs and wastage when people are not engaged. 

The second theme is the acknowledgment that the concept of one NGO single-handedly solving complex social problems (wicked problems; social messes) is a fallacy. Change is complex, authentic partnerships are required, and the pursuit of attribution is methodologically impossible. 

Finally, the emergence of strong anti-colonialism/anti-racism movements in connection with the rise of localization has forced the sector to confront long-standing unequal power dynamics, and to meaningfully address the questions of whose voices matter and who defines changes and impact. 

What does this mean for MEL?

1. There needs to be increased recognition that MEL is not a standalone activity designed to count “how much” or “how many” by a so-called “impartial and unbiased” assessor to “prove” value for money and impact. For starters, organizational culture, principles, and values drive what matters (and who decides it), what gets measured (and who decides it), and the balance of learning versus accountability (including systems, processes, capacity, and resources)—therefore everyone, particularly leadership, has a role to play in ensuring quality and effectiveness through evidence-informed learning.
2. Those closest to the work, particularly those directly targeted by programs, but also partner implementing organizations, allies, governments, and other critical stakeholders, need to define what matters and determine how it might be assessed. In turn, space must be made for all stakeholders to actively participate in assessments and subsequent sense-making. Diversity of views and experiences must be included and accounted for, and disaggregation (e.g. by gender, age, ethnicity) must be included in MEL initiatives to better identify and understand differentiated experiences and differentiated impacts.
3. Assessing complex social changes requires shifts in understanding and approaches, moving from attribution to contribution, incorporating mixed methods (quantitative and qualitative), and adopting inclusive and participatory theories and approaches (which apply not only to data collection). This underscores the importance of collaboration and the need to shift the organizational mindset away from competition and the false hope that sole ownership of solutions (often driven by fundraising and donor relations) is possible.
4. Emphasizing learning at all levels enables organizations to better understand and identify “what’s working” and to be more effective—which in turn ensures a stronger sense of accountability and transparency, with the evidence to “back it up.” Connected to this is the recognition that while upward accountability, particularly to donors, remains a strong influence for non-profit organizations, so-called “downward” accountability, to affected populations, should be primary—and NGOs themselves, as well as the sector, can and should do much more to influence donors in shifting their expectations and understanding. This includes reducing/eliminating an overreliance on results (“what has been done or delivered”) instead of outcomes (“what has changed”), valuing participation and local leadership, properly resourcing evaluations and learning activities, and helping to shift understanding of “robust”, “objective”, “reliable”, and “impact” while focusing on “contribution” as opposed to “attribution”.
5. Adopting methods and approaches that are explicitly anti-colonialist and anti-racist, prioritizing local knowledge and experience, recognizing the wholeness of people and their complex multiple realities, and putting primacy on their voices to define and assess impact. Evaluation approaches such as feminist evaluation and culturally responsive evaluation provide pathways to follow, but this means acknowledging that substantive changes take time and resources are required to support evaluation and learning. 

“Evaluation is only worth doing and only ethical if the findings are used to improve the reality of the participants. The utility of the findings starts with your methods and the extent to which the voice of the participants is reflected in them.” – Susan Wolfe, Evaluator and Community Psychologist

What does this mean for RefugePoint?

RefugePoint recognizes that the objective of MEL is to strengthen effectiveness and impact through a balance of learning and accountability, where learning sits at the heart of accountability. We understand and embrace the connectedness of organizational values (continuous learning, accountability, client-focused, excellence, and solidarity), culture and MEL, particularly as we continue on our anti-racism/anti-colonialism journey and seek to be truly refugee-centered. To that end, we have begun introducing a comprehensive agency-wide MEL system that includes and prioritizes the following components:

1. Leadership – perhaps the most critical of all of the components, leadership is essential to creating and maintaining a learning culture. Organizational leaders drive interest in generating and using learning, as well as play an important role in ensuring that the organization does not view MEL simply as a way to reassure donors and supporters.
2. Critical reflection and curiosity – going beyond recording what has happened to understand why it happened, what it means, and what should be done about it. This requires data analysis, not just data collection and reporting, with analysis and sense-making done by those closest to the work.
3. Culture and attitude – this means promoting openness and transparency, maximizing the involvement of different stakeholders, and institutionalizing input and feedback at all levels. This also helps reinforce the feeling that people’s opinions, ideas, and suggestions are valued.
4. Clarity around learning – orienting the MEL system towards what is immediately useful by being intentional about identifying what you want to learn and why. This means developing learning questions or an evidencing agenda that is properly resourced, and where the audience(s) and use(s) have been identified from the beginning.
5. Integration – MEL cannot be standalone, it must interact with all parts of the organization, be based on accurate and realistic strategies and plans, and be integrated into other management processes. Among other things, this helps to ensure that organizational resources are used effectively to monitor and evaluate objectives and plans that are not obsolete or out of date.
6. Practical learning mechanisms – this means supporting learning as well as translating that learning into improved performance. “The ultimate test of learning is whether it is applied—if not, then it is an expensive luxury,” (INTRAC: Learning-based M&E Systems, 2017). Examples of practical learning mechanisms include participatory reflection and learning spaces, report templates that focus on analysis and recommendations not only activities and results, communications products tailored to different audiences and purposes, diverse stakeholder engagement in defining, collecting, analyzing, and using evidence, and ensuring learning questions/evidence agendas have evaluations, reviews and/or assessments properly resourced to address them. 

RefugePoint is committed to a pathway that enables us to become anti-racist and anti-colonialist, where we generate and use evidence of our contributions to longer-term changes and impact, and create spaces for the voices of refugees to guide our understanding of what we need to do and how well we have done. We know that this means not only revising policies but also changing processes and culture—in other words, facilitating structural changes internally and externally that are regularly assessed and improved where needed. Over the last 12 months, RefugePoint has introduced a number of initiatives to strengthen our MEL system in ways that increase and improve learning, foster refugee-centeredness, and emphasize awareness of the impact of power differentials. These initiatives include the following: 

• Design of a measurable and evaluable 3-year plan focused on outcomes and objectives as well as results, 
• Introduction of an agency-wide quarterly review process that is participatory and evidence-informed,  
• Development of accountability and oversight committees that include refugees and guide MEL practices
• Development of programmatic theories of change, systems change strategies and learning questions, 
• Identification of indicators supported by data biographies
• Improved program dashboards and agreed historical data
• Increased use and analysis of surveys and other feedback mechanisms
• Improved use of data visualizations
• Enhanced use of databases
• Introduction of reviews and evaluations utilizing new methods and approaches
• Adoption of a utilization-focused approach to MEL—identifying audiences and users from the outset
• Creation of an anti-racism working group that drives how we apply anti-racism/anti-colonialism principles and approaches to our work and how we measure it
• Design and introduction of safeguarding policies and accountability mechanisms that ensure we are doing no harm and fostering open, transparent, and safe relationships with the people we serve
• Explicitly connecting anti-racism and refugee-centeredness to MEL—including identification and application of context-specific, culturally aware methods and approaches

Moving forward, we plan to continue the roll-out and improvement of the above and prioritize the implementation and use of reviews and evaluations to understand, learn, and demonstrate our effectiveness. We want everyone who is part of RefugePoint to be involved with defining and understanding what difference we make, in ways that are empowering, especially to refugees, but not overly onerous, and enable us to learn and adapt.