Peacebuilding is a vast, complex and rapidly evolving field. Research has a key role to play in addressing ongoing challenges in peacebuilding practice and in improving analysis, strategy development, monitoring and evaluation. Organizations involved in peacebuilding work can use research to assess and increase their effectiveness and positive impact and to mitigate unintended negative effects of intervention and impact assessment on local actors. Below I propose some areas where the work of peacebuilding organizations can benefit from investing in research:
CLARIFYING ASSUMPTIONS ABOUT WHAT ACTIONS COUNT AS ‘PEACEBUILDING’
AND WHAT OUTCOMES COUNT AS ‘IMPACT’
Many organizations want to know whether their peacebuilding investments have impact and typically begin the process of impact evaluation by trying to identify what ‘counts’ as peacebuilding indicators. They often do so, however, without making explicit the manner in which they have framed their notions of ‘peacebuilding’ and of ‘impact’. Clarifying these conceptualizations is a necessary first step in any impact evaluation and functions to inform the evaluation methodology. Theoretical framings of peacebuilding are contested, so it is important to spell out in clear terms how a given organization or intervention understands what peacebuilding looks like, what factors its success is conditioned upon, and what it is expected to yield in terms of processes and measurable achievements.
Some key questions:
- How does the organization / project define peace, its prerequisites, and indicators? What processes and activities does the organization / project associate with peacebuilding? What assumptions and theories of change underlie these conceptualizations? How can the adequacy of these assumptions be evaluated?
- How do / can organizations integrate new peacebuilding conceptualisations into established intervention models?
- What measures do / can organizations use to support ongoing critical reflexivity on the peacebuilding assumptions underpinning intervention strategy development, monitoring, and evaluation, to ensure continued relevance?
IMPACT ACROSS THE INTERVENTION LIFE-CYCLE
Further consideration may also be given to the issue of where impact evaluation fits in the intervention life-cycle. In this respect, it may be useful to consider how peacebuilding impact analysis and evaluation can contribute across six intervention phases:
1. Estimating impact (conceptualization phase)
2. Planning impact (design phase)
3. Monitoring impact (implementation phase)
4. Evaluating impact (milestone assessment phase(s)
5. Reporting impact (dissemination phase)
6. Embedding impact (learning integration phase)
Some key questions:
- In each of the six phases, how and to what extent does the organization reflect on impact?
- What can organizations do to better support reflection on impact in each phase of intervention?
- In the dissemination phase, how can organizations best leverage alternative modes of knowledge sharing, including mobile technologies and social media platforms, to maximize the impact of research dissemination?
- How can organizations improve stakeholder involvement in the dissemination and integration of learning?
METHODOLOGICAL DESIGN AND METHODS SELECTION
In each phase of the intervention life-cycle, a different methodology or set of methodologies may be needed in function with the informational needs of the research stakeholders. Each methodology has advantages/strengths and disadvantages/weaknesses. A thorough understanding and scoping of these different methodologies would facilitate appropriate selection and design for intervention partners. Peacebuilding organizations often lack specialized knowledge in this respect and only realize how important a sound methodological framework for evaluation is in even early intervention design phases when seeking post-hoc external evaluation assistance. Mechanisms to facilitate understanding of and access to relevant methodological options are needed.
The same is true for methods. Lack of specialist knowledge leads to over-reliance on impact assessment methods that are poorly adapted to capturing nuances of significance in complex peacebuilding scenarios. The systematic elaboration of the peacebuilding and impact assessment uses and limitations of methods such as discourse analysis, surveys, interviews, focus groups, card stimulus, scorecards, dialogue, scenario testing, visual methods, journaling, ethnographic observation, and others is important to the development of improved practice.
Some key questions:
- How can the organization systematize its own learning around the efficacy of methodologies used for impact assessment and ensure effective dissemination and exchange of that learning among organizations and actors across the peacebuilding field?
- How can the organization create an accessible, flexible, searchable, continuously updated and elaborated database of multi-scalar peacebuilding research methodologies, methods and indicators sets, to meet the needs of peacebuilding intervention designers, researchers and practitioners?
- How can the organization improve qualitative and quantitative data collection and analysis techniques for rich case and cross-case analysis?
- How can the organization best make use of mobile technologies, interactive e-data collection tools, and Google search trends to support ongoing conflict and peacebuilding analysis?
- How can the organization triangulate methodologies to best capture systemic interactions, for example, among more and less powerful peace and conflict actors?
ELABORATION OF IMPACT INDICATORS
The OECD-DAC (2002) defines impact evaluation as any evaluation that seeks to assess the “positive and negative, primary and secondary long-term effects produced by a development intervention, directly or indirectly, intended or unintended”. While identifying researchable and reliable indicators is difficult, particularly in complex conflict settings, a growing body of experience with different methodologies and methods offers opportunities to define and elaborate on a common core of indicator sets suited to different conflict types and modes of intervention. Through processes of exchange and discussion, peacebuilding stakeholders can generate richer outcomes and impacts indicator sets. Increasingly, international actors are dialoguing on how to constitute a well-formulated common core of reliable peacebuilding indicators in different sectors and at different scales of intervention that could form the basis of comparative peacebuilding analysis and policymaking (International Dialogue on Peacebuilding and Statebuilding, 2013). But these continue to centre on basic liberal peace concerns and have yet to encompass other key peacebuilding dimensions, such as transitional justice, national dialogues and education reform. Nor yet is the practice of shared indicator development widely adopted. As such, evaluation literature produced by larger and smaller policy and field organizations continue to talk past each other. Guidelines remain highly diverse in research quality, focus and style. A handful of evaluation guidelines guide discourse (such as Bush, 2009; Corlazzoli & White, 2013; OECD, 2012; Peacebuilding Centre, 2011; United Nations, 2010), but issues of indicator confusion, low uptake of recommended practices, and paucity of meta-evaluation persist.
Key questions could include:
- How to create and continuously update a shared international database of reliable peacebuilding impact indicators applicable to an ever-expanding range of intervention types?
- How to better facilitate interagency dialogue on reliability standards for common impact indicators, in order to improve the comparability of research?
- How to facilitate meaningful exchange between macro and micro level approaches to evaluation in peacebuilding efficacy and impact?
EQUITY AND SOCIAL JUSTICE CONSIDERATIONS
Once an initial evaluation or impact assessment strategy is devised, it is important to conduct a critical audit of the strategy in reference to conflict-sensitive and social justice criteria. Evaluators and organizations do well to consider the unintended negative effects of traditional control-centred evaluation practices on local peacebuilding actors, and to adopt methodological approaches that better support inclusive and constructive learning processes and the further development of local capacities in ongoing peacebuilding work (Bächtold, Dittli, & Servaes, 2013).
Some key questions:
- How to monitor for unintended negative impacts of peacebuilding intervention like inadvertent exclusion and elitism?
- How to map resources flows and bottlenecks, enabling and constraining factors that disproportionately affect certain populations and identifying clear recommendations for improved policy and practice?
- How to best support the development and adoption of alternative intervention norms that favour broad-based inclusion and social justice?
- How to increase not only the presence but the influence of marginalized and vulnerable populations in intervention design, monitoring, and evaluation, and to improve downward accountability mechanisms?
- How to facilitate critical dialogue with donors and implementation actors about intervention norms that reinforce social inequalities?
LOCAL CONFIDENCE IN EVALUATION AND IMPACT ASSESSMENT
Even with sound guidelines, many peacebuilding actors continue to distrust the purposes and utility of evaluation and impact assessment. This can compromise the quality of reporting that feeds into an evaluation and impedes the adoption and full learning benefit that evaluation and impact assessment can provide.
Some key questions:
- How to increase trust among implementing agencies and local actors in evaluation and impact assessment processes?
- How to facilitate multi-stakeholder analysis of (dis)continuities between peacebuilding (a) discourse, (b) policy, (c) financing, (d) management, (e) implementation / practice, and (f) experience and perceptions?
- How to overcome challenges resulting from power- and benefit-imbalances in impact assessment, like local survey fatigue, etc?
- How to ensure that evaluation and impact assessment not only ‘do no harm’, but actually leave a ‘peacebuilding footprint’?
Selected Reference Materials
- Bächtold, S., Dittli, R., & Servaes, S. (2013). Help or Hindrance ? in conflict-affected situations (swisspeace Working Paper No. 1/2013). Bern.
- Bush, K. (2009). Aid for Peace: a Handbook for Applying Peace and Conflict Impact Assessment to Peace III Projects. Ulster.
- Corlazzoli, V., & White, J. (2013). Measuring the Un-Measurable: Solutions to measurement challenges in fragile and conflict-affected environments. London.
- EU-OECD. (2015). Policy Brief on Social Impact Measurement for Social Enterprises. Luxembourg.
- International Dialogue on Peacebuilding and Statebuilding. (2013). Peacebuilding and Statebuilding Indicators – Progress , Interim List and next steps. Washington, DC.
- OECD. (2012). Evaluating Peacebuilding Activities in Settings of Conflict and Fragility: Improving Learning for Results. DAC Guidelines and References Series, OECD Publishing.
- OECD-DAC. (2002). Glossary of Key Terms in Evaluation and Results-Based Management.
- Peacebuilding Centre. (2011). Peace and Conflict Impact Assessment (PCIA) Handbook (Version 3). Ottawa: Peacebuilding Centre. Retrieved from http://peacebuildingcentre.com/pbc_documents/PCIA_HandbookENv3.3.pdf
- So, I., & Staskevicius, A. (2015). Measuring the “impact” in impact investing. Boston.
- United Nations. (2010). Monitoring Peace Consolidation: United Nations Practitioners’ Guide to Benchmarking. New York.