Participation Tools for the Pacific - Part 3: Design and Monitoring Framework
Problem Tree, Beneficiary Assessment, and Participatory Impact Pathways Analysis are useful tools when preparing the design and monitoring framework for CSO engagement.
What you need to know
Engagement of key stakeholder groups in operations financed by the Asian Development Bank (ADB) promotes good governance, transparency, innovation, responsiveness, and development effectiveness. Effective engagement of stakeholder groups, including civil society, project beneficiaries, and project-affected people, requires the understanding and effective use of participatory tools throughout the project cycle. However, while one participatory tool may work well in one context, it may not be appropriate in another. This series of explainers provides a range of tools from which practitioners can pick and choose, according to different phases of the ADB project cycle, context, and available time/resources. Some tools may be specific to particular phases in the ADB project cycle, such as monitoring and evaluation tools, while others may be used throughout the project cycle, such as participatory assessment tools.
For this piece, the focus is on Tools for Preparing the Design and Monitoring Framework.
Tools for Preparing the Design and Monitoring Framework
The Design and Monitoring Framework must be founded on a participatory approach for the following reasons:
- ADB-assisted projects should be designed to respond to the needs of beneficiaries and be designed with the direct involvement of those beneficiaries.
- Participatory approaches build ownership of projects by beneficiaries and key stakeholders.
- A group process usually creates a better and more relevant design and monitoring framework.
Problem analysis is one of the key steps of the design and monitoring process. It is done by developing a problem tree, which should be conducted in a participatory way.
This tool is used to:
- Analyze the existing situation surrounding a given problem context,
- Identify the major problems and constraints associated with the problem, and
- Visualize the cause-effect relationship diagrammatically as a problem tree.
- Starter problem. State the starter problem and place it at the center of the problem tree diagram. It may take several sessions to agree on what constitutes the starter problem, and it is important that a consensus is reached.
- Direct causes. Using vertical logic, the problems that are the direct causes of the starter problem are added to the problem tree under the starter problem. Only existing problems, not anticipated future problems, should be included.
- Direct causes to root causes. Step (ii) is repeated using direct causes as problems, and the direct causes of each of these problems are determined and placed below. The process is continued until the analysis is exhausted and very specific root causes are identified.
- Direct effects. The direct effects of the starter problem are placed above the starter problem of the problem tree.
- Direct effects to final effects. Step (iv) is repeated using direct effects as problems, and the effects of each of these problems are determined and placed above each statement.
- Review and refine. The problem tree and the interrelationship of problems, causes, and effects at different levels are analyzed and adjusted accordingly. Clarify through discussion and consultation that this will be the core problem and causes that ADB will address through its project.
Simplified Problem Tree Template
Problem tree clarifies the way forward for Samoa
CSO and government representatives met in Samoa and developed a problem tree to understand issues related to deepening ADB-government-CSO engagement. ADB convened the group to understand how it could best support this tripartite relationship in the Pacific through technical assistance.
Participants identified two to three issues each and wrote them down on colored paper; then each participant had an opportunity to explain his or her issues. The plenary grouped related issues together and presented them in a problem tree format. Participants then formed three groups to discuss the issues and come up with recommendations directed towards deepening ADB-government-CSO engagement. Each group reported the results of discussions to the rest of the participants. The discussions provided an opportunity for participants to clarify issues related to ADB-government and -civil society cooperation.
The participants made the following recommendations at the workshop:
- Raise awareness about ADB operations in the Pacific to optimize stakeholder participation and increase innovation in projects.
- Explore better ways to communicate since CSOs in the Pacific include village-level organizations and community associations who cannot access information easily, especially if it is only available on the ADB website.
- ADB engagement in the Pacific should be carefully aligned with its culture. Failure to understand the Pacific’s culture and traditional practices hinder effective engagement.
ADB. 2019. Guidelines for Preparing a Design and Monitoring Framework. Manila.
Australian Government AusAID. 2005. AusGuideline 3.3 The Logical Framework Approach. pp 26-27.
Pacific Research and Evaluation Associates. 2014. The Pacific Guide to Project Proposal Preparation Using the Logical Framework Approach: Learner Guide.
Beneficiary Assessment is a method used by the World Bank and other institutions at the design phase of programs. Beneficiary Assessment is a qualitative tool used to improve the impact of development work by systematic consultation with project stakeholders – including the vulnerable and poor – to ensure that their concerns are incorporated into the project design. It is a “systematic inquiry into people’s values and behavior in relation to a planned or ongoing intervention for social or economic change.”
Three key tools are used in Beneficiary Assessment:
- Conversational Interviews are natural, free-flowing interviews which suit the cultural style of the Pacific, similar to talanoa. The features of conversational interviews are the establishment of mutual trust and respect; a good rapport must be established between interviewers and respondents. These interviews should take no more than 45 minutes to one hour, and note taking is not encouraged during the interview (interviewers should write up immediately afterwards). These conversational interviews occur with key people in a project affected area, such as school teachers, nurses and shop owners and are structured around a number of themes to gauge current scenarios and forecast likely perceptions of project impacts on affected persons.
- Focus Group Discussions (FGD) complement and cross-check the information from stakeholders and conversational interviews. FGDs often involve between 6-12 people with a common interest or characteristic (such as youth, young mothers, female entrepreneurs). FGDs are guided by a facilitator who ensures the discussion covers the topics of the investigation but ensuring inclusive participation. A researcher should take notes on the discussion.
- Direct Observation and Participant Observation. Direct Observation involves a researcher directly observing/counting the activities and behaviors of a target group, such as women attending a community tap or houses undergoing improvement works. Participant Observation involves the researcher embedding him or herself in the community for a protracted period, ranging from several weeks to several months, to record and fully understand the activities of the target community. The emphasis is on understanding the socio-cultural and political context of the beneficiary community. This usually involves conducting a small number (5-10) of in-depth household case studies. When reporting, a Beneficiary Assessment table should incorporate the key findings. The template could include the following five columns: beneficiary group, expected benefits, possible negative impacts, relevance of project objectives to beneficiaries, measures to ensure benefits and to mitigate negative impacts.
Click here to download the sample template.
Beneficiary Assessment of the Smallholder Agriculture Development Project (SADP) in PNG
SADP’s development objective was to improve the living standards of rural communities in oil palm growing provinces in Papua New Guinea. It had two key outcomes: to increase smallholder oil palm sector productivity (including additional oil palm planting and provincial road upgrading) and to promote sustainable local governance and community participation mechanisms (including providing small community grants). A Beneficiary Assessment was undertaken to explore and assess for each major beneficiary group: their strengths, needs and concerns; how social grouping affects opportunities to participate in SADP activities; distribution of project benefits amongst different social groupings; how each social grouping can facilitate or obstruct project activities; identification of the most vulnerable and powerful social groupings; strategies to promote beneficiaries' participation, particularly vulnerable groups; potential direct and indirect adverse impacts of the SADP; and means of mitigating adverse social impacts on different social groupings. A Beneficiary Assessment template was used to map out these issues, results, and actions.
Better Evaluation. Beneficiary Assessment.
L.F Salmen. 2002. Beneficiary Assessment: An Approach Described. World Bank. p10.
Source: G. Koczberski and G.N. Curry. 2007. Beneficiaries Assessment Report for the Smallholder Agriculture Development Project (SADP), Papua New Guinea.
World Bank Group. Beneficiary Assessment.
Participatory Impact Pathways Analysis
Participatory Impact Pathways Analysis (PIPA) is a practical approach designed to help project implementers and key stakeholders explain how the project activities and outputs could contribute to project goals. It works best for complex projects or where two or more projects within a program wish to coordinate.
Examples where coordinating could occur in an ADB project: a water supply system and sanitation scheme designed as two separate projects for the same city or an integrated sustainable urban development plan that covers transport projects, infrastructure provision, sustainable waste management, slum upgrading, and policy reform. (e.g. Fiji, Vanuatu)
As a participatory process for design, monitoring, and evaluation, it goes beyond the traditional logical framework approach to engage stakeholders using participatory approaches. At the core of PIPA is a three-day participatory workshop.
At the heart of the PIPA process is a three-day participatory workshop aimed at engaging key project stakeholders. The workshop addresses the following points:
- Developing a cause and effect logic, through a problem tree.
- Developing a network perspective, through network mapping.
- Developing the outcomes logic model and an M&E plan.
Day 1 of the participatory workshop is developed to exploring the intervention’s cause and effect logic. Stakeholders construct a problem tree of the issues.
Day 2 focuses on visioning success [see description of Visioning in Tools for Assessment] and determining what success looks like for different stakeholder groups (such as ‘next users’, end-users, politically important people or organizations and the project implementers). Workshop participants draw two network maps to show the relationship between actors ‘now’ and in the ‘future’ to illustrate what is required to achieve ‘the vision’ by joining them together.
Day 3 connects these two perspectives (problem tree and network mapping) and integrates them through the development of an outcomes logic model. The outcomes logic model describes how stakeholders may act differently for the project vision to be achieved. The outcomes logic model links these outcomes to the stakeholders who can action this change, which in turn provides the basis for future M&E. It is presented in table format. Following the workshop, the group may develop an impact logic model, which describes the impact logic: outputs, adoption, outcomes and long-term impact. The logic model’s outcomes “provide an ex-ante framework of predictions of impact that can also be used in priority setting and ex-post impact assessment. PIPA engages stakeholders in a structured participatory process, promoting learning and providing a framework for 'action research' on processes of change.”*
Click here to download a sample template.
S. Alvarez et al. 2010. Participatory Impact Pathways Analysis: a practical method for project planning and evaluation. Development in Practice. 20(88). pp. 946-958
B. Douthwaite, et al. 2008. Participatory Impact Pathway Analysis: A practical application of program theory in research-for-development. Canadian Journal of Program Evaluation. pp 22, 127-159.
B. Douthwaite et al. 2008. Participatory Impact Pathways Analysis: A practical method for project planning and evaluation. ILAC Brief 17.
ILRI Tools Portal. PIPA (Participatory Impact Pathways Analysis).
USAID Learning Lab. 2013.
Some Do’s and Don’ts
The following suggestions for inclusive engagement with civil society organizations at the design stage come from AusAID’s Guidance on M&E for Civil Society Programs:
- Engage in effective power and gender analysis as key drivers of change
- Focus on building a common vision of change tailored to the local context
- Ensure that there is an appropriate and representative mix of perspectives involved, based on the power and gender analysis
- Ensure that adequate time, space and resources are built into the design for ongoing reflection, sense-making, and learning
- Ensure gender is properly considered at analysis and design
- Push CSOs or subcontractors working with CSOs to focus on narrow objectives at the expense of the process and creation of trust
- Assume that the design or analysis is correct; it will need to evolve over time
- Compartmentalize CSOs - respect their mandates and autonomy
- React punitively if your policies are challenged by CSOs; try and navigate different views with curiosity
Leave your question or comment in the section below:
YOU MIGHT ALSO LIKE
The views expressed on this website are those of the authors and do not necessarily reflect the views and policies of the Asian Development Bank (ADB) or its Board of Governors or the governments they represent. ADB does not guarantee the accuracy of the data included in this publication and accepts no responsibility for any consequence of their use. By making any designation of or reference to a particular territory or geographic area, or by using the term “country” in this document, ADB does not intend to make any judgments as to the legal or other status of any territory or area.