Participation Tools for the Pacific - Part 3: Design and Monitoring Framework

Use of participatory processes in preparing the design and monitoring framework is important for ensuring sustainability. Photo credit: ADB.

Share on:           

Published:

Problem Tree, Beneficiary Assessment, and Participatory Impact Pathways Analysis are useful tools when preparing the design and monitoring framework for CSO engagement.

Introduction

What you need to know

Engagement of key stakeholder groups in operations financed by the Asian Development Bank (ADB) promotes good governance, transparency, innovation, responsiveness, and development effectiveness.  Effective engagement of stakeholder groups, including civil society, project beneficiaries, and project-affected people, requires the understanding and effective use of participatory tools throughout the project cycle. However, while one participatory tool may work well in one context, it may not be appropriate in another. This series of explainers provides a range of tools from which practitioners can pick and choose, according to different phases of the ADB project cycle, context, and available time/resources.  Some tools may be specific to particular phases in the ADB project cycle, such as monitoring and evaluation tools, while others may be used throughout the project cycle, such as participatory assessment tools.

For this piece, the focus is on Tools for Preparing the Design and Monitoring Framework.

Tools for Preparing the Design and Monitoring Framework

The Design and Monitoring Framework must be founded on a participatory approach for the following reasons:

  • ADB-assisted projects should be designed to respond to the needs of beneficiaries and be designed with the direct involvement of those beneficiaries.
  • Participatory approaches build ownership of projects by beneficiaries and key stakeholders.
  • A group process usually creates a better and more relevant design and monitoring framework.

Problem Tree

Problem analysis is one of the key steps of the design and monitoring process.  It is done by developing a problem tree, which should be conducted in a participatory way.

This tool is used to:

  • Analyze the existing situation surrounding a given problem context,
  • Identify the major problems and constraints associated with the problem, and
  • Visualize the cause-effect relationship diagrammatically as a problem tree.

The project problem tree builds and helps clarify the problem that the project will address.

This is a requirement for the preparation of Concept Paper and also usually undertaken during TA factfinding (transaction [TRTA] or knowledge and support TA [KSTA]).

The problem tree is developed with the participation of the key stakeholder groups identified during the stakeholder analysis. It can be carried out in a half- to one-day workshop, depending on the nature and complexity of the development problem. It can also be performed in a series of smaller stakeholder workshops and the results of each merged into a comprehensive problem tree.

  1. Starter problem. State the starter problem and place it at the center of the problem tree diagram. It may take several sessions to agree on what constitutes the starter problem, and it is important that a consensus is reached.
  2. Direct causes. Using vertical logic, the problems that are the direct causes of the starter problem are added to the problem tree under the starter problem. Only existing problems, not anticipated future problems, should be included.
  3. Direct causes to root causes. Step (ii) is repeated using direct causes as problems, and the direct causes of each of these problems are determined and placed below. The process is continued until the analysis is exhausted and very specific root causes are identified.
  4. Direct effects. The direct effects of the starter problem are placed above the starter problem of the problem tree.
  5. Direct effects to final effects. Step (iv) is repeated using direct effects as problems, and the effects of each of these problems are determined and placed above each statement.
  6. Review and refine. The problem tree and the interrelationship of problems, causes, and effects at different levels are analyzed and adjusted accordingly.  Clarify through discussion and consultation that this will be the core problem and causes that ADB will address through its project.

Simplified Problem Tree Template

Source: Asian Development Bank Strategy, Policy, and Review Department.   

Problem tree clarifies the way forward for Samoa

CSO and government representatives met in Samoa and developed a problem tree to understand issues related to deepening ADB-government-CSO engagement. ADB convened the group to understand how it could best support this tripartite relationship in the Pacific through technical assistance.

Participants identified two to three issues each and wrote them down on colored paper; then each participant had an opportunity to explain his or her issues. The plenary grouped related issues together and presented them in a problem tree format. Participants then formed three groups to discuss the issues and come up with recommendations directed towards deepening ADB-government-CSO engagement. Each group reported the results of discussions to the rest of the participants. The discussions provided an opportunity for participants to clarify issues related to ADB-government and -civil society cooperation.

The participants made the following recommendations at the workshop:

  • Raise awareness about ADB operations in the Pacific to optimize stakeholder participation and increase innovation in projects.
  • Explore better ways to communicate since CSOs in the Pacific include village-level organizations and community associations who cannot access information easily, especially if it is only available on the ADB website.
  • ADB engagement in the Pacific should be carefully aligned with its culture. Failure to understand the Pacific’s culture and traditional practices hinder effective engagement.

ADB. 2019. Guidelines for Preparing a Design and Monitoring Framework. Manila.

Australian Government AusAID. 2005. AusGuideline 3.3 The Logical Framework Approach. pp 26-27.

Pacific Research and Evaluation Associates. 2014. The Pacific Guide to Project Proposal Preparation Using the Logical Framework Approach: Learner Guide.

Beneficiary Assessment

Beneficiary Assessment is a method used by the World Bank and other institutions at the design phase of programs. Beneficiary Assessment is a qualitative tool used to improve the impact of development work by systematic consultation with project stakeholders – including the vulnerable and poor – to ensure that their concerns are incorporated into the project design. It is a “systematic inquiry into people’s values and behavior in relation to a planned or ongoing intervention for social or economic change.”

Beneficiary Assessment provides an inclusive approach to project design and greater ownership amongst beneficiaries. Insights can be gained from Beneficiary Assessment about how local people and beneficiaries perceive a project; it can help create a framework for how beneficiaries will be engaged in the project, from inception through to completion; and it can provide a roadmap for how non-project beneficiaries will participate in and engage with project activities. It is not designed to supplement traditional forms of data gathering for projects. Rather, it is a complement to ensure that projects are demand-driven and sustainable.

Beneficiary Assessment is a method is used for identifying and designing development projects, but can also be carried out during project implementation. Beneficiary Assessment usually takes between 3 to 5 months to implement in the Pacific and is a low-cost method, relative to project costs (The average cost of a Beneficiary Assessment is $40,000 to $60,000, excluding overhead costs).

The Beneficiary Assessment method may be directed by a social specialist, but it is important to use a team of locally-based interviewers (both male and female), fluent in local languages, with training and orientation in gender-sensitive approaches, conversational techniques and data collection methods provided to interviewers. CSOs can be engaged to facilitate a Beneficiary Assessment, including training of interviewers.

Three key tools are used in Beneficiary Assessment:

  • Conversational Interviews are natural, free-flowing interviews which suit the cultural style of the Pacific, similar to talanoa. The features of conversational interviews are the establishment of mutual trust and respect; a good rapport must be established between interviewers and respondents. These interviews should take no more than 45 minutes to one hour, and note taking is not encouraged during the interview (interviewers should write up immediately afterwards). These conversational interviews occur with key people in a project affected area, such as school teachers, nurses and shop owners and are structured around a number of themes to gauge current scenarios and forecast likely perceptions of project impacts on affected persons.
  • Focus Group Discussions (FGD) complement and cross-check the information from stakeholders and conversational interviews. FGDs often involve between 6-12 people with a common interest or characteristic (such as youth, young mothers, female entrepreneurs). FGDs are guided by a facilitator who ensures the discussion covers the topics of the investigation but ensuring inclusive participation. A researcher should take notes on the discussion. 
  • Direct Observation and Participant Observation. Direct Observation involves a researcher directly observing/counting the activities and behaviors of a target group, such as women attending a community tap or houses undergoing improvement works. Participant Observation involves the researcher embedding him or herself in the community for a protracted period, ranging from several weeks to several months, to record and fully understand the activities of the target community. The emphasis is on understanding the socio-cultural and political context of the beneficiary community. This usually involves conducting a small number (5-10) of in-depth household case studies. When reporting, a Beneficiary Assessment table should incorporate the key findings. The template could include the following five columns: beneficiary group, expected benefits, possible negative impacts, relevance of project objectives to beneficiaries, measures to ensure benefits and to mitigate negative impacts.

Click here to download the sample template.

Beneficiary Assessment of the Smallholder Agriculture Development Project (SADP) in PNG

SADP’s development objective was to improve the living standards of rural communities in oil palm growing provinces in Papua New Guinea. It had two key outcomes: to increase smallholder oil palm sector productivity  (including additional oil palm planting and provincial road upgrading) and to promote sustainable local governance and community participation mechanisms (including providing small community grants). A Beneficiary Assessment was undertaken to explore and assess for each major beneficiary group: their strengths, needs and concerns; how social grouping affects opportunities to participate in SADP activities; distribution of project benefits amongst different social groupings; how each social grouping can facilitate or obstruct project activities; identification of the most vulnerable and powerful social groupings; strategies to promote beneficiaries' participation, particularly vulnerable groups; potential direct and indirect adverse impacts of the SADP; and means of mitigating adverse social impacts on different social groupings. A Beneficiary Assessment template was used to map out these issues, results, and actions.

Source: G. Koczberski and G.N. Curry. 2007. Beneficiaries Assessment Report for the Smallholder Agriculture Development Project (SADP), Papua New Guinea

Participatory Impact Pathways Analysis

Participatory Impact Pathways Analysis (PIPA) is a practical approach designed to help project implementers and key stakeholders explain how the project activities and outputs could contribute to project goals. It works best for complex projects or where two or more projects within a program wish to coordinate.

Examples where coordinating could occur in an ADB project: a water supply system and sanitation scheme designed as two separate projects for the same city or an integrated sustainable urban development plan that covers transport projects, infrastructure provision, sustainable waste management, slum upgrading, and policy reform. (e.g. Fiji, Vanuatu)

As a participatory process for design, monitoring, and evaluation, it goes beyond the traditional logical framework approach to engage stakeholders using participatory approaches. At the core of PIPA is a three-day participatory workshop.

PIPA is particularly useful when used for gaining commitment and understanding of key stakeholders, or when two or more project teams wish to work better together. This approach clarifies to stakeholders why a project exists and its potential for achieving impact. Through participatory workshops, the PIPA helps stakeholders identify and discuss assumptions about how the project activities and outputs could contribute to project goals. PIPA is used to conduct an evaluation of likely project impacts and how they will occur as well as identify areas for collaboration with other projects so as to be able to programmatically integrate. PIPA provides a framework for learning-based Monitoring and Evaluation (M&E).

PIPA can be used at the beginning, middle or at the end of a project. Three days are required for the participant workshop.

Social or M&E specialist to train NGOs to undertake this work with project stakeholders. It is important to have project leaders involved in the workshop process.

At the heart of the PIPA process is a three-day participatory workshop aimed at engaging key project stakeholders. The workshop addresses the following points:

  1. Developing a cause and effect logic, through a problem tree.
  2. Developing a network perspective, through network mapping.
  3. Developing the outcomes logic model and an M&E plan.

Day 1 of the participatory workshop is developed to exploring the intervention’s cause and effect logic. Stakeholders construct a problem tree of the issues.

Day 2 focuses on visioning success [see description of Visioning in Tools for Assessment] and determining what success looks like for different stakeholder groups (such as ‘next users’, end-users, politically important people or organizations and the project implementers).  Workshop participants draw two network maps to show the relationship between actors ‘now’ and in the ‘future’ to illustrate what is required to achieve ‘the vision’ by joining them together.

Day 3 connects these two perspectives (problem tree and network mapping) and integrates them through the development of an outcomes logic model. The outcomes logic model describes how stakeholders may act differently for the project vision to be achieved. The outcomes logic model links these outcomes to the stakeholders who can action this change, which in turn provides the basis for future M&E.  It is presented in table format. Following the workshop, the group may develop an impact logic model, which describes the impact logic: outputs, adoption, outcomes and long-term impact. The logic model’s outcomes “provide an ex-ante framework of predictions of impact that can also be used in priority setting and ex-post impact assessment. PIPA engages stakeholders in a structured participatory process, promoting learning and providing a framework for 'action research' on processes of change.”*

Click here to download a sample template.

 

*S. Alvarez et al. 2010. Participatory Impact Pathways Analysis: a practical method for project planning and evaluation. Development in Practice. 20(88). pp. 946-958.

S. Alvarez et al. 2010. Participatory Impact Pathways Analysis: a practical method for project planning and evaluation. Development in Practice. 20(88). pp. 946-958

B. Douthwaite, et al.  2008. Participatory Impact Pathway Analysis: A practical application of program theory in research-for-development. Canadian Journal of Program Evaluation. pp 22, 127-159. 

B. Douthwaite et al. 2008. Participatory Impact Pathways Analysis: A practical method for project planning and evaluation. ILAC Brief 17.

ILRI Tools Portal. PIPA (Participatory Impact Pathways Analysis).

USAID Learning Lab. 2013.

 

Some Do’s and Don’ts

The following suggestions for inclusive engagement with civil society organizations at the design stage come from AusAID’s Guidance on M&E for Civil Society Programs:

Do

  • Engage in effective power and gender analysis as key drivers of change
  • Focus on building a common vision of change tailored to the local context
  • Ensure that there is an appropriate and representative mix of perspectives involved, based on the power and gender analysis
  • Ensure that adequate time, space and resources are built into the design for ongoing reflection, sense-making, and learning
  • Ensure gender is properly considered at analysis and design

Don’t

  • Push CSOs or subcontractors working with CSOs to focus on narrow objectives at the expense of the process and creation of trust
  • Assume that the design or analysis is correct; it will need to evolve over time
  • Compartmentalize CSOs - respect their mandates and autonomy
  • React punitively if your policies are challenged by CSOs; try and navigate different views with curiosity

 

Lainie C. Thomas
Unit Head, Climate Change and Sustainable Development Department, Asian Development Bank

Lainie Thomas heads the Fragile and Conflict-Affected Situations (FCAS) and Small Island Developing States (SIDS) unit. Prior to her current assignment, she supported civil society participation in ADB’s operations through its NGO and Civil Society Center and led health and education projects in Southeast Asia. Before joining ADB, she worked for a range of international and local nongovernment organizations. With more than 25 years of development experience, she has also worked in the field in Kenya, South Sudan, Azerbaijan, Somalia, Cambodia, and The Gambia.

Emma Walters
Management and Participatory Engagement Consultant, Asian Development Bank

Emma has 20 years’ experience working with Australian and international organizations. Over the past eight years, she has worked on projects designed to increase civil society participation in ADB work, with a focus on the Pacific. Her expertise includes facilitation, training, and partnerships for international development. Since 2012, she has also worked with the University of Sydney as a soft-skills trainer on Australian-government funded short course programs on agricultural research in Africa.

Suzanne M. Nazal
Senior Social Development Consultant (Civil Society and Participation), Sustainable Development and Climate Change Department, Asian Development Bank

Suzanne Nazal organizes knowledge events, develops knowledge products, and provides technical support related to strengthening civil society participation and engagement in ADB’s operations. She also co-manages the social media platforms for ADB’s NGO and Civil Society Center. Suzanne worked with several NGOs in the Philippines before joining ADB. She holds a master's degree in urban and regional planning from the University of the Philippines.

Ninebeth S. Carandang
Principal Social Development Specialist, Human and Social Development Sector Office, Sectors Group

Ninebeth Carandang is in charge of ensuring safeguards compliance of ADB-funded projects in the Pacific region, supporting the engagement of civil society organizations at various stages of the project, and managing the resolution of project grievances including concerns raised by civil society. She has over a decade of work experience in the field of social development, poverty, and gender.

Asian Development Bank (ADB)

The Asian Development Bank is committed to achieving a prosperous, inclusive, resilient, and sustainable Asia and the Pacific, while sustaining its efforts to eradicate extreme poverty. Established in 1966, it is owned by 68 members—49 from the region. Its main instruments for helping its developing member countries are policy dialogue, loans, equity investments, guarantees, grants, and technical assistance.

Follow Asian Development Bank (ADB) on
Leave your question or comment in the section below:
Disclaimer

The views expressed on this website are those of the authors and do not necessarily reflect the views and policies of the Asian Development Bank (ADB) or its Board of Governors or the governments they represent. ADB does not guarantee the accuracy of the data included in this publication and accepts no responsibility for any consequence of their use. By making any designation of or reference to a particular territory or geographic area, or by using the term “country” in this document, ADB does not intend to make any judgments as to the legal or other status of any territory or area.