Participation Tools for the Pacific - Part 3: Design and Monitoring Framework

The project problem tree builds and helps clarify the problem that the project will address.

This is a requirement for the preparation of Concept Paper and also usually undertaken during TA factfinding (transaction [TRTA] or knowledge and support TA [KSTA]).

The problem tree is developed with the participation of the key stakeholder groups identified during the stakeholder analysis. It can be carried out in a half- to one-day workshop, depending on the nature and complexity of the development problem. It can also be performed in a series of smaller stakeholder workshops and the results of each merged into a comprehensive problem tree.

1. Starter problem. State the starter problem and place it at the center of the problem tree diagram. It may take several sessions to agree on what constitutes the starter problem, and it is important that a consensus is reached.
2. Direct causes. Using vertical logic, the problems that are the direct causes of the starter problem are added to the problem tree under the starter problem. Only existing problems, not anticipated future problems, should be included.
3. Direct causes to root causes. Step (ii) is repeated using direct causes as problems, and the direct causes of each of these problems are determined and placed below. The process is continued until the analysis is exhausted and very specific root causes are identified.
4. Direct effects. The direct effects of the starter problem are placed above the starter problem of the problem tree.
5. Direct effects to final effects. Step (iv) is repeated using direct effects as problems, and the effects of each of these problems are determined and placed above each statement.
6. Review and refine. The problem tree and the interrelationship of problems, causes, and effects at different levels are analyzed and adjusted accordingly.  Clarify through discussion and consultation that this will be the core problem and causes that ADB will address through its project.

Simplified Problem Tree Template

Source: Asian Development Bank Strategy, Policy, and Review Department.   

Problem tree clarifies the way forward for Samoa

CSO and government representatives met in Samoa and developed a problem tree to understand issues related to deepening ADB-government-CSO engagement. ADB convened the group to understand how it could best support this tripartite relationship in the Pacific through technical assistance.

Participants identified two to three issues each and wrote them down on colored paper; then each participant had an opportunity to explain his or her issues. The plenary grouped related issues together and presented them in a problem tree format. Participants then formed three groups to discuss the issues and come up with recommendations directed towards deepening ADB-government-CSO engagement. Each group reported the results of discussions to the rest of the participants. The discussions provided an opportunity for participants to clarify issues related to ADB-government and -civil society cooperation.

The participants made the following recommendations at the workshop:

• Raise awareness about ADB operations in the Pacific to optimize stakeholder participation and increase innovation in projects.
• Explore better ways to communicate since CSOs in the Pacific include village-level organizations and community associations who cannot access information easily, especially if it is only available on the ADB website.
• ADB engagement in the Pacific should be carefully aligned with its culture. Failure to understand the Pacific’s culture and traditional practices hinder effective engagement.

ADB. 2019. Guidelines for Preparing a Design and Monitoring Framework. Manila.

Australian Government AusAID. 2005. AusGuideline 3.3 The Logical Framework Approach. pp 26-27.

Pacific Research and Evaluation Associates. 2014. The Pacific Guide to Project Proposal Preparation Using the Logical Framework Approach: Learner Guide.

Beneficiary Assessment provides an inclusive approach to project design and greater ownership amongst beneficiaries. Insights can be gained from Beneficiary Assessment about how local people and beneficiaries perceive a project; it can help create a framework for how beneficiaries will be engaged in the project, from inception through to completion; and it can provide a roadmap for how non-project beneficiaries will participate in and engage with project activities. It is not designed to supplement traditional forms of data gathering for projects. Rather, it is a complement to ensure that projects are demand-driven and sustainable.

Beneficiary Assessment is a method is used for identifying and designing development projects, but can also be carried out during project implementation. Beneficiary Assessment usually takes between 3 to 5 months to implement in the Pacific and is a low-cost method, relative to project costs (The average cost of a Beneficiary Assessment is $40,000 to $60,000, excluding overhead costs).

The Beneficiary Assessment method may be directed by a social specialist, but it is important to use a team of locally-based interviewers (both male and female), fluent in local languages, with training and orientation in gender-sensitive approaches, conversational techniques and data collection methods provided to interviewers. CSOs can be engaged to facilitate a Beneficiary Assessment, including training of interviewers.

Three key tools are used in Beneficiary Assessment:

• Conversational Interviews are natural, free-flowing interviews which suit the cultural style of the Pacific, similar to talanoa. The features of conversational interviews are the establishment of mutual trust and respect; a good rapport must be established between interviewers and respondents. These interviews should take no more than 45 minutes to one hour, and note taking is not encouraged during the interview (interviewers should write up immediately afterwards). These conversational interviews occur with key people in a project affected area, such as school teachers, nurses and shop owners and are structured around a number of themes to gauge current scenarios and forecast likely perceptions of project impacts on affected persons.
• Focus Group Discussions (FGD) complement and cross-check the information from stakeholders and conversational interviews. FGDs often involve between 6-12 people with a common interest or characteristic (such as youth, young mothers, female entrepreneurs). FGDs are guided by a facilitator who ensures the discussion covers the topics of the investigation but ensuring inclusive participation. A researcher should take notes on the discussion. 
• Direct Observation and Participant Observation. Direct Observation involves a researcher directly observing/counting the activities and behaviors of a target group, such as women attending a community tap or houses undergoing improvement works. Participant Observation involves the researcher embedding him or herself in the community for a protracted period, ranging from several weeks to several months, to record and fully understand the activities of the target community. The emphasis is on understanding the socio-cultural and political context of the beneficiary community. This usually involves conducting a small number (5-10) of in-depth household case studies. When reporting, a Beneficiary Assessment table should incorporate the key findings. The template could include the following five columns: beneficiary group, expected benefits, possible negative impacts, relevance of project objectives to beneficiaries, measures to ensure benefits and to mitigate negative impacts.

Click here to download the sample template.

Beneficiary Assessment of the Smallholder Agriculture Development Project (SADP) in PNG

SADP’s development objective was to improve the living standards of rural communities in oil palm growing provinces in Papua New Guinea. It had two key outcomes: to increase smallholder oil palm sector productivity  (including additional oil palm planting and provincial road upgrading) and to promote sustainable local governance and community participation mechanisms (including providing small community grants). A Beneficiary Assessment was undertaken to explore and assess for each major beneficiary group: their strengths, needs and concerns; how social grouping affects opportunities to participate in SADP activities; distribution of project benefits amongst different social groupings; how each social grouping can facilitate or obstruct project activities; identification of the most vulnerable and powerful social groupings; strategies to promote beneficiaries' participation, particularly vulnerable groups; potential direct and indirect adverse impacts of the SADP; and means of mitigating adverse social impacts on different social groupings. A Beneficiary Assessment template was used to map out these issues, results, and actions.

Source: G. Koczberski and G.N. Curry. 2007. Beneficiaries Assessment Report for the Smallholder Agriculture Development Project (SADP), Papua New Guinea

Better Evaluation. Beneficiary Assessment.

L.F Salmen. 2002. Beneficiary Assessment: An Approach Described. World Bank. p10.

Source: G. Koczberski and G.N. Curry. 2007. Beneficiaries Assessment Report for the Smallholder Agriculture Development Project (SADP), Papua New Guinea.

World Bank Group. Beneficiary Assessment.

PIPA is particularly useful when used for gaining commitment and understanding of key stakeholders, or when two or more project teams wish to work better together. This approach clarifies to stakeholders why a project exists and its potential for achieving impact. Through participatory workshops, the PIPA helps stakeholders identify and discuss assumptions about how the project activities and outputs could contribute to project goals. PIPA is used to conduct an evaluation of likely project impacts and how they will occur as well as identify areas for collaboration with other projects so as to be able to programmatically integrate. PIPA provides a framework for learning-based Monitoring and Evaluation (M&E).

PIPA can be used at the beginning, middle or at the end of a project. Three days are required for the participant workshop.

Social or M&E specialist to train NGOs to undertake this work with project stakeholders. It is important to have project leaders involved in the workshop process.

At the heart of the PIPA process is a three-day participatory workshop aimed at engaging key project stakeholders. The workshop addresses the following points:

1. Developing a cause and effect logic, through a problem tree.
2. Developing a network perspective, through network mapping.
3. Developing the outcomes logic model and an M&E plan.

Day 1 of the participatory workshop is developed to exploring the intervention’s cause and effect logic. Stakeholders construct a problem tree of the issues.

Day 2 focuses on visioning success [see description of Visioning in Tools for Assessment] and determining what success looks like for different stakeholder groups (such as ‘next users’, end-users, politically important people or organizations and the project implementers).  Workshop participants draw two network maps to show the relationship between actors ‘now’ and in the ‘future’ to illustrate what is required to achieve ‘the vision’ by joining them together.

Day 3 connects these two perspectives (problem tree and network mapping) and integrates them through the development of an outcomes logic model. The outcomes logic model describes how stakeholders may act differently for the project vision to be achieved. The outcomes logic model links these outcomes to the stakeholders who can action this change, which in turn provides the basis for future M&E.  It is presented in table format. Following the workshop, the group may develop an impact logic model, which describes the impact logic: outputs, adoption, outcomes and long-term impact. The logic model’s outcomes “provide an ex-ante framework of predictions of impact that can also be used in priority setting and ex-post impact assessment. PIPA engages stakeholders in a structured participatory process, promoting learning and providing a framework for 'action research' on processes of change.”*

Click here to download a sample template.

 

*S. Alvarez et al. 2010. Participatory Impact Pathways Analysis: a practical method for project planning and evaluation. Development in Practice. 20(88). pp. 946-958.

S. Alvarez et al. 2010. Participatory Impact Pathways Analysis: a practical method for project planning and evaluation. Development in Practice. 20(88). pp. 946-958

B. Douthwaite, et al.  2008. Participatory Impact Pathway Analysis: A practical application of program theory in research-for-development. Canadian Journal of Program Evaluation. pp 22, 127-159. 

B. Douthwaite et al. 2008. Participatory Impact Pathways Analysis: A practical method for project planning and evaluation. ILAC Brief 17.

ILRI Tools Portal. PIPA (Participatory Impact Pathways Analysis).

USAID Learning Lab. 2013.