Introduction What you need to know Engagement of key stakeholder groups in operations financed by the Asian Development Bank (ADB) promotes good governance, transparency, innovation, responsiveness, and development effectiveness. Effective engagement of stakeholder groups, including civil society, project beneficiaries, and project-affected people, requires the understanding and effective use of participatory tools throughout the project cycle. However, while one participatory tool may work well in one context, it may not be appropriate in another. This series of explainers provides a range of tools from which practitioners can pick and choose, according to different phases of the ADB project cycle, context, and available time/resources. Some tools may be specific to particular phases in the ADB project cycle, such as monitoring and evaluation tools, while others may be used throughout the project cycle, such as participatory assessment tools. For this piece, the focus is on Tools for Preparing the Design and Monitoring Framework. Tools for Preparing the Design and Monitoring Framework The Design and Monitoring Framework must be founded on a participatory approach for the following reasons: ADB-assisted projects should be designed to respond to the needs of beneficiaries and be designed with the direct involvement of those beneficiaries. Participatory approaches build ownership of projects by beneficiaries and key stakeholders. A group process usually creates a better and more relevant design and monitoring framework. Problem Tree Problem analysis is one of the key steps of the design and monitoring process. It is done by developing a problem tree, which should be conducted in a participatory way. What This tool is used to: Analyze the existing situation surrounding a given problem context, Identify the major problems and constraints associated with the problem, and Visualize the cause-effect relationship diagrammatically as a problem tree. Why The project problem tree builds and helps clarify the problem that the project will address. When This is a requirement for the preparation of Concept Paper and also usually undertaken during TA factfinding (transaction [TRTA] or knowledge and support TA [KSTA]). Who The problem tree is developed with the participation of the key stakeholder groups identified during the stakeholder analysis. It can be carried out in a half- to one-day workshop, depending on the nature and complexity of the development problem. It can also be performed in a series of smaller stakeholder workshops and the results of each merged into a comprehensive problem tree. How Starter problem. State the starter problem and place it at the center of the problem tree diagram. It may take several sessions to agree on what constitutes the starter problem, and it is important that a consensus is reached. Direct causes. Using vertical logic, the problems that are the direct causes of the starter problem are added to the problem tree under the starter problem. Only existing problems, not anticipated future problems, should be included. Direct causes to root causes. Step (ii) is repeated using direct causes as problems, and the direct causes of each of these problems are determined and placed below. The process is continued until the analysis is exhausted and very specific root causes are identified. Direct effects. The direct effects of the starter problem are placed above the starter problem of the problem tree. Direct effects to final effects. Step (iv) is repeated using direct effects as problems, and the effects of each of these problems are determined and placed above each statement. Review and refine. The problem tree and the interrelationship of problems, causes, and effects at different levels are analyzed and adjusted accordingly. Clarify through discussion and consultation that this will be the core problem and causes that ADB will address through its project. Simplified Problem Tree Template Source: Asian Development Bank Strategy, Policy, and Review Department. Case Study Problem tree clarifies the way forward for Samoa CSO and government representatives met in Samoa and developed a problem tree to understand issues related to deepening ADB-government-CSO engagement. ADB convened the group to understand how it could best support this tripartite relationship in the Pacific through technical assistance. Participants identified two to three issues each and wrote them down on colored paper; then each participant had an opportunity to explain his or her issues. The plenary grouped related issues together and presented them in a problem tree format. Participants then formed three groups to discuss the issues and come up with recommendations directed towards deepening ADB-government-CSO engagement. Each group reported the results of discussions to the rest of the participants. The discussions provided an opportunity for participants to clarify issues related to ADB-government and -civil society cooperation. The participants made the following recommendations at the workshop: Raise awareness about ADB operations in the Pacific to optimize stakeholder participation and increase innovation in projects. Explore better ways to communicate since CSOs in the Pacific include village-level organizations and community associations who cannot access information easily, especially if it is only available on the ADB website. ADB engagement in the Pacific should be carefully aligned with its culture. Failure to understand the Pacific’s culture and traditional practices hinder effective engagement. References and Further Reading ADB. 2019. Guidelines for Preparing a Design and Monitoring Framework. Manila. Australian Government AusAID. 2005. AusGuideline 3.3 The Logical Framework Approach. pp 26-27. Pacific Research and Evaluation Associates. 2014. The Pacific Guide to Project Proposal Preparation Using the Logical Framework Approach: Learner Guide. Beneficiary Assessment What Beneficiary Assessment is a method used by the World Bank and other institutions at the design phase of programs. Beneficiary Assessment is a qualitative tool used to improve the impact of development work by systematic consultation with project stakeholders – including the vulnerable and poor – to ensure that their concerns are incorporated into the project design. It is a “systematic inquiry into people’s values and behavior in relation to a planned or ongoing intervention for social or economic change.” Why Beneficiary Assessment provides an inclusive approach to project design and greater ownership amongst beneficiaries. Insights can be gained from Beneficiary Assessment about how local people and beneficiaries perceive a project; it can help create a framework for how beneficiaries will be engaged in the project, from inception through to completion; and it can provide a roadmap for how non-project beneficiaries will participate in and engage with project activities. It is not designed to supplement traditional forms of data gathering for projects. Rather, it is a complement to ensure that projects are demand-driven and sustainable. When Beneficiary Assessment is a method is used for identifying and designing development projects, but can also be carried out during project implementation. Beneficiary Assessment usually takes between 3 to 5 months to implement in the Pacific and is a low-cost method, relative to project costs (The average cost of a Beneficiary Assessment is $40,000 to $60,000, excluding overhead costs). Who The Beneficiary Assessment method may be directed by a social specialist, but it is important to use a team of locally-based interviewers (both male and female), fluent in local languages, with training and orientation in gender-sensitive approaches, conversational techniques and data collection methods provided to interviewers. CSOs can be engaged to facilitate a Beneficiary Assessment, including training of interviewers. How Three key tools are used in Beneficiary Assessment: Conversational Interviews are natural, free-flowing interviews which suit the cultural style of the Pacific, similar to talanoa. The features of conversational interviews are the establishment of mutual trust and respect; a good rapport must be established between interviewers and respondents. These interviews should take no more than 45 minutes to one hour, and note taking is not encouraged during the interview (interviewers should write up immediately afterwards). These conversational interviews occur with key people in a project affected area, such as school teachers, nurses and shop owners and are structured around a number of themes to gauge current scenarios and forecast likely perceptions of project impacts on affected persons. Focus Group Discussions (FGD) complement and cross-check the information from stakeholders and conversational interviews. FGDs often involve between 6-12 people with a common interest or characteristic (such as youth, young mothers, female entrepreneurs). FGDs are guided by a facilitator who ensures the discussion covers the topics of the investigation but ensuring inclusive participation. A researcher should take notes on the discussion. Direct Observation and Participant Observation. Direct Observation involves a researcher directly observing/counting the activities and behaviors of a target group, such as women attending a community tap or houses undergoing improvement works. Participant Observation involves the researcher embedding him or herself in the community for a protracted period, ranging from several weeks to several months, to record and fully understand the activities of the target community. The emphasis is on understanding the socio-cultural and political context of the beneficiary community. This usually involves conducting a small number (5-10) of in-depth household case studies. When reporting, a Beneficiary Assessment table should incorporate the key findings. The template could include the following five columns: beneficiary group, expected benefits, possible negative impacts, relevance of project objectives to beneficiaries, measures to ensure benefits and to mitigate negative impacts. Click here to download the sample template. Case Study Beneficiary Assessment of the Smallholder Agriculture Development Project (SADP) in PNG SADP’s development objective was to improve the living standards of rural communities in oil palm growing provinces in Papua New Guinea. It had two key outcomes: to increase smallholder oil palm sector productivity (including additional oil palm planting and provincial road upgrading) and to promote sustainable local governance and community participation mechanisms (including providing small community grants). A Beneficiary Assessment was undertaken to explore and assess for each major beneficiary group: their strengths, needs and concerns; how social grouping affects opportunities to participate in SADP activities; distribution of project benefits amongst different social groupings; how each social grouping can facilitate or obstruct project activities; identification of the most vulnerable and powerful social groupings; strategies to promote beneficiaries' participation, particularly vulnerable groups; potential direct and indirect adverse impacts of the SADP; and means of mitigating adverse social impacts on different social groupings. A Beneficiary Assessment template was used to map out these issues, results, and actions. Source: G. Koczberski and G.N. Curry. 2007. Beneficiaries Assessment Report for the Smallholder Agriculture Development Project (SADP), Papua New Guinea References and Further Reading Better Evaluation. Beneficiary Assessment. L.F Salmen. 2002. Beneficiary Assessment: An Approach Described. World Bank. p10. Source: G. Koczberski and G.N. Curry. 2007. Beneficiaries Assessment Report for the Smallholder Agriculture Development Project (SADP), Papua New Guinea. World Bank Group. Beneficiary Assessment. Participatory Impact Pathways Analysis What Participatory Impact Pathways Analysis (PIPA) is a practical approach designed to help project implementers and key stakeholders explain how the project activities and outputs could contribute to project goals. It works best for complex projects or where two or more projects within a program wish to coordinate. Examples where coordinating could occur in an ADB project: a water supply system and sanitation scheme designed as two separate projects for the same city or an integrated sustainable urban development plan that covers transport projects, infrastructure provision, sustainable waste management, slum upgrading, and policy reform. (e.g. Fiji, Vanuatu) As a participatory process for design, monitoring, and evaluation, it goes beyond the traditional logical framework approach to engage stakeholders using participatory approaches. At the core of PIPA is a three-day participatory workshop. Why PIPA is particularly useful when used for gaining commitment and understanding of key stakeholders, or when two or more project teams wish to work better together. This approach clarifies to stakeholders why a project exists and its potential for achieving impact. Through participatory workshops, the PIPA helps stakeholders identify and discuss assumptions about how the project activities and outputs could contribute to project goals. PIPA is used to conduct an evaluation of likely project impacts and how they will occur as well as identify areas for collaboration with other projects so as to be able to programmatically integrate. PIPA provides a framework for learning-based Monitoring and Evaluation (M&E). When PIPA can be used at the beginning, middle or at the end of a project. Three days are required for the participant workshop. Who Social or M&E specialist to train NGOs to undertake this work with project stakeholders. It is important to have project leaders involved in the workshop process. How At the heart of the PIPA process is a three-day participatory workshop aimed at engaging key project stakeholders. The workshop addresses the following points: Developing a cause and effect logic, through a problem tree. Developing a network perspective, through network mapping. Developing the outcomes logic model and an M&E plan. Day 1 of the participatory workshop is developed to exploring the intervention’s cause and effect logic. Stakeholders construct a problem tree of the issues. Day 2 focuses on visioning success [see description of Visioning in Tools for Assessment] and determining what success looks like for different stakeholder groups (such as ‘next users’, end-users, politically important people or organizations and the project implementers). Workshop participants draw two network maps to show the relationship between actors ‘now’ and in the ‘future’ to illustrate what is required to achieve ‘the vision’ by joining them together. Day 3 connects these two perspectives (problem tree and network mapping) and integrates them through the development of an outcomes logic model. The outcomes logic model describes how stakeholders may act differently for the project vision to be achieved. The outcomes logic model links these outcomes to the stakeholders who can action this change, which in turn provides the basis for future M&E. It is presented in table format. Following the workshop, the group may develop an impact logic model, which describes the impact logic: outputs, adoption, outcomes and long-term impact. The logic model’s outcomes “provide an ex-ante framework of predictions of impact that can also be used in priority setting and ex-post impact assessment. PIPA engages stakeholders in a structured participatory process, promoting learning and providing a framework for 'action research' on processes of change.”* Click here to download a sample template. *S. Alvarez et al. 2010. Participatory Impact Pathways Analysis: a practical method for project planning and evaluation. Development in Practice. 20(88). pp. 946-958. References and Further Reading S. Alvarez et al. 2010. Participatory Impact Pathways Analysis: a practical method for project planning and evaluation. Development in Practice. 20(88). pp. 946-958 B. Douthwaite, et al. 2008. Participatory Impact Pathway Analysis: A practical application of program theory in research-for-development. Canadian Journal of Program Evaluation. pp 22, 127-159. B. Douthwaite et al. 2008. Participatory Impact Pathways Analysis: A practical method for project planning and evaluation. ILAC Brief 17. ILRI Tools Portal. PIPA (Participatory Impact Pathways Analysis). USAID Learning Lab. 2013. Some Do’s and Don’ts The following suggestions for inclusive engagement with civil society organizations at the design stage come from AusAID’s Guidance on M&E for Civil Society Programs: Do Engage in effective power and gender analysis as key drivers of change Focus on building a common vision of change tailored to the local context Ensure that there is an appropriate and representative mix of perspectives involved, based on the power and gender analysis Ensure that adequate time, space and resources are built into the design for ongoing reflection, sense-making, and learning Ensure gender is properly considered at analysis and design Don’t Push CSOs or subcontractors working with CSOs to focus on narrow objectives at the expense of the process and creation of trust Assume that the design or analysis is correct; it will need to evolve over time Compartmentalize CSOs - respect their mandates and autonomy React punitively if your policies are challenged by CSOs; try and navigate different views with curiosity Related Links Participation Tools for the Pacific - Part 1: Engaging Pacific Civil Society Organizations Participation Tools for the Pacific - Part 2: Stakeholder Analysis Participation Tools for the Pacific - Part 4: Assessment Participation Tools for the Pacific - Part 5: Implementation Participation Tools for the Pacific - Part 6: Monitoring and Evaluation Downloadable Copy Participation Tools for the Pacific - All Parts Participation Tools for the Pacific - Part 3 Ask the Experts Lainie C. Thomas Principal Operations Coordination Specialist (Conflict-Affected Situations), Sustainable Development and Climate Change Department, Asian Development Bank Prior to her current assignment, Lainie supported civil society participation in ADB’s operations through its NGO and Civil Society Center and led health and education projects in Southeast Asia. Before joining ADB, she worked for a range of international and local nongovernment organizations. With more than 25 years of development experience, she has also worked in the field in Kenya, South Sudan, Azerbaijan, Somalia, Cambodia, and The Gambia. Emma Walters Management and Participatory Engagement Consultant, Asian Development Bank Emma has 20 years’ experience working with Australian and international organizations. Over the past eight years, she has worked on projects designed to increase civil society participation in ADB work, with a focus on the Pacific. Her expertise includes facilitation, training, and partnerships for international development. Since 2012, she has also worked with the University of Sydney as a soft-skills trainer on Australian-government funded short course programs on agricultural research in Africa. Suzanne M. Nazal Senior Social Development Officer (Civil Society and Participation), Sustainable Development and Climate Change Department, Asian Development Bank Suzanne organizes knowledge events, develops knowledge products, and provides technical support related to strengthening civil society participation and engagement in ADB’s operations. She also co-manages the social media platforms for ADB’s NGO and Civil Society Center. Suzanne worked with several NGOs in the Philippines before joining ADB. She holds a master's degree in urban and regional planning from the University of the Philippines. Ninebeth S. Carandang Principal Social Development Specialist, Pacific Department, Asian Development Bank Ninebeth is in charge of ensuring safeguards compliance of ADB-funded projects in the Pacific region, supporting the engagement of civil society organizations at various stages of the project, and managing the resolution of project grievances including concerns raised by civil society. She has over a decade of work experience in the field of social development, poverty, and gender. Asian Development Bank (ADB) The Asian Development Bank is committed to achieving a prosperous, inclusive, resilient, and sustainable Asia and the Pacific, while sustaining its efforts to eradicate extreme poverty. Established in 1966, it is owned by 68 members—49 from the region. Its main instruments for helping its developing member countries are policy dialogue, loans, equity investments, guarantees, grants, and technical assistance. Follow Asian Development Bank (ADB) on Leave your question or comment in the section below: View the discussion thread.