Participation Tools for the Pacific - Part 4: Assessment

Baseline data provides a reference point from which to measure change. Without this data, it is very difficult to design an effective program and then to measure the impact of the program. Baselining also gives opportunities for community engagement and buy-in.

Baselining usually occurs during the design phase of a project. However, sometimes baseline data may be retrospectively constructed during project implementation.

Baselining may be conducted by the project team, experts (statisticians, sociologists, economists, social specialists) and experienced CSOs. It should involve beneficiaries to build ownership of improving the baseline conditions.

Baselining takes a number of forms, ranging from simple exercises in ranking and scoring to complex household surveys involving high-level data collection and analysis. Two simple baselining tools are the Ten Seed Technique and Seasonal Calendars. The picture below shows baselining in low literacy settings.

Photo credit: Lisa McMurray

International Federation of Red Cross and Red Crescent Societies (IFRC). 2013. Baseline Basics. Geneva. IFRC.

FAO. What is a participatory baseline?

K. S. Freudenberger (no date) Rapid Rural Appraisal and Participatory Rural Appraisal: A Manual for Catholic Relief Services Field Workers and Partners. Maryland. CRS.

The Ten Seed Technique can be used at various points of the project cycle. It is especially useful in the assessment, problem identification, and solution identification stages.

An appealing factor of the ten seed technique is that it can be used by anyone and is mostly used by field-based project staff.

The Ten Seed Technique is simple to learn and very easy to use. It is especially useful in low literacy or low numeracy communities, as it can be achieved without text or numbers.

1. Identify the domain to be ranked (e.g. preferred farming practices, research questions, perception of health risks).
2. Brainstorm with a group of beneficiaries the range of potential options within the domain.
3. Draw the options in a circle on large paper, with one option per segment. This can also be done on the ground.
4. Give each participant 10 seeds (may be stones, sticks, red sticky dots, or bottle tops).
5. Each participant then ‘votes’ using their 10 seeds in order of preference.  They place all seeds or some seeds on the segment that they give priority to. They may spread their 10 seeds across 10 options, or five, or two, or place all 10 seeds on one option.
6. The facilitator then summarizes the findings, describing those options that received most seeds and those that receive less. This generates a discussion and debate among participants that allows for rich detail to emerge

Example of the Ten Seed Technique in use. Photo credit: Emma Walters.

Center for Sustainable Development. Ten Seed Technique.

R. Jayakaran. 2002. The Ten Seed Technique. World Vision International. China.

TorqAid. Ten Seed Technique (TST) and Seasonal Calendar.

A seasonal calendar exercise can be performed at any time. If mapping a short period, such as a day or week, the calendar may be completed in real time by the participant. If it is mapping a year, it can be completed by participants from memory. For this reason, seasonal calendars cannot be relied upon for complete accuracy and may be used in conjunction with other baselining tools.

Participants complete the seasonal calendar, with guidance from project or field staff, usually in a workshop setting. It may be appropriate to ask different groups to complete their own seasonal calendars (men/women) to see the gender differences in perceptions.

Seasonal calendars provide a visual representation of the changes in resources in a household or community. The visual approach of seasonal calendars works well in low literacy settings. It helps communities and project workers see the changes in resources over time (a day, a year) and what changes are desirable.

1. Determine the focus topics of the calendar. It may include:
   • Prevalence of disease
   • Food insecurity/scarcity
   • Lower price for sale of crops
   • Harvest
   • Rainfall
2. Draw a matrix on large paper or on the ground with the horizontal axis (columns) indicating each month of the year or season, and the vertical axis (rows) indicating the topics to be explored, using symbols to indicate each topic.
3. Ask the group to consider each row (topic) to discuss and agree when this occurs throughout the year. Start with rainfall, then follow with the planting of root crops, harvest and so on. Use stones or sticks or marks to indicate the relative amount of each item (e.g. months with more rainfall get more stones). 

Click here to download a sample Seasonal Calendar.

Better Evaluation. Seasonal Calendars.

FAO. PRA Toolbox. Seasonal Calendar.

TorqAid. Ten Seed Technique (TST) and Seasonal Calendar.

Involving the community in mapping its resources and characteristics provides rich data for project design and provides a more responsive and sustainable intervention. Mapping also provides baseline data for later comparison. Mapping is not only an end product (with a data or spatial map produced), it is also a process of building community awareness and engagement, plus gives an opportunity for self-determination or self-organization. In the Pacific, visual tools may generate more understanding an acceptance than heavily-text based approaches, and may bridge the linguistic divide between project designers and local communities.

Community mapping is most often performed in the project preparation and design stages, but it can also be useful during implementation for monitoring of a project, or in evaluation to compare baseline against endline data.

A social specialist can facilitate mapping exercises. Many local CSOs have mapping techniques or tools they regularly use and these may be engaged to work with local communities on mapping exercises.

Community mapping can be a highly technical process, using mobile phone technologies, global positioning systems, and spatial analysis software. Conversely, community maps can be as simple as stones and lines drawn in the sand, or drawings with crayons on large paper.

Mapping to identify village resources: municipality of Baucau, Suku Trilolo in Aldeia Lialailesu. Photo credit: Fernando Pires, Caritas.

 

Forms of community mapping beyond mapping natural resources and physical assets can include ‘historical mapping’, which may be used to map changes in environment and the likely causes and effects.

‘Social mapping’ illustrates household characteristics: income, use of resources, school attendance and social capital, or level of engagement in community groups, activities or projects.

A wide range of processes is involved in community mapping.

Community mapping in Fiji means better planning 

A non-profit organization, Community Architect Network (CAN), supports groups of young professionals to work on participatory design and planning and city-wide mapping and surveys, seeking design solutions to improve the quality of life and health of the community. Their community mapping handbook explains how people in Fiji used community mapping for the communities of Lautoka to come together and record their own situation, leading to citywide upgrading: 

“In the city of Lautoka, Fiji, satellite images from Google Earth were used to overlay maps of the individual settlements, distinguishing between those on state land and those on native land. This satellite imagery helped to pinpoint existing infrastructure and that which was lacking or degraded. It also made it possible to identify areas of vacant land that could serve as possible relocation sites. All of this information was compiled onto one large map that was presented to the Lands Department. This city map functions as a sort of “virtual” land bank with regard to vacant land. In conjunction with the city map, the community members also carried out “people mapping”, identifying key skills of community residents that could be useful during upgrading, such as carpentry, masonry and mat weaving.” 

“Following from this process a number of small-scale upgrading projects were launched and comprehensive site planning was initiated for the land close to the city. Unfortunately, this land has been found to be very flood prone and will not be developed. However, the design ideas and financial structures designed by the community can be applied to another site.” 

Source: Community Architects Network (CAN). Community Mapping for housing by people. Asian Coalition for Housing Rights.  

Community Architects Network (CAN). Community Mapping for housing by people. Asian Coalition for Housing Rights.

N. Eliasov. 2013. Asset Based and Community-Driven Development. Port Elizabeth: Ikhala Trust and Elamanzi.

Pacific Institute. Issues We Work On: Community Mapping Initiative.  

Transect Walks provide a useful data collection tool, that can be used to triangulate with other methods of data collection. By including a range of stakeholders, different groups’ understanding and perception of the issues and potential solutions can be explored.

Transect Walks are most often done during project preparation, often to assist with site selection, but can occur during any point of the project cycle: during project preparation, during project implementation and as part of monitoring and evaluation. If Transect Walks are conducted several times throughout the project cycle, the first walk may serve as a baseline for later walks, while the end will provide rich data for monitoring and evaluation purposes. Any changes – positive or negative – can be monitored by conducting several walks throughout the project cycle.

Transect walks should include a facilitator, project staff, and community members but also should include key stakeholders such as CSO representatives, government staff, private sector representatives, and any other relevant stakeholders with an interest in the project. Including a range of stakeholders provides diverse perspectives.

1. Identify the key stakeholders and invite them to conduct a Transect Walk.
2. Explain the purpose of the Transect Walk: to walk an imaginary line from one section of the community or project area to another.  Explain how the information from the walk will be used.
3. Decide the route of the Transect Walk: aim to get a good cross-section of the community or project area.
4. Divide the area into zones: for rural setting, this might be riverbank, field, communal washing area, or hilltop. For urban, it might be the marketplace, shop front, port, or church.
5. Decide the spheres of information you want about each zone: for rural it might be plants growing, soil conditions, erosion; for urban it might be institutions, financial assets, weekly purchases, accumulated rubbish.
6. As a group, conduct a walk across the different zones, noting findings about each of the spheres of information.
7. Map the findings on charts or tables (see downloadable template).
8. Analyze the findings by asking questions such as:
   
   1. What resources are abundant? What resources are scarce?
   2. Where do people get water and where does the water need to be?
   3. Where do people obtain firewood?
   4. Are there differences between the activities or presence of women and men in the different zones?
   5. What differences are there between zones, and why?
   6. What constraints or problems are there?
   7. What opportunities are there?
   8. What recommendations does this information lead to?

Click here to download a sample template.

S. Keller. Sustainable Sanitation and Water Management Toolbox - SSWM.info. Transect Walk.

CGIAR, FAO, KM4 Dev Community, UNICEF, UNDP.  Knowledge Sharing Toolkit. Transect Walk.

World Bank. Tool Name: Transect Walk.

Visioning is used to identify and articulate a common desired or preferred future among stakeholders. From this point of a democratically agreed future state consensus, stakeholders can then determine the necessary and logical steps to take them to this desired future state. Visioning is best conducted with a cross-representation of stakeholders so that key groups can identify common interests and agree on a preferred future state. Visioning uses participation as a source of ideas, to engage the community, discover community issues and build consensus.

Visioning is best used at the beginning of the planning process. It usually takes place after problem identification and development of a problem tree but before any decisions about activities and process are made.

A facilitator working with key stakeholder representatives; an ADB social development specialist or facilitator from a local NGO can conduct visioning exercises.

The facilitator asks a visioning question to the group of participants. The visioning process can either be visual or narrative. It is important that the facilitator encourage this process to be creative and free-flowing. Encourage participants to mentally wear a ‘white hat’, not a ‘black hat’. This means not focusing at this stage on problems and barriers to achieving the vision, but rather thinking expansively about ‘what success looks like’ for that community or project or organization.

The visioning exercise can start with the current state and then move onto the desired future state, or may simply start with the facilitator asking participants to draw or explain the desired future state. For a visual visioning exercise:

• Break participants into small groups with each stakeholder group represented in each group.
• Ask each group to draw a picture that shows the current state (such as “where are we now?” in terms of where this organization or community or development challenge). Ask each group to present their picture and describe the key points. The facilitator should draw out common themes across the group presentations.
• Then ask each group to draw what they would like their community or organization or development challenge to be like in 5 years’ time if it was highly successful and if the participants had the power and influence to make that happen.  The facilitator then asks each group to present its pictures and describe them in detail, and draw out common themes of what all groups would like to see.

For a narrative visioning exercise:

• The facilitator asks a question of the broader group about what the future would look like in 5 (or 10 or 15) years’ time. In the case of an organization aiming to expand and develop, the visioning question could be “Imagine that organization X is as successful as it possibly could be in 5 years’ time. What would it look like? What areas would it be active in? How would it support itself? What would its budget be? What resources would it have access to or own? How many staff would it have? What will the relationships with other stakeholders be?” and other questions to tease out what success looks like for that organization.
• The facilitator gathers the information from participants and puts it on a flip chart.

Visioning helps an NGO country team redesign its program 

Caritas Australia’s Timor Leste program initiated a comprehensive design initiative in 2015-2016, following a changing country context and a recent project evaluation. The exercise began with a community mapping exercise and then included visioning activities.  The team focused on two of their four key outcomes areas, Just and Peaceful Relationships, and Sustained and Economic Wellbeing. Communities members discussed, based on their mapping results, what these two outcomes looked like to them. Visioning allowed the communities to 'dream' of a desired scenario and then work backwards to establish a pathway to achieve the vision. 

The re-design took more than a year, but the team recognized the need to take adequate time to complete a thorough and comprehensive program design with meaningful community participation. Once the mapping and visioning exercises concluded, the team led two three-day-design workshops. Each workshop included key Caritas representatives and key partners. Building upon the findings from the consultations, partners, and staff collaboratively build two program logics, one for Protection and another for Sustainable Livelihoods.  From these two program logics, the teams designed a monitoring and evaluation framework to assesses key outcomes, intermediate, and immediate changes. 

Source: Lisa McMurray and Caritas Australia.  

N. Eliasov. 2013. Asset Based and Community-driven Development. Port Elizabeth: Ikhala Trust and Elamanzi.

DFID. 2003. Tools for Development: A handbook for those engaged in development activity. DFID.

Victorian Government Department of Sustainability and Environment. 2005. Effective Engagement: building relationships with community and other stakeholders – Book 3 The Engagement Toolkit. Victorian Government. Melbourne. 

Sustainable Sanitation and Water Management Toolbox. Visioning.