ToR-Final Performance Evaluation for Emergency Response and Economic Recovery Program for Eastern DRC Full Time NEW

Background

General information about Samaritan’s Purse and the DRC context

Samaritan’s Purse is a non-denominational evangelical Christian organization that provides spiritual and physical relief to people in need around the world. As a trusted Christian organization for over 50 years, Samaritan’s Purse has helped meet the needs of people affected by war, poverty, natural disasters, disease, and famine with the goal of sharing God’s love through His Son, Jesus Christ.

The Democratic Republic of the Congo (DRC) remains the most food-insecure country in the world according to the Integrated Food Security Phase Classification (IPC). According to the IPC Global Dashboard, 22,436,761 people in the DRC are in Phase 3 and above of the IPC scale (above the food crisis level). Another 50,950,544 Congolese are in Phase 2, or “stressed” food insecurity. One of the main reasons for food insecurity in the DRC is violence and displacement, particularly in the eastern provinces of the country. Internally displaced Congolese, host communities and returnees remain vulnerable, as they often flee their home villages with few of their belongings and lose their land, livestock, homes and livelihoods.

Full Statement of Work for Evaluation

Evaluation Purpose

The purpose of the proposed Evaluation is to evaluate the effectiveness and relevance of Samaritan’s Purse’s Food Assistance, Water, Sanitation and Hygiene (WASH), and Shelter & Settlement (S&S) Non-Food Item (NFI) activities in relation to the goal of reducing NFI vulnerability, improving food consumption, and reducing negative coping mechanisms. The results of this evaluation will be used to learn about the effectiveness of the activities undertaken during the “Emergency Response and Economic Recovery for Eastern DRC” project in order to improve humanitarian assistance to vulnerable populations in future interventions.

Background Information

Activity Information

Activity Name: Emergency Response and Economic Recovery for Eastern DRC

Implementer(s): Samaritan’s Purse

Award Number: 720BHA23CA00017

Period of Performance: April 10, 2023 to April 9, 2025

Active Geographic Region: Ituri, Tshopo, Haut-Uélé, and North Kivu Provinces

Background Information of Samaritan’s Purse and DRC Context

Samaritan’s Purse is a nondenominational evangelical Christian organization providing spiritual and physical aid to hurting people around the world. As a trusted Christian organization for more than 50 years, Samaritan’s Purse has helped meet the needs of people who are victims of war, poverty, natural disasters, disease, and famine with the purpose of sharing God’s love through His Son, Jesus Christ.

The Democratic Republic of the Congo (DRC) remains the most food insecure country in the world, according to the Integrated Food Security Phase Classification (IPC).[1] According to the IPC global dashboard, 15,799,000 people in DRC are in Phase 3 and above of the IPC scale (above the level of food crisis). An additional 25,101,000 Congolese are in Phase 2 “stressed” food insecurity. A primary reason for the food insecurity in DRC is violence and displacement, particularly in the eastern provinces of the country. Internally displaced Congolese, host communities, and returnees remain vulnerable as they often flee their home villages with few of their possessions and loss of their land, livestock, houses, and livelihoods.

Description of Activity

Samaritan’s Purse has been implementing an emergency response humanitarian program in eastern DRC with funding from the United States Agency for International Development (USAID) through Food for Peace (FFP) and the Office of US Foreign Disaster Assistance (OFDA) from 2014 to March 18, 2021, with a coordination office in Bunia. Commencing on April 2, 2021, the Emergency Response and Economic Recovery for Eastern DRC project received funding from BHA. Subsequently, the current iteration of this project, also funded by BHA, was initiated on April 10, 2023.

The program’s current iteration, which is the focus of this evaluation, began on April 10, 2023, and is anticipated to end on April 9, 2025. This eighth iteration is funded by the USAID Bureau for Humanitarian Assistance (BHA). The program’s overall goal is to assist 348,000 displaced people (58,000 households [HHs]), needy host families, and returnees in North Kivu, Ituri, Haut-Uélé and Tshopo Provinces.

Theory of Change (ToC):

The underlying premise of the program’s ToC is that, provided the target populations receive the emergency food, WASH, and S&S NFIs as planned, their Food Consumption Scores (FCSs) will rise, their hygiene will improve, their NFI vulnerability will decrease, and their negative coping mechanisms will decrease. Internally displaced persons (IDPs) and host communities will gradually become more socially cohesive as a result of this. On the other hand, benefiting HHs will learn about improved agricultural practices and, in turn, apply significant newly acquired technologies relevant for diversified crop production by giving vulnerable, conflict-affected, and agriculturally dependent HHs the necessary seeds, tools, and agricultural information to support the application of new technologies for efficient crop production. The program also envisions that the money injected through market-based voucher fairs, or unconditional cash transfers (UCTs), will stimulate local economic activities and improve the lives of a great number of people, including non-direct project beneficiaries, if targeted HHs use the money received in their local markets, as local traders also supply necessary food and NFIs.

Program Description

The program has four crosscutting sub-sectors that help to balance the provision of emergency needs with the contribution of commodity and business facilitation to economic recovery. These sectors include S&S, WASH, Agriculture, and Food Assistance. When targeted HHs are both NFI vulnerable and food insecure, and no other actor is providing food assistance, the program is expected to deliver 58,000 S&S NFI and shelter kits, as well as food and WASH assistance. In circumstances when targeted HHs are food secure or receive food aid from other implementing partners, the kits may be provided as stand-alone support. Depending on market conditions, security, and the presence of a financial service provider (FSP) to deliver cash to beneficiaries, the program expects around 45% of the total S&S NFI and shelter material kits to reach beneficiaries through voucher fairs, 45% of assistance to be delivered through UCTs, and 10% of assistance to be provided through direct distribution.

Shelter & Settlements

The objective of this sector is to meet emergency needs of 40,000 HHs affected by conflict by facilitating them to access S&S NFIs and shelter items through direct kit distributions, fairs or cash transfer modalities. Through this sector, the program plans to provide a package that includes kitchen items (pots, dishes, and utensils), standard items like soap, clothes and mats, and a shelter materials kit. Three sets of long-lasting insecticide-treated nets (LLINs) are also expected to be provided to each HH in the NFI package as a supplementary delivery alongside the voucher fairs or cash transfers.

Water, Sanitation and Hygiene

To alleviate the suffering and promote the dignity of conflict-affected people, the program aims to distribute WASH NFI kits alongside S&S NFIs and shelter materials (tarps and ropes). A hygiene kit (soap, menstrual hygiene materials, a plastic bucket, and women’s underwear) as well as water storage and transport items are to be included in the WASH NFI kits.

Agriculture

To strengthen the agriculture-based livelihoods and food security of the disaster-affected population through increased and diversified food production, the program intends to provide agricultural inputs and trainings to at least 8,000 returnees and host community HHs to help them supplement their short-term emergency food production and diet diversification. Specifically, the sector aims to provide vegetable and cereal seeds, farming equipment, and important agricultural knowledge, such as seed use sensitization messages, to ensure that beneficiaries are well prepared to get the most out of the inputs given. This is intended to improve the coping ability and resilience of the most vulnerable HHs in host communities while also allowing them to grow their own food in the short term.

Food Assistance

In order to provide emergency food assistance to save lives and improve food security for conflict-affected HHs, the program is expected to adopt a Multisectoral approach which will combine food and NFI interventions. Accordingly, the program aims to provide 36 kg of maize flour, 11 kg of ordinary beans, 3 liters of vegetable oil and 0.5 kg of iodized table salt per month for three successive months to each of the 58,000 HHs. The provision of a half-month food ration is accepted by the Food Security Cluster in eastern DRC, where the large majority of IDPs live with host families who serve as a coping mechanism for displacement. It is thus assumed that the IDPs will be able to meet their remaining half-month food ration requirements through income from work performed for the host families and/or through other positive local community coping strategies.

Evaluation Type

Samaritan’s Purse will conduct a performance evaluation that will employ a variety of methodologies, including internal baseline and endline assessment dataset comparison, desk-based literature review, focus group discussions (FGDs) with beneficiaries, and interviews with key stakeholders, including beneficiaries, community leaders and members, civil society representatives, and Samaritan’s Purse Emergency Response Program Managers, to explore issues identified in the evaluation’s datasets from internal assessments (e.g., effects on local markets, management, and suggestions for improvement, etc.). The data collection tools for collection of key data should be similar to the tools used during routine monitoring (to the extent possible) so that results may be comparable.

Evaluation Criteria & Questions

The evaluation will use the following Organization for Economic Co-operation and Development’s Development Assistance Committee (OECD-DAC) criteria as a framework for achieving the objectives:

• Effectiveness
• Relevance

Effectiveness

1. Were the interventions an effective way to meet 50% of the food and NFI needs of conflict-affected communities in eastern DRC for three months?
2. Did the food and NFI programming activities meet the objectives laid out in the proposal?
3. Was the Samaritan’s Purse project monitoring and evaluation (M&E) methodology for project outcome indicators appropriate and did it provide an accurate representation of project success? The criteria to be considered to evaluate the project’s agricultural component include the increased availability of food, diet diversification, improved ability to cope with food insecurity, and market strengthening.

Relevance

1. How well did the project prioritize the needs of women, girls, the elderly and people living with disabilities while responding to the specific food and NFI needs?
2. What were the food and NFI needs that were most relevant to beneficiaries including the marginalized?
3. How well suited are the different modalities of implementation, including commodity vouchers, direct distributions, and cash transfers, in the operational context for the Food Assistance, WASH, S&S NFI, and Agriculture sectors? How do the results of these modalities in the related sectors vary among the targeted population in terms of their relevance in the local contexts in which they were used?

Evaluation Methods & Limitations

The final evaluation of the project is based on document analysis and interviews with key persons. It is an external evaluation, carried out by external consultants hired for this specific purpose. The quality of the evaluation ‘product’ will depend very much on the methods used to collect and analyze data. The consultant will consider comments received and will reflect them, as appropriate, without compromising his/her independence and impartiality. In conducting the evaluation, the evaluator needs to consider relevant international standards, including the OECD-DAC criteria as detailed in the “Evaluation Criteria and Questions” above.

With respect to the evaluation model, given the specificity of the subject, both quantitative and qualitative models are recommended in order to cover all aspects of the program and shall be included in detail in the evaluation proposal (inception report). Analysis of the collected information will be used for a determination of the project’s effectiveness and relevance by applying the principles of the qualitative models. All evaluation findings need to be at least crosschecked and triangulated through various sources and methods in order to ensure their credibility and reliability.

Qualitative Methods

• Review of project reports and documentation: (e.g., project proposal, baseline reports, endline reports, post-distribution monitoring, and semi-annual and annual reports of achievements and beneficiary databases with personal identifiable information removed.
• Stakeholder interviews: these will involve structured and semi-structured interviews with key stakeholders including beneficiaries, community leaders and members, civil society representatives, and Samaritan’s Purse Emergency Response Program Managers (administered either face-to-face, by telephone, or by webcam).
• Focus group discussions with vendors and beneficiaries: a conscious effort should be made to ensure that separate FGDs are held for men and women.

Quantitative Methods

• An examination of the appropriateness and effectiveness of the food assistance intervention: this includes its complementarity with the Rapid Response and Population Movement Program (RRMP).
• A costing analysis: this is to determine which option (food, cash, or vouchers) offers the greatest benefit for funds invested.
• An analysis of the influence of the aid on the beneficiaries’ overall food consumption: this will be done by statistically comparing baseline and endline changes in the FCS, Reduced Coping Strategies Index (rCSI), Household Hunger Scale (HHS), and other food security indicators. The data from the aforementioned analysis will be disaggregated in accordance with the International Dietary Data Expansion (INDDEX) food security disaggregation standards.

Additionally, the chosen methodologies should be appropriate for rural areas as well as for illiterate, low-income populations, and should consider that sanitation and hygiene practices are often a culturally sensitive topic. The evaluator is responsible for ensuring informed interviewee consent prior to beginning any surveying activities. The evaluator will also ensure the privacy and confidentiality of beneficiaries’ interviews.

Indicators to be Analyzed during Performance Evaluation

• S05-Number and percent of individuals reporting satisfaction with the quality of the NFIs received
• S06-Number and percent of households meeting emergency NFI needs of identified settlement(s) through use of cash/vouchers
• C01-Reduced NFI Vulnerability Score (NFI Cluster Indicator)
• FS01-Percent of households with poor, borderline, and acceptable Food Consumption Score (FCS)
• FS02-Mean and median Reduced Coping Strategies Index (rCSI)
• FS03-Percent of households with moderate and severe Household Hunger Scale (HHS) scores

Limitations

Before entering the field and throughout staff deployment, the contractor should consider the risks related to insecurity, bad weather, bad roads, and seasonal disease outbreaks. The contractor should also have a response plan that includes adhering to security and safety standards, protocols, and procedures, including briefings on road and security conditions, routes to be driven, and points of contact. This is in addition to using tablets to collect data, collaborating with local people to gather data, and receiving security briefings before going out into the field.

Sampling

The main sampling strategy will be systematic sampling, in which HH representatives will be selected at regular intervals from the sampling frame—in this case, the beneficiaries served at each site. The evaluators will be required to visit various sites in at least three of the four provinces in which the program is implemented, namely Ituri, Tshopo, Haut-Uélé and North Kivu Provinces. In addition, the consultant is expected to use a multistage cluster sampling technique, with each cluster member being chosen at random from the list of program beneficiaries. The provinces, health zones, and communities will be the clusters in this scenario. While the number of stakeholders interviewed will be informed by their availability, it is planned to be representative of at least three of the provinces where the program works. The stakeholders to be interviewed will include beneficiaries, community leaders and members, civil society representatives, and Samaritan’s Purse Emergency Response Program Managers. The performance of this iteration is expected to be measured through the established outcome and output indicators, as defined in the program M&E plan. While the program’s objectives and outcome indicators are the same for all transfer modalities, it is strongly recommended that sampling be designed in a way that permits the grantor and the organization to compare the performance of different assistance modalities in terms of outcomes. In other words, the sample should be sufficiently large to allow for statistical validity, even if results for different transfer modalities are analyzed separately. Samaritan’s Purse will help to make appointments with the key informants ahead of the interviews and FGDs. Where necessary, and in consideration of local factors such as security and proximity, Samaritan’s Purse may recommend purposive sampling in the selection of the key informants.

Evaluation Timelines & Deliverables

Period of Performance for Evaluation

The anticipated period of performance for the evaluation will be March to April 2025. The evaluation will be undertaken over a period of 50 workdays, starting with the desk review of project documents and preparation of the questionnaires and ending with the production and submission of the final evaluation report. In the proposal defining the activities, output, and timeframe, the assessment consultant will give a matrix for the evaluation process demonstrating the duration and breakdown of activities. It is important to note that final intervention activities will be ongoing while the evaluation takes place.

Timeline of Evaluation Activities

The evaluation activities will run for an estimated 50 days and will evaluate the entire life of the project up until the period of evaluation. The proposed evaluation phases are the following:

Phase 1: Inception Report: This will be required within eight workdays of the contract award; the consultant must submit a detailed inception report that shall detail the evaluation design and operational work plan, which must include the proposed data collection and analysis methods to address the key questions of the evaluation. The inception report shall also include questionnaires and interview protocols and must not exceed 15 pages.

Phase 2: Evaluation Report (1st Draft): The consultant must submit a draft evaluation report and a PowerPoint version to the organization for preliminary comments prior to final mission debriefing. This will facilitate preparation of a final draft report that will be submitted to the organization upon the Evaluation Team’s departure.

Phase 3: Debriefing: After acceptance of the inception report, and immediately at the close of fieldwork before the consulting team departs, the team must present the major findings of the evaluation to the organization through a PowerPoint presentation. The debriefing must include a discussion of findings, conclusions, and recommendations.

Phase 4: Evaluation Report (Interim, 2nd Draft): The consultant must submit a draft report of the findings, conclusions and recommendations to Samaritan’s Purse, including revisions based on Samaritan’s Purse’s comments from the debriefing. The written report must address the evaluation questions, clearly describing findings, conclusions, and recommendations. Samaritan’s Purse will provide comments on the draft report within two weeks of submission.

Phase 5: Evaluation Report (Finalized Copy): After receiving Samaritan’s Purse’s comments on the interim evaluation report, the final evaluation report must be submitted to the organization. The final report must incorporate the team’s responses to Samaritan’s Purse’s comments and suggestions. The format shall include an executive summary (highlighting key lessons learned), table of contents, list of acronyms, evaluation design and methodology, findings, conclusions and recommendations, and lessons learned. The report shall be submitted in English in electronic copy.

Phase 6: Various Datasets: Following the last review, the consultant will forward the final report together with all of the datasets from various data collections that were part of the evaluation. These will comprise the FGDs, key informant interviews (KIIs) and desk review notes, in addition to the HH surveys.

Activity Expected Output
a) Inception report (March 10) – The evaluation design and operational work plan are provided.
b) Draft evaluation report (March 24)– Draft shared with Samaritan’s Purse for feedback.
c) Validation workshop (debriefing)(March 31)– Discussion and agreement of results and way forward.
d) Interim evaluation report(April 7) – Report integrating Samaritan’s Purse’s comments and feedback is submitted.
e) Final report(April 19)– Report with integrated Samaritan’s Purse comments on interim report is submitted.

Evaluation Findings Dissemination

The preliminary evaluation results will be shared with the implementation team, beneficiary representatives, local leaders, and representatives from the DRC sector clusters (WASH, Health, and Food Security) during a dissemination workshop that the program will hold. Finally, in order to strengthen his/her conclusions, the evaluator will include the feedback he/she obtained during this presentation in the evaluation report.

Evaluation Team Composition

Consultant Team Profile

QUALIFICATIONS AND EXPERIENCE

The consulting team should have the following skills, knowledge and experiences:

• A master’s degree in development studies, social sciences or economics, or a bachelor’s degree with relevant experience in place of a master’s degree.
• Working experience and understanding of humanitarian response operations and principles.
• At least 10 years of humanitarian programming experience, preferably in conflict settings, with a thorough understanding of cash, direct, and voucher assistance modalities in sectors such as Food Assistance and NFIs (S&S and WASH).
• Experience in conducting evaluations (USAID desirable), ideally leading an evaluation team, and experience in designing evaluation methodologies/tools, data analysis, etc.
• Proven ability and comprehensive understanding of quantitative and qualitative research methodologies and analysis.
• Experience working in the humanitarian or development sector in Sub-Saharan Africa.
• Excellent research skills, including the ability to collect, collate and analyze large amounts of qualitative data and identify critical aspects to succinctly communicate complex subject matter (in written and oral form) to make it accessible to wider audiences.
• Capacity to work collaboratively with multiple stakeholders.
• Strong analytical, presentation, and writing skills in English and French (recommended); knowledge of Swahili or any of the other local languages is a plus.

Administration Arrangements

The awarded bidder will report to the USAIDizi Monitoring and Evaluation Manager and will be expected to work closely with the Samaritan’s Purse Program Development Unit.

Tagged as: Democratic Republic of the Congo, Samaritan's Purse