TERMS OF REFERENCE (TOR) Real-time Program Evaluation of Caafimaad+ Consortium in Somalia Full Time NEW

Title: Consultancy to undertake Real-time Program Evaluation of Caafimaad+ HIP 2024 Action.

Location: 23 districts in Banadir, Bay, Bakool, Gedo, Lower Shabelle, Lower Juba, Middle Shabelle, and Mudug regions

Tentative time of Assessment: 15 January 2025 – 30 June 2025

Key Working Contacts: Caafimaad Plus Consortium Management Unit

A. Background

Caafimaad+ Consortium, a Heath & Nutrition Consortium in Somalia initiated in 2020, implements integrated Health, Nutrition, WASH, and Protection services in twenty three (23) districts of 8 regions across five states in Somalia. The consortium comprises five INGOs namely Action Against Hunger (Lead), Concern Worldwide, International Medical Corps, SOS Children’s Village, and Trocaire, alongside 4 local NGOs Juba Foundation, Lifeline Gedo, SHACDO, and YouthLink. The overall objective of the program is “Contribute to the reduction of morbidity and mortality associated with acute malnutrition, natural disasters (drought/floods), displacements, and other shocks among vulnerable population in South and Central Somalia”. The key objectives are:

• Increase access to a package of basic health services at community, primary and secondary health levels, especially for women, children and vulnerable populations
• Increase access to free quality curative and preventive nutrition services at community, primary, and secondary levels
• Improving access to safe, sufficient water, sanitation facilities and waste management services at health facility level
• Women, girls, boys and men in target locations have improved access to safe, timely, lifesaving GBV services
• Vulnerability of the targeted communities to external shocks is reduced through agile life-saving interventions answering to immediate needs brought by rapid onset disasters

The consortium partners implement in 23 districts located in Banadir, Bay, Bakool, Gedo, Lower Shabelle, Lower Juba, Middle Shabelle, and Mudug regions of Somalia. The implementation locations are mostly characterized by high levels of humanitarian needs that have been caused by massive displacements from various shocks such as protracted and worsening drought, floods, conflict, loss of livelihood, and others. An evaluation is required to assess the program’s effectiveness, efficiency, and impact to inform future planning and decision-making.

B. Purpose of the Evaluation

The evaluation, which will be conducted in real-time over the stated period will aim to:

• Assess the program’s achievements against its objectives and intended outcomes. To measure interventions by Caafimaad Plus in terms of safety, accessibility, satisfaction, accountability and participatory aspects of the actions/activities/strategies
• Analyze the effectiveness and efficiency of program implementation. The consultant will evaluate the effectiveness of the program interventions, including beneficiary reach, program outputs, and key outcomes. It will also assess the effectiveness of program monitoring, including accountability to beneficiaries. Other areas of assessment will be the various implementation approaches at the facility and community level, integration of services, and engagement of the community and key stakeholders.
• Assess the level of integration between the health and nutrition sectors: within the consortium by utilizing the WHO analytical framework, which outlines six core components of health systems (leadership and governance, service delivery, health system financing, health workforce, medical products, vaccines and technologies, and health information systems). Systematizing the use of this analytical framework to evaluate integration efforts will allow the consortium to monitor, compare, and share progress and lessons learned effectively within its network.
• Identify lessons learned and best practices. The evaluation will identify learning opportunities and best practices. It will also identify gaps in implementation, the added value of strategies among the consortium partners, the effectiveness of sectoral integration, and community and facility linkages. Furthermore, it will look at the effectiveness of collaboration and synergy in coordination, complementarity, and localization.
• Provide actionable recommendations for program improvement or scale-up. Detailed recommendations will be provided for each gap identified, with narrative of how they will result in improvement.

C. Scope of Work

The consultant/team will:

• Review Documentation: Examine program plans, program reports, and program data. The evaluation will focus on the entire project funded by EU Humanitarian Aid, including all its technical areas. It is worth looking at more in-depth processes used in delivery of EPHS packages and overall beneficiary satisfaction on services offered. The evaluation will also take an in-depth look into the integration of activities in all supported sectors (Nutrition, Health, Protection and WASH) in the geographical locations of implementation. Secondary data review of routine monitoring data and other project records/documents. The evaluator will undertake a desk review of project materials, including the project documents and proposals, interim report, Indicator tracking table, dashboard and relevant materials from secondary sources (Annex IV).
• Develop an Evaluation Framework: Develop a methodology and indicators to guide the evaluation. Based on Action Against Hunger’s Evaluation Policy and Guidelines, the evaluator will assess the project against the OECD DAC criteria using mixed methods (qualitative and quantitative) approach.
• Develop data collection tools; A set of detailed data collection tools to proposed by the consultants and discussed and agreed upon by the ACF regional and country MEAL team, and the Caafimaad+ consortium Technical Working Group (TWG).
• Data Collection and Analysis: Conduct quantitative and qualitative data collection through surveys, interviews, focus groups, or other methods. The consultant will include an evaluation matrix as an annex to specify data collection will be done to answer the evaluation questions, pointing out the limitations to the methodology. Provide details of sample size and frame, and the choice of sites for field visits.
• Field Mission: The consultant will detail the types of people they plan to interview, the reason for interview, and the interview methodologies. These will include but are not limited to key project stakeholders such as the program management, program staff, consortium partners staff, MOH officials, Donor representatives, local authorities, and communities.

D. Assessment of Key Areas:

1. The consultant will assess these based on the following criteria:
   • Relevance: Alignment with National Guidelines, community needs and priorities.
   • Effectiveness: Achievement of program objectives.
   • Efficiency: Optimal use of resources including coordination and complementarity.
   • Impact: Measurable changes in health outcomes**.**
   • Sustainability: Potential for long-term benefits.

E. Reporting:

The consultants will produce an interim report which will be presented to the Consortium Management Unit (CMU) and key stakeholders. The Final Evaluation Report incorporating the comments from key stakeholders will thereafter be developed and presented to the CMU.

F. Other Considerations

• The geographical scope of the evaluation will cover the project sites in different regions of Somalia. However, simple random sampling of some consortium operating areas and communities will be done in to maximize on the time allocated for the evaluation.
• Gender concerns should be addressed in line with the Action Against Hunger Gender Policy throughout the evaluation process.
• All data should be disaggregated by sex and age and different needs of women, men, boys and girls should be considered throughout the evaluation process.
• Gender and decision making of both men and women and access and control of assets, resources and services in the EU Humanitarian Aid funded project should be considered.
• Capacity and participation of both men and women in the program will be evaluated especially on gender balance of staff.

G. Deliverables

The consultants will provide the following key deliverables:

• Inception Report: Detailing the evaluation plan, methodology, and tools.
• Interim/ Draft Evaluation Report: Summary of preliminary findings
• Stakeholder Presentation: Summary of key findings and recommendations for stakeholders.
• Final Evaluation Report: Comprehensive document that captures feedback/ comments from stakeholders and will include: an Executive summary, Methodology, Findings and analysis, Recommendations, and Case studies or success stories where applicable

H. Timeline

• The consultant will provide proposed timeline for completing the evaluation within the provided timeframe, clearly indicating the milestones such as: Contract Signing, Inception Report, Field Data Collection, Draft Report Submission, and Final Report Submission.

I. Budget

Provide a budget on a custom template detailing the cost of the consultancy by budgetary line items.

J. Desired Qualifications

Education (knowledge):

• Team Lead to have PhD or MSc in Public Health, Nutrition, Development, or Statistics related field
• More than 10 years of experience in research in humanitarian context
• Previous experience in Program Evaluation in Somalia or similar the context.
• Strong and high-level experience in building and maintaining partnerships and drive to achieve results for impact.
• Excellent writing skills demonstrated through previous evaluation reports
• Experience with presentation to Donors, Clusters, and other key stakeholders

Skills and Abilities:

• Strong English skills both oral and written. Strong oral skills in Somali language will be an added advantage
• Very good report-writing skills demonstrated in previous work
• Understanding of Somalia health system and Clusters is an advantage.
• Consultant who can access the community is highly desirable.

K. Conditions of Work

The consultant(s) will work in districts within the eight Caafimaad+ supported region of Banadir, Bay, Bakool, Gedo, Lower Juba, Lower Shabelle, Middle Shabelle, and Mudug regions.

L. Administrative responsibilities are assigned below:

Caafimaad Plus

• Airfare to and from Mission
• Entry visa to country of mission
• Domestic Accommodation
• Domestic transportation

The Consultant

• Health Insurance for him/herself
• Equipment for use (Laptop etc.)
• Food/per diem.

M. Submission Guidelines

Interested applicants who meet the above requirements should submit the following:

• CVs of team members detailing the relevant qualifications and experience.
• References: Telephone and email contacts of at least three previous clients who can validate technical expertise.
• A technical proposal outlining the approach, methodology, and timeline.
• Financial proposal for the entire consultancy (line-item budget)
• A detailed work plan indicating the activities for evaluation.
• Samples of reports for similar evaluations previously conducted.

Tagged as: Action Against Hunger USA, Somalia