Project Final Evaluation Support for Firefighting and Civil Security Crisis Response in Lebanon Civil Protection

The final evaluation aims to assess the relevance, efficiency, effectiveness, sustainability, and contribution to impact of the intervention in the short to medium term, based on the project’s implementation period.

This evaluation is an opportunity for Expertise France to obtain an external perspective on the activities implemented under the funding of this project, and for the project holder to take the time to reflect and to build a strategic reflection around the activities conducted and the results achieved.

This evaluation is a milestone for the project to ensure its accountability to the CDCS, the implementing agency (Expertise France), and the project beneficiaries, namely the Lebanese Ministry of the Interior and Municipalities, the Beirut Municipality Fire Brigade and the Lebanese General Directorate of Civil Defense.

In addition to the core evaluation criteria, the evaluation will also include a capitalization component, aimed at documenting and systematizing lessons learned good practices, and challenges. This will help generate transferable knowledge that can inform future programming, replication, and potential scaling up, both within the organization and with partners. These elements will be crosscutting and applied to all project outcomes and specific objectives.

Overall objectives and expectations of the assignment

The main objective of this evaluation is to:

- Capture project's achievements, challenges, and best practices to guide the exit strategy and inform future similar projects

-Evaluate progress against the intervention’s logical framework and offer recommendations for enhancement

-Assess the overall accountability toward the projects’ stakeholders.

-Identify key lessons learned, challenges faced, and the project's adaptability in responding to changing circumstances.

As per donor’s guidelines and request, this evaluation will serve as a tool to both evaluate and capitalize on the project’s good practices but also to provide recommendations. Those recommendations are of several levels: the governance of the project itself, the project's ambitions in relation to the reality of operations, the evaluated impacts and the perception. In parallel, recommendations should be made on how to sustain the actions undertaken and enable partners to continue their efforts in the medium and long term.

The evaluator should provide evidence to explain the analyses, cause and effect linkages and attempt to identify any factors that are enhancing or hindering progress. Their work must foster accountability, decision-making, and learning.

CRITERIA AND QUESTIONS

The evaluation will use the criteria defined by the Organization for Economic Co-operation and Development (OECD) Development Assistance Committee (DAC): relevance, effectiveness, efficiency, impact and sustainability.

The evaluation questions are detailed below. They will be reviewed by the evaluator during the evaluation start-up phase, in order to suggest a final version in the inception report approved by the steering committee.

EVALUATION METHODOLOGY AND ORGANIZATION OF THE MISSION

The final evaluation methodology will be developed by the firm/consultant. The methodology should adopt a mixed-method approach comprising both qualitative and quantitative data including innovative approaches and methods. In addition, the endline evaluation should develop a sound sampling methodology as needed. The data collection should include the use of a number of approaches to gain a deeper understanding of the outcomes of the project, including:

-Desk Review of background documents (project document, logframe indicators, project monitoring data, progress reports, etc.).

-Quantitative and qualitative approaches with beneficiaries and stakeholders groups (approx.. 10 stakeholders). These approaches could include surveys, key informant interviews (e.g., with EF project staff members, partners’ key staff, local authorities, etc.) to gather substantial anecdotal evidence on the effectiveness, efficiency, relevance and impact of the project activities implementation and delivery, or focus group discussions (e.g., project beneficiaries, others, etc.). The list of stakeholders (approx. 10 people) will be handed to the selected consultant.

Other participatory approaches could be included as recommended by the consultant, ensure covering the project activities implemented since 2024.

Once the consultant has presented their observations, they then put together their findings and appraisal of the project in relation to each evaluation questions and present their general conclusions to provide an overall assessment of the evaluated project. The conclusions must be ranked in order of importance and in order of reliability.

The consultant will identify strategic and/or operational lessons and/or recommendations. These should be linked to the findings, grouped and prioritized (cf. The Evaluation Guide).

Recommendations should also be made where gender issues have arisen during project delivery, particularly if the project has negative impact on gender equality.

Expected deliverables

An inception report, which will serve as an agreement between parties on how the evaluation will be conducted. Items to address:

-Understanding of the issues and questions raised in the ToR

-Data sources; how to assess the questions in the ToR

-Evaluation methodology, including sampling methodologies taking into consideration the the number of the targeted stakeholders.

-Schedule of final evaluation activities (timeline) Appropriate validated draft data collection tools (e.g., methodological guidelines, group interview questions)

A max 20-25 page draft and final evaluation report (in MS Office and PDF for final), excluding annexes and in English will develop the framework of the evaluation in more depth based on the terms of reference, the documents collected and the reconstructed intervention logic. Specifically, this will involve: (i) outlining the key questions for the evaluation to focus on; (ii) outlining the stages of reasoning that will make it 6 6 1 6 possible to answer the questions (judgment criteria); (iii) specifying the indicators to be used to answer the questions and the corresponding sources of information (documentation, interviews, focus groups, surveys, etc.). The final report should cover the below sections: - - - - - - - - Executive Summary in bullet points (max. 2 pages) Introduction Methodology, including sampling and limitations Schedule of interviews and of the general data collection method implemented Analysis and findings of the evaluation. The analysis should cover the key log-frame indicators and any other criteria agreed between EF and the selected company/consultant. Address concerns, lessons learned and comments from EF. Stories of change and quotes from respondents. Recommendations for project sustainability and next steps after project closure.

A 3-page capitalization report, using the template attached to this document.

Organizing field assignments (optional)

The consultant will be provided with the list of organizations and project stakeholders to meet with during the field visit phase. Those stakeholders will be aware of the consultant presence and mission. The consultant should therefore be able to organize its week accordingly to meet all stakeholders. The work plan and repartition of role will be discussed further and validated during the kick-off meeting.

The maximum amount allocated for this study is 6 000 euros, all costs included. No additional costs will be taken into account by EF. Therefore, the consultant will preferably be based in Lebanon in order to collect face-to-face data without travel expenses.

The work will mainly be done in French (interviews and report), Arabic (interviews) and English (interviews).