BBR final evaluation Giro555

Project Title

Beirut Explosion Response Plan: Save lives, reduce suffering, and enhance resilience of vulnerable groups in Lebanon who have been impacted by the Beirut Port Blast through a targeted, localized response that will address the root causes of crisis while saving lives in the short term

Donors

Oxfam Novib (funded through Giro555), OGB Appeal, DANIDA, Beirut Pooled Appeal (Oxfam Italy, Oxfam IBIS, Oxfam Germany, Oxfam Belgium, Oxfam Aotearoa (was NZ), Oxfam Hong Kong, Oxfam America, and Oxfam Intermon), Quebec Government (MRIF), and Oxfam Australia

Project Location

Greater Beirut (areas affected by the Beirut Port blast)

Project Duration

Phase 1: Response Plan: August 2020 – February 2021

Phase 2: Recovery Plan: November 2020 – December 2021

Evaluation to be conducted in

March - April 2022 (approximately 35 days)

Oxfam in Lebanon

Oxfam has been working in Lebanon since 1993 on the provision of rapid emergency humanitarian assistance to vulnerable communities, as well as long-term development support to Civil Society Organizations (CSOs) and local authorities. Through partnerships with local organizations, Oxfam in Lebanon has responded to several conflicts in Lebanon, including the July War in 2006, and the clashes in Nahr el Bared camp (NBC) in 2007, the Syria Crisis since 2013, and most recently the Beirut Port Blast.  

Oxfam in Lebanon’s work is guided by a 5-year vision, which aims “to protect and empower marginalized women and men (Lebanese, Palestinian and Syrian populations including refugees) to enjoy their basic rights, and to enable them to live in dignity within a more equitable society.” Such a vision heavily relies on rights-based, citizen-based, intersectionality, and partnership approaches in the development of the programs and projects. To implement this vision, Oxfam’s work focuses on two key programmatic work streams, namely Humanitarian - Saving Lives Now and in the Future, and Governance and Economic Justice with policy and gender justice as cross cutting themes.

Project Overview

Oxfam seeks an external consultant for a final evaluation of our project Beirut Explosion Response Plan: Save lives, reduce suffering and enhance resilience of vulnerable groups in Lebanon who have been impacted by the Beirut Port Blast through a targeted, localized response that will address the root causes of crisis while saving lives in the short term, which is funded by Oxfam Novib, OGB Appeal, DANIDA, Quebec government (MRIF), Oxfam Australia and the Beirut Pooled Appeal fund.

On August 4, 2020 at approximately 18h00 (Beirut time), a warehouse at the Beirut Port containing 2,750 tons of improperly stored ammonium nitrate exploded. After an initial explosion, a subsequent blast caused widespread damage, with reports of destruction more than 20 kilometers from the port area. The blast has had a devastating impact on the city, and its consequences have a long-lasting impact on a city already suffering from an economic, political, and health crisis.

On August 5, 2020, the United Nations established an Emergency Operations Centre (EOC), led by OCHA and experts from the UN Disaster Assessment and Coordination (UNDAC) team, conducted a rapid analysis of the situation on the ground to help coordinate emergency activities in response to the Beirut Port explosion. Several other organizations conducted different assessment exercises. To compile all the information, the Assessment and Analysis Cell (AA) within the EOC started publishing situational analysis that includes a meta-analysis of all the assessments that were being done. To validate the secondary data as presented in the assessments, Oxfam carried out a series of consultations with its local partners. The triangulation of information highlighted the following emergency needs: Shelter, Livelihoods, Food Security, Health, Protection and Water Sanitation and Hygiene (WASH).

The response strategy was guided by Oxfam's overarching humanitarian approach, which identifies localization, community engagement, social accountability, and feminist approaches as the four pillars to inform our humanitarian work. The country team has a strong foundation of investment in local humanitarian leadership and works to sustain the partnerships developed with active local CSOs. Therefore, Oxfam’s Beirut Explosion response strategy was heavily focused on supporting local partners, existing and new, in their response.

Guided by this strategy and responding to the identified needs, Oxfam in Lebanon implemented a targeted response, one that heavily relied on a complementary and reinforcing relationship with selective partners. The response targeted the most vulnerable groups, both within the city and those living outside of Beirut who were impacted by the blast. Oxfam’s response plan was divided into two phases:

• An emergency response phase (first 3 to 7 months): address urgent, life-saving needs
• A recovery phase (following 13 months): seek to support the most affected groups to recover, build back better, and strengthen their adaptive and absorptive resilience capacity 

The Response

The emergency response phase focused on supporting the provision of direct emergency services, light shelter repair, multi-purpose cash, temporary cash assistance and early business support.

Overall Objective: Save lives, reduce suffering, and enhance resilience of vulnerable groups in Lebanon who have been impacted by the Beirut Port Blast through a targeted, localized response that will address the root causes of crisis while saving lives in the short term. 

Specific Objective 1: Delivery of immediate, quality, safe and inclusive, life-saving emergency assistance to those most impacted community members living in affected areas, including areas outside of Beirut.

Result 1: Access to emergency food assistance improved

Result 2: Appropriate medical assistance provided to the affected communities in Beirut and Mount Lebanon

Result 3: Damaged houses affected by the blast in Beirut and Mount Lebanon are rehabilitated and repaired

Result 4: The affected communities have access to improved GBV and mental health services

Result 5: The affected communities have improved access to legal support and services

Result 6: The affected communities have received cash assistance

Result 7: The rights of the affected population are advocated for at a local and national level

Result 8: Both partners’ and Oxfam’s referral systems/Complaints, Referrals and Feedback mechanisms established are strengthened

Specific Objective 2: Provide coordinated and integrated economic support to affected micro, small, and medium enterprises (MSMEs), and employees, focusing on the most vulnerable groups, within Beirut and across the country 

Result 1: The affected MSMEs/SEs are provided with material assistance

Result 2: The affected MSMEs/SEs are receiving business development support/non-financial support

Specific Objective 3: Contribute to the advancement of Oxfam’s Local Humanitarian Leadership (LHL) agenda through a partner-centered response that prioritizes the rights and dignity of affected populations

Result 1: Local actors partnered with Oxfam in the response have greater voice, visibility, access and presence in decision making spaces

Result 2: Local actors partnered with Oxfam in the response have greater programmatic and organizational capacity as a result of the partnership

Partners

Taking an inclusive and intersectional approach to the response, Oxfam supported local partners who support sub-populations that are already subject to different types of vulnerability such as elderly residents, people with disability, women, refugees, and the LGBTQI+ community. Following consultation and close discussions, Oxfam in Lebanon identified the following partners with whom to jointly implement the response. Together, Oxfam and partners reached 4,797 direct beneficiaries (individuals/households) and 76 MSMEs with the following services:

Partner

Target Groups

Activities

Lebanese Centre for Human Rights (CLDH)

Families of the victims of the explosion and survivors in affected areas

• Emergency cash assistance- ECA
• Physiotherapy
• Psychosocial support
• Legal consultation and representation
• Medical consultations

KAFA (enough) Violence & Exploitation

Women and girls 

Lebanese and other nationalities 

• Emergency cash assistance- ECA
• Temporary cash assistance- TCA
• Shelter repairs
• Distribution of food parcels

ABAAD

Affected communities in Beirut and its suburbs

• Emergency cash assistance
• SGBV case management
• Mental health and psychosocial support

Lebanese Union for People with Physical Disabilities (LUPD)

Elderly and people with physical disabilities

A total of 100 unique individuals received the following:

• Distribution of assistive devices
• Physiotherapy
• Occupational therapy
• Psychosocial support

• Emergency cash assistance

Helem

Members of the LGBTQI+ community

• Emergency relief funds
• SGBV Case management

• Psychotherapy
• Individual counseling
• Psychiatric referrals
• Training on LGBTQI+ mainstreaming to NGOs

Oxfam and Basmeh & Zeitooneh (B&Z)

Vulnerable families from all nationalities who meet certain criteria

• Temporary cash assistance- TCA

Samidoun

Neighborhoods heavily damaged by the Beirut Port blast

• Shelter rehabilitation

Lebanese Observatory for Workers and Employees Rights (LOWER)

Individuals from all nationalities who used to work in the Port and its surrounding and lost their job

• Rapid labor assessment of businesses and employees affected by the blast in the port and surrounding areas

Legal Agenda

Individuals and families of individuals in damaged areas

• Development of a victims’ collective/association (50 individuals)
• Distribution of a legal guide about the trial before the court of justice, reconstruction after the blast, compensation for damages, and victims’ associations 

Mada Association

MSME owners and employees

• Support to SMEs in working capital (rehabilitation and/or equipment and/or supplies)
• Business coaching to SMEs

Nabad for Development

Vulnerable families and MSMEs that were affected by the Beirut Port blast

• Distribution of food parcels and hygiene kits
• Distribution of tarpaulins
• Rehabilitation and provision of equipment and material to MSMEs

Purpose of Final External Evaluation

The goal of this external final evaluation is to assess the overall performance of the project against OECD DAC evaluation criteria and other international quality standards. It should provide an objective analysis of the project’s performance in terms of partnership approach, relevance, coherence, effectiveness, efficiency and sustainability.  The evaluation should present results, conclusions, lessons learned, and recommendations that can guide Oxfam in future emergency responses.

External Evaluation Objectives:

• To gauge the relevance and effectiveness of the different types of assistance provided by Oxfam and partners in response to the Beirut Port blast and whether the support provided was consistent with the emergent needs and priorities
• To evaluate partners’ satisfaction in the partnership with Oxfam and assess if/how the response design supported Oxfam’s local humanitarian leadership agenda
• To assess the design, planning, delivery, and management of the project by Oxfam and its partners in accordance with the OECD DAC criteria (Relevance, Effectiveness, Coherence, Efficiency, Impact, Sustainability), Oxfam Programme Standards, Core Humanitarian Standards (including the IASC Gender with Age Marker), and in relation to targets set in the project MEAL framework
• To identify key internal and external factors that contributed to, affected, or impeded the achievements of the project, and how Oxfam and partners managed these factors
• To draw lessons learned from the project and make recommendations that will help inform the design of Oxfam’s future humanitarian projects

Evaluation Criteria and Key Questions

The following are proposed evaluation questions, based on OECD/DAC criteria for development evaluation:

Relevance:

• To what extent did the project design and activities respond to the needs, expectations, and priorities of individuals and enterprises affected by the Beirut Port blast, especially with the rapidly changing economic environment?
• Were the needs of specific vulnerable groups (people with disabilities, LGBTQI+ community, marginalized women, refugees etc.) adequately considered in the project design?
• How gender-intentional was the program design and implementation, and how well or not did ongoing management and implementation adhere to Oxfam’s Transformative Leadership for Women’s Rights approach?
• Was the response built around local capacities of partners and affected communities? Has local ownership been established?

Effectiveness:

• To what extent were the project’s objectives and expected outcomes fulfilled?
• Did the economic support to enterprises help them to overcome the impact of the Beirut Port blast and maintain their operations?
• Did the life-saving emergency response help alleviate the suffering of individuals and support them to recover from the Beirut Port blast?
• Based on beneficiary baseline/endline data, to what extent were some interventions/activities more effective than others?
• What were the constraints that limited the effectiveness of the support? Were these mitigated properly?
  • How well were the partnerships working? Was the visibility, presence or capacity of the Beirut Response partners enhanced due to the partnership?

Coherence:

• What were the synergies or misalignment with interventions implemented by other organizations as a response to the Beirut Port blast?
• How well or not did the Beirut emergency response coordinate with the major area and national coordination mechanisms?
• How was the coordination between Oxfam and partners? Also, between the Business Development Support (BDS) coaches and the enterprises?

Efficiency:

• How well were resources used, especially given implementation changes driven by COVID-19 and fluctuating exchange rate, and subsequent strains on logistics, resources, and movement?
• Was the project able to adapt to any changing conditions?
• How did governmental decisions (such as lockdown measures) affect service provision and the efficiency of the project?

Impact:

• What are the long-term effects, both positive and negative, that can be reasonably attributed either to the intervention? To what extent do different sub-populations experience those long-term effects in different ways?
• To what extent have the project outcomes contributed to saving lives, reducing suffering and enhancing resilience of vulnerable groups in Lebanon? Did the project reduce the target populations’ use of negative coping mechanisms? Has the project contributed to the improvement of the conditions of the target populations?

Sustainability:

• Were the necessary measures taken to build on local capacities of partners?
• What concrete steps were/are being taken to enhance the sustainability of the project’s achievements? What is the likelihood of benefits to continue over the medium and long-term?
• To what extent are supported enterprises able to maintain their operations in light of the changing context and the shift in priorities/needs?

Target Audiences

The primary audiences for and users of the evaluation findings will be Oxfam’s project team, management, technical advisors, partner staff, donors, and the wider humanitarian response community in Lebanon.

Methodology

The evaluation must be conducted in line with evaluation best practices, use quantitative and qualitative data, and draw upon both primary and secondary data collection techniques. The sample size should be adequate and representative of all target groups. The evaluator is expected to develop a detailed methodology that will be reviewed and approved by Oxfam. The evaluation methodology must make use of existing baseline and endline data, Situation Reports (sitreps), partner reports, donor reports, tracking sheets, Logical Framework, and other available monitoring and evaluation data that Oxfam’s MEAL team will provide.

The methodology requires that the consultant(s) have experience with participatory approaches to learning and inquiry, particularly in seeking the views of key stakeholders such as:

• Targeted individuals
• Targeted enterprises
• Management, project, and administrative staff of implementing partners KAFA, CLDH, LUPD, Helem, ABAAD, Mada, Nabad, LOWER, Legal Agenda, Samidoun and Basmeh & Zeitooneh, 
• Oxfam regional and country staff involved in the implementation of the project (e.g. Project Managers (PMs), Global Humanitarian Team (GHT) staff, Regional Humanitarian Coordinator, and field team)
• Oxfam Novib staff

Timeframe, Key Activities, and Expected Level of Effort

Timeframe

The evaluation is to be carried out in March and April 2022, with the final report submitted to Oxfam as per the timeline below. The exact dates will be confirmed with the selected consultant(s).

Total expected level of effort: 35 working days

• Inception Phase:

• Desk review of project documents, including but not limited to the original project proposal, interim and final narrative reports, partner reports and existing internal and external research reports (6 days, all documents will be shared by Oxfam via the online platform Box)
• Preparation of an Inception Report, including evaluation questions and a detailed evaluation plan (to be endorsed by Oxfam) (4 days)

• Primary data collection (12 days, a mix of in-person and remote)
• Data analysis and preparation of draft evaluation report (7 days)
• Validation workshop (1 day)
• Finalize the final evaluation report and send it to Oxfam (5 days). Oxfam will then prepare a management response to be annexed to the evaluation report

Consultant Qualifications

• Demonstrated experience (minimum 5 years) working with NGOs in emergency humanitarian responses
• Demonstrated experience in conducting humanitarian project evaluations
• Knowledgeable about the social, political, economic, and cultural context in Lebanon
• Knowledgeable and experienced in research and quantitative and qualitative evaluation methods
• Knowledgeable/experienced with one or more of the technical areas of intervention (at least with evaluation of some of the following: Emergency Food Security and Vulnerable Livelihoods (EFSVL), health services (including mental health), cash assistance, MSMEs support, etc.)
• Understanding of local humanitarian leadership and (humanitarian) capacity development
• Excellent skills in research, data collection, and analysis
• Experience working in Lebanon
• Strong verbal communication skills in Arabic and English and facilitation skills
• Excellent report-writing and presentation skills in English and Arabic

Expected Outputs

1. Develop a detailed inception report and evaluation plan (to be submitted after the document review but before the data collection - to be endorsed by Oxfam), outlining the proposed methodology including budget, timeline, and data collection tools
2. Draft evaluation report (English), with up to 3 rounds of comments/reviews before validation, to be presented to Oxfam and partners
3. Online validation workshop and slide deck (English and Arabic) to present and discuss draft findings with Oxfam and partners
4. Final report (English) incorporating Oxfam and partner comments.

Please note that Oxfam will review all outputs and provide comments for the consultant to consider and incorporate. Partner comments must also be taken into account before the final report is submitted and approved.

Instructions for interested consultants

Payment will be done in two instalments: 25% upon contract signature, and 75% upon Oxfam’s approval of the final evaluation report.

Costs to include in the offer: Consultants should include the following costs in their proposed budget: daily rate, cost of travel (in the case of an international consultant living outside Lebanon), per diems, communication costs, miscellaneous costs (e.g., visa costs, depending on nationality of consultant).

Note that payment will be made based on the budget in the offer (not based on actual expenses incurred by the consultant). No receipts will be requested from the consultant towards the end of the evaluation.

Codes of Behaviour

The evaluation process must adhere to Oxfam’s guidelines for the ethical conduct of evaluations and research in addition to the Non-Staff Code of Conduct. The consultant and team are expected to consider key ethical implications at every stage of the evaluation.

Sharing and Using Findings

Oxfam International’s Policy on Program Evaluation requires Confederation members to act on the commitment to transparency by making public the Executive Summary and a Management Response to all final evaluations.

Disclosure

Although free to discuss with the authorities on anything relevant to the assignment, under the terms of reference, the consultant is not authorized to make any commitments on behalf of Oxfam. All data collected as part of this consultancy belongs to Oxfam and public dissemination of the data and evaluation products can only be done with the written consent of Oxfam.