Consultant - Final evaluation of the Mental Health Support After the Beirut Blast Project

Contents

1. Project summary. 3

2. Jesuit Refugee Service.. 3

3. Background.. 3

4. Project description.. 4

Strategy A: Health.. 4

Strategy B: Livelihoods. 5

Strategy C: Nutrition.. 6

Strategy D: Programme Management. 6

5. Research Purpose & Objectives. 7

6. Evaluation Target Audience.. 7

7. Methodology. 7

7.1 Process. 7

7.2 Geographic location.. 8

7.3 Qualitative data collection.. 8

7.4 Quantitative data.. 8

7.5 Evaluation criteria.. 9

8. Roles & Responsibility. 9

9. Timeline & Products. 10

10. Evaluation report format. 12

11. Budget. 12

12. Requirements. 12

13. Application submission.. 13

Project name

Mental Health Support After the Beirut Blast

Location

Beirut, Lebanon

Project goal

Syrian refugee and vulnerable Lebanese families in Burj Hammoud and Karantina are provided necessary services to respond to the immediate needs from the 04 August 2020 Beirut explosion and begin the process of recovery.

Project duration

15 September 2020 – 31 December 2021

Final project evaluation purpose

The purpose of the final project evaluation is to i) determine the extent to which the project goal and outcomes were achieved, highlighting accomplishments and under-achievements and the main factors attributing to them ii) identify and document lessons learned and best practices, and iii) provide action-oriented recommendations for enhancing future similar programming. 

Evaluation duration

1 February to 4 March 2022 (25 working days)

Final report date

4 March 2022

The Jesuit Refugee Service (JRS) is an international Catholic organisation with a mission to accompany, serve and advocate on behalf of refugees and other forcibly displaced persons, that they may heal, learn, and determine their own future.

JRS believes that our journeying with refugees is the most important way for us to express our solidarity with them and our concern for their wellbeing. In a world where refugees are more than ever in need of welcome, protection, and justice, and yet are increasingly rejected, demonised, and denied their fundamental human rights, JRS offers accompaniment to refugees as a sign of hope and a way towards healing.  In even the most desperate of situations, we remain with refugees to assure them that the world has not forgotten them, and that they are not alone.

JRS Lebanon started implementing multifaceted emergency response programs in Burj Hammoud in 2013. Due to the protracted displacement, JRS recognized the need to transition from an emergency response to a more development-focused approach with an eye towards durable solutions. Therefore, JRS established the Frans van der Lugt (FVDL) centre in 2014 to provide a range of services including emergency and basic assistance, psychosocial support, early childhood education and learning support, and adult education.

At least 178 people were killed and more than 6,000 wounded following two explosions that erupted in Lebanon’s capital on 4 August 2020. The second blast extended from Beirut’s industrial port into the city’s densely populated residential neighbourhoods and shopping districts. Dozens of nearby buildings collapsed or were severely damaged and an estimated 300,000 people were rendered homeless.  

The impact of the explosion on the needs of the affected population were compounded by the destruction of Beirut’s port and key supplies stored in the area, including Lebanon’s national wheat silo. Lebanon depends on imports to satisfy around 85% of its food needs. The country was already in a precarious food security situation, as inflation has seen the cost of goods increase exponentially in 2020 with the trend continuing into 2021, while importers have struggled to secure enough dollars to pay for imported items. 

Lebanon has been rocked by an economic crisis that has seen the value of the Lebanese Pound (LBP) decline by 85% at the end of July 2020, exacerbating vulnerabilities among both host and refugee communities. Between 2019 and May 2020, the poverty rate jumped from 28% to 55%, while extreme poverty rose from 8% to 23%. During the same period, the COVID-19 pandemic led to the imposition of lockdown measures that placed pressure on struggling households by restricting individual opportunities for income generation and negatively affecting the economy as a whole. On the morning of the blast, Lebanon’s main coronavirus hospital announced being on the verge of reaching full capacity in its critical care ward. In the days following the explosion, Lebanon repeatedly logged record daily highs of new coronavirus cases, with cases rocketing from 155 on 1 August to 605 on 19 August, necessitating a further lockdown in the country. This has placed significant pressure on a healthcare system that was struggling before the blast damaged major hospitals and clinics.

In the aftermath of the explosion, the Lebanese Cabinet declared a two week State of Emergency that was extended to 18 September 2020. The severity of the disaster, coupled with pre-existing crisis conditions, necessitated rapid life-saving emergency relief activities and more long-term mental health and psychosocial support to assist those households affected by the disaster who lacked the means to absorb the latest shock.

In response to the Beirut Blast, JRS implemented a one-year project focused on health, livelihoods, nutrition and programme management. Project participants included individuals and families engaged in JRS interventions prior to the Beirut Blast as well as newly-identified families.

The project combined emergency services meant to meet the immediate needs of families with more long-term mental health services which aimed to help girls, boys, women and men cope with the effects of the Beirut Blast.

Strategy A:  Health 

Expected result 1:  Resilience of vulnerable Lebanese and Syrian refugee families is enhanced by providing health and layered MHPSS services assistance. 

Activities: Home visits, hygiene kit distributions, awareness raising and mental health services

People served within this project were selected based on a vulnerability assessment conducted in the aftermath of the Blast as well as vulnerability assessments conducted for vulnerable individuals identified through internal referral by the home visits teams or self-referral to the JRS MHPSS team (social workers, psychologist, clinical psychologist). The team also received referrals from external actors whose needs were identified by these actors.

Hygiene kits were distributed and awareness raising information was disseminated to families in effort to prevent the spread of COVID-19. JRS consulted with UNICEF and other humanitarian partners to determine the appropriate content of the hygiene kits. In addition to visiting families, home visit team members shared material on coping mechanisms with recipients of hygiene kits. The materials shared were prepared by JRS as well as external entities such as the Ministry of Health and WHO.

JRS extended MHPSS support to individuals in Burj Hammoud and Karantina. The intervention is modelled on the IASC Guidelines on MHPSS in an emergency context, which describes a multi-layered intervention pyramid model for support in effort to protect and improve people’s mental health and psychosocial well-being.

Basic services, including food and cash distributions (described further below), are representative of the first level of support in the IASC model and were designed to meet the immediate needs of families affected by the blast.

JRS staff provided the second level of support, community and family support, which aimed to create a sense of belonging in a community of care through home visits and mass communication on constructive coping methods. 

The third stage of the intervention pyramid, non-specialized support, is again provided through home visit teams. The teams have received Psychological First Aid (PFA) training and participated in various relevant trainings on protection topics.

The highest level of support in this response is specialized mental health services.  A total two JRS psychologists provided this level of support for members of the affected population with severe mental health disorders.  Psychologists directed the training of all other JRS staff participating in the MHPSS response, received tehcnical training themselves and led group therapy sessions and provided one-on-one treatment. JRS continued a relationship with the psychiatric department of Hotel Dieu Hospital for cases that require psychopharmacological assistance. JRS also worked in close collaboration with CEMEDIPP which provides speech therapy, psychological therapy, and psychomotor therapy to project participants. Finally, JRS worked to develop an additional referral network consisting of a private psychiatrist, paediatrician, speech therapist and neurologist. 

Strategy B: Livelihoods 

Expected result 2:  Resilience of vulnerable Lebanese or Syrian refugee families is enhanced through unrestricted cash assistance.

Activities: Needs assessment, cash distributions

JRS conducted a needs assessment to identify households that had damage to their housing and/or its contents. Affected households needing to pay for repairs or the replacement of essential household NFIs were provided with one-off unconditional cash assistance not exceeding 250 USD per family (as recommended in Lebanon Humanitarian INGO Forum meetings).

JRS recognized that it did not have the capacity to assess building cost needs.  However, there was a very real and immediate need for basic household goods, rent, and repair materials.  Unrestricted cash assistance eligibility was based on different factors, including shelter damage. Cash-based interventions have been recognized as an effective response modality in empowering families to meet their specific basic needs and provides both financial security to households and boosts the local economy while allowing recipients to prioritize their own needs.  

Strategy C: Nutrition 

Expected result 3: Resilience of vulnerable Lebanese or Syrian refugee families is enhanced through food assistance

Activity: Food distributions

JRS prioritised the provision of in-kind food assistance to address the unmet food security needs of the most vulnerable families, who may face particular barriers to accessing markets, for instance, due to the destruction of shops as a result of The Blast. According to an initial rapid needs assessment, residents in the vicinity of the blast had reduced their food intake after the explosion and between 50 to 100 percent of people assessed in Nabaa, Burj Hammoud, Karantina and Karm el Zaytoun are in need of food assistance.[1]  

Food baskets were tailored to support the needs of a family of five for one month. It was anticipated that each family would receive two food baskets during the implementation of the project. The content of the food basket was to take into account damage and destruction to beneficiary homes that might prevent the preparation of cooked meals. The basket was intended to primarily contain dry items as well as items that can withstand difficult storage conditions in order to avoid spoilage.  

The food basket were intended to contain most (73%) of the full daily requirement of 2,100 kcal/person/day with minimum macronutrient and micronutrient requirements. As advised by the Lebanon Food Security and Agriculture Sector, specific items might have been substituted based on availability and beneficiary preference whilst maintaining the needed quantity and quality of the overall basket of items.

Strategy D: Programme Management 

Expected result 4: Effective coordination of humanitarian response  

Activities: Attending coordination meetings

JRS regularly attended coordination meetings with other humanitarian stakeholders in the Beirut Blast emergency response and recovery. Key coordination groups include the Lebanon Humanitarian INGO Forum, Education working group, Health working group, Beirut Forward Emergency Room meetings organised by the Lebanese Army forces,  the Lebanese Red Cross, and the inter-agencies. In order to support a coherent and coordinated response by the humanitarian community, JRS sought to ensure that its interventions remained complementary to the efforts of other organisations and avoided overlap by flexibly adapting to changing realities on the ground and actions within the community. Changes to project implementation were discussed with the project donor in advance. 

An external final project evaluation is needed upon the completion of the Mental Health Support After the Beirut Blast Project. The evaluation should reflect the perspectives of members of the affected community engaged in the intervention, JRS staff members, and external stakeholders responding to The Beirut Blast emergency response.

Final project evaluation objectives are as follows:

• Determine the extent to which the project goal and outcomes were achieved, highlighting accomplishments and under-achievements and the main factors attributing to them
• Identify and document lessons learned and best practices
• Provide action-oriented recommendations for enhancing future similar programming

The recommendations should be practical to implement under the current structure of JRS programming and resources available i.e., financially feasible and take into consideration the competencies and number of staff currently within JRS.

Development of key research questions by the consultant in consultation with JRS staff will guide the scope of the evaluation. 

The project evaluation is intended for the two respective JRS Lebanon project teams directly involved in the project. The project evaluation will also inform JRS Lebanon programming as a whole as lessons learned and recommendations can inform services, including basic assistance and mental health, provided across all JRS Lebanon programming.

Findings will be shared among key stakeholders including members of the affected community, JRS staff, the project funder Cordaid, the JRS MENA Regional Office as well as select donors. The consultant may not share the final report with external parties without the written consent of the JRS Lebanon country director.

7.1 Process

The project evaluation should be a collaborative process in which the consultant works in direct coordination with key JRS Lebanon staff including the project director, M&E officer and country director as well as JRS MENA Regional Office staff including the programme officer and M&E officer. The evaluation will be reviewed and approved by JRS Lebanon and the JRS MENA Regional Office based in Beirut. The inception report detailing the proposed methodology, evaluation matrix including evaluation questions according to criterion, and work plan shall be approved by JRS prior to the start of data collection.

The project donor Cordaid will also have a role in various stages of the evaluation including the planning phase and reporting and follow up phase (evaluation phases described further below under 8. Roles and Responsibilities).

The methodology and the analysis must adopt a gender mainstreaming approach i.e., ensure a gender perspective is incorporated into the design of data collection tools, analysis of findings and evaluation conclusions. Age must also be taken into consideration in the design, analysis and reporting as well as the varied backgrounds of project participants, including host and refugee communities.

The IASC Guidelines on MHPSS in emergencies context, which informs the design of the project, promotes a multi-sectoral response to protect and improve people’s mental health and psychosocial support. Thus, the complementarity of the project interventions should be analysed and inform both lessons learned and recommendations for future programming.

There is a high level of vulnerability among the target population. The consultant must include in their technical proposal submitted to JRS Lebanon for the tender process how they will take into consideration the protection concerns of participants in the design and implementation of the evaluation in line with the Do No Harm principle. Additionally, the consultant must provide examples of how they have considered the protection needs of participants in previous research.

The privacy and safety of project participants are of paramount importance and should be safeguarded at all stages of the evaluation.

7.2 Geographic location

Fieldwork will be conducted in Burj Hammoud, Beirut. Considerations must be made during the data collection phase given the rise in fuel costs and adherence to Covid-19 preventative measures.  

7.3 Qualitative data collection

Qualitative methods should include key informant interviews and focus group discussions with project staff, project participants, and external actors active in the response. Project staff include project directors, mental health and health specialists, social workers and home visits team members. Project participants include recipients of multi-purpose cash, food and hygiene items as well as individuals receiving MHPSS services.

7.4 Quantitative data 

Quantitative data collection was built into the design of the project and was collected throughout the life of the project. This data will be reviewed by the consultant in the planning phase of the evaluation as part of the desk review. If deemed necessary by the consultant, any quantitative data collection during the project evaluation will be determined based on consultation with the JRS Lebanon project staff as well as the JRS MENA RO staff and will take into consideration factors including survey fatigue and the principle of Do No Harm.  

The project logframe should serve as a frame of reference for the evaluation. Table 1 below includes the project objective and all project indicators.

Table 1 Project objective, outcome and output indicators

Project Objective: Syrian refugee and vulnerable Lebanese families in Burj Hammoud and Karantina are provided necessary services to respond to the immediate needs from the 04 August 2020 Beirut Explosion and begin the process of recovery.

Level

Indicator*

Outcome

By mid-September 2021, 80% of individuals report a better coping strategy

By mid-September 2021, 80% of people are able to meet their basic needs after receiving cash assistance

By mid-September 2021, 80% of families can satisfy the basic nutrition needs in their own household

Output

By the end of October 2021, 850 vulnerable Lebanese or Syrian families are reached with hygiene awareness raising activities

By the end of October 2021, 1,080 hygiene kits are distributed to vulnerable Lebanese or Syrian families

By the end of October 2021, 850 vulnerable Lebanese or Syrian have received layered MHPSS services (400 specialized MHPSS and 450 PSS cases)

By the end of October 2021, 540 vulnerable Lebanese or Syrian families have received unrestricted cash assistance.

By mid-September 2021, 1,080 food baskets are distributed to vulnerable Lebanese or Syrian families

By mid-September 2021, JRS has participated in at least 80% of meetings conducted by different coordination groups

By mid-September 2021, JRS has created 2 new referral partnership

 *The project was intended to conclude in mid-September, however, additional contributions provided by Cordaid has allowed the project to continue through 31 December 2021. Therefore, time frames for indicators vary according to type of service.

7.5 Evaluation criteria

Evaluation judgments shall be determined based on, at minimum, the six OECD-DAC evaluation criterion – relevance, coherence, effectiveness, efficiency, impact, and sustainability. The consultant will draft evaluation questions in coordination with JRS staff for all evaluation criterion. The weight of each evaluation criteria will also be determined in consultation with JRS staff. The criterion will be thoughtfully applied to the project evaluation. Evaluation questions shall be designed in such a way as to promote critical thinking and organizational reflection, as opposed to a mechanistic manner, with the ultimate goal of assessing project outcomes, learning, and improving JRS Lebanon programming.

Triangulation of information must be considered in terms of data sources, designing data collection tools, and when presenting analysis of findings in the evaluation report.

Table 2 Roles & Responsibilities

Evaluation phase

Role

Task

Planning

JRS MENA Regional Office

• Introduction meeting with JRS Lebanon and research consultant

JRS Lebanon project team

• Provide documentation and list of potential evaluation participants

Evaluation consultant

• Desk review
• Design methodology and finalise with input from JRS Lebanon project team and JRS MENA RO
• Prepare tools and finalize with input from JRS Lebanon project team and JRS RO
• Inception report

JRS Lebanon HR

• Training for evaluation consultant (and enumerators, if necessary) on JRS Child Protection Policy and Procedures

Data collection and analysis

JRS Lebanon project team

• Facilitate data collection process in Beirut by contacting all selected participants and ensuring their presence at designated safe spaces for data collection

Evaluation consultant

• Implement data collection process
• Ensure data quality
• Hold debriefing sessions with project teams at the end of each day in the field

Reporting and follow up

Evaluation consultant

• Submit first draft report in English
• Host workshop with JRS Lebanon and JRS RO staff
• Reflect discussion feedback in final report
• Submit final report
• Share all finalized data collection tools and data collected (transcripts of KIIs and FGDs in English and language in which they were conducted)

JRS MENA Regional Office, JRS Lebanon project team

• Submit all draft reports to consultant with comments and questions

JRS MENA Regional Office, JRS Lebanon project team, Evaluation consultant

• Participation in evaluation workshop

The evaluation will begin on 1 February and conclude on 4 March 2022 (25 working days). The consultant is expected to follow the Lebanon work week schedule i.e., communication with Lebanon staff and data collection to be conducted during regular working hours, only.

Table 3 Timeline & Products

Date

Item

Deliverable

Days

1 – 4, 7 February

• Introduction meeting with team members
• Desk review

5

8 – 11, 14 February

• Evaluation design
• Tool development in coordination with JRS staff
• Development of field visits plan in coordination with JRS staff
• Train evaluator, enumerators (as necessary) on JRS CP Policy & Procedures
• Train enumerators on tools (if necessary)

• Product 1: Inception report including detailed assessment methodology with a plan of action and deadlines
• Product 2: qualitative and quantitative tools

5

February 15 - 18

• Fieldwork

• Notes from key informant interviews with JRS staff and external actors
• Notes from focus group discussion with JRS staff and external actors
• Notes from qualitative and quantitative methods with project participants

4

21-25, 28 February

1 March

• Data analysis
• Prepare draft report
• Submit draft report to JRS Lebanon and Regional Office

• Product 3: Submission of draft report on 28

7

2 – 4, 7 March

• Workshop to discuss findings of 1st draft
• Prepare final report
• Submission of final report to JRS Lebanon and Regional Office

• Product 4: submission of final report on 4

4

The evaluation report should be designed according to the contents presented in the table below.

Table 4 Report template

Cover page

Table of Contents

Table of Figures and Tables

Acknowledgments

List of abbreviations

Executive Summary

Introduction

Project background

Project summary

Methodology (sampling, tools, limitations)

Findings

Analysis of findings based on evaluation criterion

Lessons learned

Best practices

Conclusions

Recommendations for future programming

Annexes

The budget includes all expenses associated with the evaluation including transportation, any enumerators hired by the research consultant (if necessary), and any other costs associated with the evaluation which may be accrued. 

Payment for services will be made in three instalments. The first instalment will be 20% of the budget upon the signing of the contract. The second instalment will be 40% of the total after submission of 1st draft report. The remaining 40% of the total to be paid upon submission of the final report as agreed upon by the consultant and JRS Lebanon and the JRS Regional Office.

Interested candidates must submit a detailed budget with the breakdown of the project phases and products.

Applicants must meet the qualifications detailed below.

• Advanced degree in psychology, social work, public health or other relevant field, or the equivalent combination of work and education experience in a related area
• A minimum of 7 years of experience conducting field-based evaluations using mixed methods in a humanitarian context, preferably in Lebanon
• Experience conducting evaluations of mental health projects is required
• Must possess strong analytical skills
• Strong cultural sensitivity
• Fluency in English and Arabic (reading, writing and speaking).