Call for Consultant - Evaluator

1.Consultancy Information

Project Title: “REFORM: Towards a strengthened quality response and inclusive and dignified durable solution for GBV survivors in Lebanon”

Duty Station: Home-based with travel as needed and subject to the contextual constraints

Duration: between September 2022 to December 2022

2. ABOUT ABAAD              

ABAAD – Resource Center for Gender Equality, is an UN ECOSOC accredited organization that aims to achieve gender equality as an essential condition to sustainable social and economic development in the Middle East and North Africa (MENA) region. ABAAD is a women-led and supported organization that works on three pillars: (a) Direct Services, (b) Capacity building and resource development, and (c) Advocacy and policy reform.  ABAAD, since its establishment in 2011, has been identified as a leading actor on gender equality in the region and as a reliable reference and partner by the local, regional and international entities that promote gender equality, peace building and sustainable development.

3. Background and Context

ABAAD, through the REFORM project, intends to strengthen national systems capacities to enhance protection and mitigation of gender-based violence (GBV) against women and girls in Lebanon among Syrian refugees and vulnerable host communities, and to contribute to supporting vulnerable groups in displacement situations to be capacitated and empowered to move out of dependency on aid and choose a durable solution, as appropriate. This project will also ensure opportunities for development, prevention, safety, and protection for women and girl survivors and at-risk of GBV in Lebanon through addressing and enhancing their abilities to cope with adversity, mitigate and deconstruct conflict in their direct environments and communities, and tackle the legislative frameworks and policies that affect them directly.

Specific Objectives:

1. Protection space for vulnerable refugees & host communities is enhanced.
2. A more conducive environment for durable solutions is enhanced.

A baseline, mid-point, were conducted throughout this three-year program cycle to assess progress, efficiency, effectiveness and ensure recommendation of remedial actions throughout the implementation on the project action plan.

The outcomes and outputs of the 2 interventions were:

ANNEX 1

5.Purpose and Objectives:

In line with ABAAD’s accountability principle towards the beneficiaries and the donor – The European Development and Protection Programme (RDPP) - ABAAD is seeking an evaluator to provide an assessment of the overall project progress and results against the objectives and indicators of achievement.

The evaluation will assess the impact, effectiveness, and sustainability of the project; explore the key factors that have contributed to the achieving or not achieving of the intended results; and determine the extent to which the project is contributing to improving public service delivery; addressing crosscutting issues of gender equality and women’s empowerment human rights, advocacy part; and forging partnership at different levels: Lebanese government, local communities and donor.

The intended users of the evaluation are:

The primary users: Project stakeholders and partners, in particular RDPP.

The secondary users: Other organizations operating in the same communities, especially those with a programmatic focus on gender.

The evaluation exercise is intended to inform ABAAD and RDPP about the project’s impact on the direct and indirect beneficiaries of the project within their respective communities, programme recommendation and lessons learned.

6.Scope:

The evaluation will cover activities that have taken place since the beginning of the project May 1,2019 until December 31, 2022. Assess the change pathways of observed results (on an outcome and impact levels) of the project against planned objective and related indicators, with a focus on testing the causal-effect linkages between different results horizontally and vertically.

Stakeholders to participate in the assignment:

• ABAAD, project and organization staff
• Representatives of Ministry of social affairs (MOSA) and ministry of interior (ISF members), National mental health (NMH)
• Representatives of social development center SDC
• Project beneficiaries and surrounding communities (direct and indirect participants)
• Other humanitarian and development actors in Lebanon (UNHCR, GBV task force…)
• RDPP, project and organization staff

Geographical Scope:

Evaluation of project activities in the following implementation areas: Beirut-Mount Lebanon, Beqaa, Men center, MWH  

7.Evaluation Question

Referencing and adopting from Organization for Economic Cooperation and Development (OECD) Development Assistance Committee (DAC) evaluation criteria, the evaluation seeks to answer the following questions while focusing on relevance, effectiveness, efficiency, impact and sustainability. We need to understand why it happened, factors helped/challenged, channels of change, and testing of the proposed TOC against actual.

Relevance:

• Did the project’s theory of change clearly articulate assumptions about why the project approach is expected to produce the desired change? Was the theory of change grounded in evidence? And if not why not?
• Was the project relevant to the needs and priorities of the target groups/beneficiaries? Were they consulted during design and implementation of the project?  
• Was RDPP funding used to leverage political/religious, windows of opportunity for engagement? in what ways this was done? And how well does the this fit the objectives and influenced targeted partners?
• Was the project well-timed to address a conflict factor or capitalize on a specific window of opportunity?
• Does the intervention meet the practical and strategic gender, status of origin(refugee/host community)  and protection needs of all genders?   What can be improved?
• How well does the capacity building of local authorities and community partners fit their needs?

Effectiveness:

• In what ways the capacity building did/did not influence both the accessibility and the performance of the GBV protection services they provided?
• What factors (internal or external) have contributed to achieving or not achieving intended project outputs and outcomes?
• hat are the factors in that contributed in achieving or not in achieving change in reported SGBV cases as a result of enhancing reporting systems and activities 
• To what extent has the project succeeded in fulfilling female and male beneficiaries’ practical and strategic needs including but not limited to improved access to services, enhanced practical capacity, and gaining new skills?

Efficiency:

• To what extent has there been a reasonable use of financial and human resources? Have resources (funds, human, resources, time, expertise, etc.) been allocated strategically to achieve outcomes? If not, what can be improved?
• Where there any constraints to more cost-efficient ways? What prevented the team from adopting those ways?
•  To what extent have the MEAL systems utilized by ABAAD enabled effective and efficient project management and reporting?
• In reference to the overall goal and TOC approaches ( enhancing direct beneficiary abilities to cope, or, enhancing the capacity of the service providers, or tackling the legislative frameworks and policies that affect them directly), which pathways were more cost-efficient? Why?

Impact:

• How did the REFORM project intervention enhance protection space for vulnerable refugees & host communities? What types of change was observed at the impact level  as a result of different outcomes/contribution?
• To what extent did the media campaign change the public’s opinion regarding gender sensitive, and gender equality issues.
• To what extent have the projects’ objectives and impacts contributed to equal power relations between men and women? And how did it influence men’s perceptions and attitudes towards gender roles?
• Does the platform AIOMIS created as part of the capacity development component meet ABAAD’s needs and is it in line with its strategic vision?
• How did COVID-19 impact the project’s implementation?
• To what extent did the project tackled the legislative frameworks and policies, and what type of impact resulted if any? Why / why not.
• To what extent has the project supported innovative solutions, employed rights-based approaches, and contributed to increased local ownership of issues and solutions?Sustainability:

• Did the intervention contribute to greater gender equality within a wider social scale? If so how, and to what extent?
•  Are there any social or political risks that may threaten the sustainability of the project’s outputs??
• To what extent have relevant Ministries or national offices integrated project outcomes/ outputs into ongoing policies and practices?
• In what ways, and would this result in any anticipated long-term sustained changes?
• What are the sustainability prospects of the AIOMIS platform? are there enough resources at the organization to sustain and maintain the platform?

8. Methodology

The evaluation follows a mixed-methods approach of non-experimental design. The methods planned include but not restricted to: (i) a desk review including but not limited to the project’s documents, location and implementation context, review of publicly available similar project reports and documents on gender and equality by UN, NGOs, and other local and international organizations.  (ii) FGD (with beneficiaries from all nationalities and genders) and KIIs (with project staff and other relevant stakeholders), in addition to 2 quantitative surveys with females and male beneficiaries. Qualitative data is to be transcribed and thematically analyzed. Quantitative data is to be analyzed using univariate and bivariate statistical methods. Where possible, results are to be stratified by age  gender(M/F), status of origin, and status of origin(refugees/ host community) . A sample large enough is to be collected in order to guarantee valid statistical results where necessary.

The evaluation will follow ADC and OECD/DAC norms and standards as well as ethical guidelines for evaluations.   

9. Review products/Deliverables

The consultants /evaluation team will be expected to deliver the following:

1. Evaluation inception report:

including the following sections:

• Background, Purpose and Objectives.
• Evaluation Design and Approach:
• Methods
• Evaluation Matrix
• Data Collection Instruments
• Data Analysis
• Limitations, Risks and Mitigation Measures
• Quality Assurance and Ethical Considerations
• Workplan
• Annexes

The inception report will be shared with Abaad for feedback. The consultant(s) shall submit a revised inception report based on the received feedback. Deadline: 2 weeks after evaluation kick off

2. Data collection and field work: The fieldwork phase will cover all qualitative and quantitative data collection and triangulation including involvement of project beneficiaries and other stakeholders. present their preliminary findings.
3. Draft evaluation report: ABAAD and RDPP will review the draft evaluation report and provide an amalgamated set of comments to the evaluator within 10 days, addressing the content required (as agreed in the inception report)

Deadline: No later 10 weeks after the evaluation kick-off meeting (November 30 2022). The evaluation report should include:

• Presentation of the evaluation with key findings;

• Presentation of recommendations
• All data must be disaggregated by sex, age, status of origin(refugees/host community)and disability;
• Submission and presentation of the draft report to RDPP and ABAAD
• Period of comments and feedback from ABAAD
• Submission final draft report, inclusive of comments.
• Submission of final report
• Presentation of final report (via PowerPoint presentation)

4. Final evaluation report based on the feedback received from Abaad. To be submitted no later than 2 weeks after receiving the feedback exact data (20 December 2022)
5. Evaluation results presentation to be conducted in a meeting which includes the project’s primary stakeholders. Deadline: to be agreed on with ABAAD.

10.Evaluation management arrangement:

ABAAD’s MEL advisor will ensure oversight of the evaluation process ensuring that the agreed milestones are timely met and the agreed methodology has been used. 

Final Evaluation Report: A draft of the final evaluation report [no less than 25 and no more than 40 pages, including a two-to-three-page executive summary with key findings and recommendations and excluding annexes will be submitted to ABAAD.  Findings and recommendations must be structured according to the evaluation questions. An outline of the final report structure should be agreed upon during the inception phase. At least the following must be included: cover page; result assessment form (as an annex); executive summary (should be included in the report and also as an independent document); background and objectives of the assignment; methodology (with mention of limitations, if any); summary of findings; analysis of findings addressing guiding questions; conclusions, lessons learned, and recommendations; and agenda of all activities carried out (annex).

ANNEXES:

ANNEX 1: REFORM project Logframe

Theory of Change

If a protection space for vulnerable refugees and host communities is enhanced, and if a more conducive environment for durable solutions is promoted, and if the legislative frameworks and policies that affect them directly are addressed and modified, then opportunities for development, prevention, safety, and protection for women and girl survivors and at-risk of gender-based violence in Lebanon are ensured.

Overall Objective

Output Description

Activities

Area of Implementation

Strategic Objective 1: Protection space for vulnerable refugees & host communities is enhanced

Output 1.1.

Enhanced &
sustained capacity of local
authorities and civil society
actors enabling vulnerable
rights holders to enjoy rights
and access to quality
protection services

A 1.1.1 Training of Trainers’ workshops and coaching sessions targeting ISF members on the psychological considerations while dealing with men with abusive behaviours and/or women survivors.

Beirut, Mount Lebanon

A 1.1.2. National TOTs (targeting mainly MOSA SDCs) on the produced Case Management training manual and the National GBV SOPs & its related mobile app under Phase 1.

8 Governorates: Beirut, Mount Lebanon, South Lebanon, Nabatieh, North Lebanon, Akkar, Beqaa, Baalbeck

1.1.4. Gendered capacity assessment is conducted to MH facilities in partnership with the National Mental Health Program

Facilities across Lebanon

Output 1.2.

Holistic GBV service
provision is ensured to
women and girls survivors or
at risk of GBV in life
threatening situations

1.2.1. Provide women and children survivors of or at risk of GBV with safe sheltering in Mount Lebanon and North Lebanon

Mount Lebanon, North Lebanon

1.2.2. Provide women and girls survivors of or at risk of GBV with GBV multi secotral services

Sin el Fil, Ghobeiry, Msaytbeh

1.2.3. Provide women and girls survivors or at risk of GBV with PSS & life skills sessions

Sin el Fil, Ghobeiry, Msaytbeh

1.2.4 A 1.2.4: Conduct protection sensitive referrals to emergency and protection cash assistance managed by UNHCR (ECAP and PCAP) as needed

Sin el Fil, Ghobeiry, Msaytbeh

1.2.5: ABAAD’s Mobile Unit conducts community interven tions in hard-to-reach areas in Lebanon

North Lebanon and Beqaa

1.2.6: Produce community videos featuring “stories of hope” from refugee and host communities to be used for pro moting social stability and posi tive messaging. These videos will be screened in various com munity structures, registration offices & mobile unit touring.

N/A

Output 1.3

Men with abusive behaviors have improved access to quality engendered psychotherapy or PSS services

1.3.1. Provide engendered psychotherapy sessions (target ing men and male youth with violent behaviors from both host and refugee community) through Men Centers.

Beirut,  Baalbeck

1.3.2. Conduct training work shops on masculinities and stress and anger management (two days each) targeting 225 men and male youth

Beirut, Baalbeck

Output 1.4

Refugees are better sensitized on gender and GBV issues in destination countries of resettlement (third countries)

1.4.1. Produce a directory of GBV services in destination countries with UNHCR and embassies counselor sectors to be integrated in pre-settlement sessions

N/A

1.4.2. Develop sensitization and IEC materials In collaboration with UNHCR and embassies of destination countries, ABAAD will lead on the development of sensitization and IEC material based on the GBV directory

Various areas across Lebanon

1.4.3: Conduct a Gender & GBV prevention-focused induction for women and girls & men and boys as a preparation for their resettlement to third countries.

Embassies, airline companies

Output 1.5

ABAAD has improved organizational capacity to manage and implement programs

1.5.1. ABAAD receives capacity development on Data Storage and Management and on Security & Contingency Planning and other areas.

N/A

Strategic Objective 2: A more conducive environment for durable solutions is enhanced.

Output 2.1.

Research published and disseminated to relevant stakeholders

2.1.1. Conduct a regional comparative study on social norms & behavioral change strategies (Lebanon, Syria, Jordan, & Iraq) on GBV prevalence among Syrian families.

Lebanon, Syria, Jordan and Iraq

2.1.2. Produce Fact Sheets based on study’s findings including practical recommendations to be used in advocacy meetings with stakeholders

N/A

Output 2.2.

Context-specific policy dialogue and support to national actors for legislative changes, policy adjustments and initiatives that support reform and durable solutions

2.2.1: Conduct lobbying meetings with national, government, and male religious actors to enhance justice for sexual assault survivors to put an end to VAW

N/A

2.2.2 Production and dissemination of materials, throughout cycle of project and through the 16 Days of Activism in 2019 throughout 2021

Various areas across Lebanon