Call for Expression of Interest for Independent Cluster Evaluation for Employment Intensive Infrastructure Programmes in Lebanon (Phase III & IV) & Jordan (Phase V)

1. Background

The civil war in Syria led an influx of refugees to neighbouring countries including Lebanon and Jordan. It is estimated that Lebanon hosts about 1.5 million Syrians as of 2021, whereas 1.36 million Syrians reside in Jordan.

With the influx of refugees, labour market challenges have been exacerbated in both countries. High unemployment, competition for work and informality have contributed to social tension between Syrian refugees and host communities. Unskilled local workers perceive their unemployment and the poor working conditions have resulted from competition with Syrian workers. Syrians have traditionally worked in agriculture, construction, and services and have mostly relied on temporary and informal work. Migration flows generated a heavy strain on public infrastructure, services, natural resources, and economy, including the labour market.

To minimize the labour market impact of the crisis, the Government of Lebanon has exempted Syrians from the general prohibition on foreigners working and have identified areas of work, namely agriculture, construction and environment activities, as the sectors where Syrian refugees can legally work. However legal stay and permit requirements are complex and have changed frequently. To regulate their employment, refugees can either have a Lebanese sponsor or apply through UNHCR and work in the identified sectors. Yet, employers are not motivated to arrange work permits. While there are some challenges, such as lack of documentation, limited professions and sponsors, complexity, cost, time and effort, workers with permits benefit from increased confidence, rights and entitlements and improved working conditions.

Similarly, the Government of Jordan has restricted employment of refugees but gradually eased the regulations, including amendments to the work-permit issuing process and de-linking work-permits from employers in some sectors, such as construction and agriculture. Moreover, the Government of Jordan introduced new pathways for refugees to stay legally in the country. Refugees not registered with UNHCR are now entitled to obtain a Ministry of Interior card for them to stay legally in the country, including outside of camps. Currently, employment of Syrian refugees is permitted in five sectors: agriculture, construction, manufacturing, food and beverage services and wholesale & retail trade.

The Government of Lebanon through the Lebanon Crisis Response Plan has indicated a strong need for assistance in job creation and infrastructure to enable communities to cope with increased demand. Key mechanisms highlighted at the London and subsequent conferences as a means of creating productive infrastructure and employment, were the Local Host Support Programme and the Employment Intensive Investment Programme (EIIP) in Lebanon.

Germany, through the German Federal Ministry for Economic Cooperation and Development (BMZ), launched an initiative called “Partnership for Prospects – Cash for Work” focused on Syria and its neighbours, with the purpose of creating jobs for refugees using employment intensive construction methods and skills development. With support from BMZ through the Kreditanstalt für Wiederaufbau Development Bank (KfW), the ILO and UNDP have jointly been implementing EIIP Phases I and II in Lebanon, engaging with Ministry of Social Affair and Ministry of Labour. Phase III continues the same partnership with the UNDP with adjustments based on experiences gained and lessons learnt in Phase I & II.

The Government of Jordan and international actors agreed on the implementation of the Jordan Compact to promote sustainable livelihoods for Syrian refugees and vulnerable Jordanians. The Jordan Compact is to respond to concrete vulnerabilities and needs in priority sectors, including Livelihoods. The response plan designed for the Livelihoods sector includes a large number of initiatives to stimulate local economic development and support sustainable employment creation through vocational training, employability skills, job matching and placement services, self-employment and entrepreneurship. In this regard, it is critical to ensure that there is a capacity to respond to the short-term needs of the vulnerable Syrian refugees and Jordanians. This is where the EIIP assumes an unequivocal and enormous strategic importance as an instrument for social cohesion and stability. The ILO has been partnering with BMZ and KfW to assist the government in ensuring that Syrian refugees and Jordanians can access better living conditions through increased employment and improved infrastructure.

2. Project Background

Lebanon EIIP Phase III & IV

Phase III and IV of the EIIP Lebanon build on Phases I & II, using the similar objectives and approaches. The emphasis is on decent employment creation for Lebanese host community members and Syrian refugees through the construction of locally prioritised infrastructure, maintenance, and environmental works. The project will work with local contractors, distributing wages and improving working conditions. The project objectives are,

Phase III

1. Decent employment generated for Lebanese host communities and Syrians refugees through sustainable infrastructure development and environmental works and maintenance of public assets.
2. Enhanced capacity for decent job creation and asset management through institutional development and training.

Phase IV

1. Direct employment creation using EIIP, LRBT and DWP approaches for sustainable infrastructure development, environmental works and maintenance of public assets
2. Enhancing capacity for job creation and asset management with the Ministry of Labour (MoL), the Ministry of Social Affairs (MoSA) and the private sector

EIIP Lebanon Phase III and IV contributed to Decent Work Country Programme 2017-2020 for Lebanon, primarily the second pillar (improving decent working conditions, enhancing productive employment opportunities) but also the first pillar (establishing a sound legislative environment, improving governance and social dialogue). The projects fall under the livelihood sector of the Government-led Lebanon Crisis Response Plan, particularly under Livelihood Outcome 1 (Stimulate local economic development and market systems to create income generating opportunities and employment), Outcome 2 (Improve workforce employability), and Outcome 3 (Strengthen policy development and enabling environment for job creation). The project is aligned with the United Nations Strategic Framework (UNSF) for Lebanon 2017-2020 with a focus on supporting the country to preserve peace and consolidate stability.

Jordan EIIP Phase V

Jordan EIIP Phase V also builds on previous phases, while the implementation period overlaps with Phase III and IV. The overall development objective of the project is that Syrian refugees and Jordanians have better living conditions because of increased employment and improved infrastructure. The Project outcomes are as follow:

1. To generate employment opportunities and to improve the access to the labour market for Syrian refugees and Jordanians
2. To improve infrastructure through the use of labour-intensive methods for men and women

The project contributes to Decent Work Country Programme 2018-2022 for Jordan, particularly priority 1 (Employment creation contributes to economic and social stability) and priority 2 (Decent working conditions for all create a level playing field for male and female Jordanians, refugees and migrants). It is aligned with United Nations Sustainable Development Framework (UNSDF) for Jordan 2018-2022 priority 3 Enhanced Opportunities. The Project is also aligned with the Jordan Response Plan, specifically objective 1 (Enhance self-reliance and living conditions of Syrian refugees and vulnerable Jordanian) and 2 (Meeting the humanitarian and resilience needs of Syrian refugees and vulnerable Jordanians).

Project durations and budgets can be found on the 2nd page of this ToR.

3. Evaluation Background

ILO considers evaluation as an integral part of the implementation of development cooperation activities. Provision is made in the projects in accordance with ILO evaluation policy and based on the nature of the projects and the specific requirements agreed upon at the time of the project design and during the projects as per established procedures.

This cluster evaluation serves as the independent final evaluation for EIIP Lebanon Phase III and Jordan Phase V and as the independent mid-term evaluation for EIIP Lebanon Phase IV. EIIP Lebanon and Jordan share similar characteristics, including their focuses and areas of work. Given that they are also funded by the same donor, it is strategic to cluster evaluations and generate lessons learnt and good practices collectively for the on-going and future phases as well as other similar projects.

4. Evaluation Purpose and objectives

This cluster evaluation will examine the relevance, coherence, effectiveness, efficiency, sustainability, and potential impact of the projects. It will provide recommendations for immediate adjustments of EIIP Lebanon Phase IV, while it also provides recommendations for the future phase of EIIP in Lebanon and Jordan. This evaluation will consider previous evaluations of EIIP projects for Jordan and Lebanon to respond some of the evaluation questions, particularly under sustainability and potential impact as well as to identify persistent challenges in EIIP projects.  

Specifically, the evaluation will examine the following aspects: 

• Changes in context and review of assumptions (relevance):  Is the project’s design adequate to address the problems at hand? Were the project objective and design relevant given the political, economic, and financial context?
• Results in terms of outcomes and outputs achieved (effectiveness): How have the projects contributed towards project’s goals? To what extent did it contribute to the ILO’s Programme & Budget, Country Programme Outcomes, and more largely SDGs?
• Use of resources in achievement of projected performance (efficiency): How have the resources been used to fulfil the project performance in an efficient manner with respect to cost, time and management staff?
• Assessment of impact (impact): To what extent have the projects contributed long-term intended impact? 
• Sustainability: Will the projects’ effects and built capacity remain over time? 

The evaluation will comply with ILO evaluation policy, and the UNEG ethical guidelines will be followed.

5. Scope of Evaluation

The evaluation will review the project outcomes and outputs to date. The geographical coverage will be aligned with the scope of the projects. The evaluation should cover from the starting date of the projects to April 2022. As cross-cutting themes, the evaluation will take specific note of integration of gender mainstreaming, disability inclusion, International Labour Standard, social dialogue, and environmental sustainability as well as COVID-19 response.

6. Clients of Evaluation

The primary clients of this evaluation are constituents in Lebanon and Jordan including government entities, the BMZ/KfW, partner UN agencies, and ILO ROAS & EMP/INVEST. Secondary users include other project stakeholders and units within the ILO that may benefit from the knowledge generated by the evaluation.

7. Evaluation Criteria and Questions

The evaluation utilizes the standard ILO evaluation framework and follows the OECD/DAC evaluation criteria:

Relevance and strategic fit

5. Are the project objectives aligned with sectoral national priorities? How do the projects fit into the national dialogue in relation to the Humanitarian Development Peace Nexus?
6. How do the projects contribute to the ILO’s Programme & Budget, Decent Work Country Programmes, United Nations Sustainable Development Framework, and SDGs?
7. To what extent were the EIIP technologies and implementation modalities appropriate for the situation in Lebanon and Jordan during the project implementation period?

Coherence and validity of the design

5. Are the project strategies and structures coherent and logical?
6. Do the projects make a practical use of a monitoring and evaluation framework? How appropriate and useful are the indicators in assessing the project’s progress? Are indicators gender sensitive? How evaluable are the projects’ set-up for Phase IV Lebanon?
7. To what extent did the project design take into account specific gender equality and non-discrimination concerns, including inclusion of people with disabilities?

Project progress and effectiveness

5. What progress has the project made towards achieving the overall objective and outcomes?
6. How did outputs and outcomes contribute to ILO’s mainstreamed strategies including gender equality, social dialogue, labour standards, and environmental sustainability?
7. To what extent did the project respond emerging and changing needs in terms of COVID-19 pandemic in both countries and the economic crisis in Lebanon? What could have been done better?

Efficiency of resource use

5. To what extent have project activities been cost-efficient? Have resources (funds, human resources, time, expertise etc.) been allocated strategically to achieve outcomes? To what extent can the project results justify the time, financial and human resources invested in the project?
6. To what extent has the project been able to build on other ILO or non-ILO initiatives either nationally or regionally, in particular with regard to the creation of synergies in cost sharing?

Effectiveness of management arrangements

5. How does the project governance structure facilitate good results and efficient delivery? And if not, why not?
6. How effective was communication among the project teams, the regional office and the responsible technical department at ILO headquarters? Have the projects received adequate technical and administrative support/response from the ILO backstopping units?

Impact orientation

5. What is the likely contribution of the projects to the impact of the intervention?

Sustainability

5. Are the results achieved by the projects likely to be sustainable? What measures have been considered to ensure that the key components of the project are sustainable beyond the life of the project, including potential exit strategy?
6. To what extent was sustainability of impact taken into account during the design of the project?

8. Methodology

This evaluation is summative and relies on both quantitative and qualitative approaches to respond evaluation questions and fulfil the purpose. It consists of,

• Desk review of existing documents: The evaluator will conduct systematic analysis of existing documents and obtain existing qualitative and quantitative evidence prior to primary data collection. The desk review also facilitates assessment of the situation and available data to plan the evaluation and develop the inception report.
• Key information interviews: Online individual interviews will be conducted with a pre-agreed list of stakeholders who have in-depth exposure and understanding of the project and their context. Interview guide(s) will be developed during the inception phase to stimulate a discussion on concerned evaluation questions.
• Focus group discussions: Focus group discussions with beneficiaries both in Jordan and Lebanon to collect their insights in the project interventions.
• Evaluation debriefing: The evaluator will present evaluation findings.

Any changes to the methodology should be discussed with and approved by the Regional Evaluation Officer during the inception phase.

9. Work Assignments

1. Kick-off meeting

The evaluator will have an initial consultation with the evaluation manager, relevant project team members and programme officers. The objective of the consultation is to reach a common understanding regarding the status of the project, the priority assessment questions, available data sources and data collection instruments and an outline of the final assessment report. The following topics will be covered: status of logistical arrangements, project background and materials, key evaluation questions and priorities, outline of the inception and final report.

2. Desk Review

The evaluator will review project background materials before conducting interviews. Documents to review include but are not limited to Regional Refugee & Resilience Plan, Lebanon Crisis Response Plan, Jordan Response Plan, United Nations Strategic Framework for Lebanon, United Nations Sustainable Development Framework for Jordan, ILO Programme and Budget, Decent Work Country Programmes, previous EIIP evaluations, project document including results framework, and project progress reports.

3. Inception Report

The evaluator will draft an Inception Report, which should describe, provide reflection and fine-tune the following issues:

• • Project background
  • Purpose, scope and beneficiaries of the evaluation
  • Evaluation matrix, including criteria, questions, indicators, data source, and data collection methods  
  • Methodology and data collection tools
  • Main deliverables
  • Management arrangements and work plan

4. Primary Data Collection (Key Informant Interviews & Focus Group Discussions)

Following the inception report, the evaluator will interview constituents/stakeholders. Individual or group interviews will be conducted. The preliminary list is as follows and will be agreed during the inception phase,

Lebanon

• • Ministry of Social Affairs
  • Ministry of Labour
  • Municipalities
  • KfW
  • UNDP
  • ILO EIIP Lebanon Project Team

Jordan

• • Ministry of Local Administration
  • Ministry of Public Work and Housing
  • Ministry of Labour
  • Governorate Public Works Directorates
  • Municipalities
  • KfW
  • ILO EIIP Jordan Project Team

Other

• • ILO EMPINVEST
  • ILO ROAS

Focus Group Discussions will be also conducted with direct beneficiaries.

5. Final Report

The final report will follow the format below and be in a range of 40-50 pages in length, excluding the annexes:

1. Title page
2. Table of Contents, including List of Appendices, Tables
3. List of Acronyms or Abbreviations
4. Executive Summary with key findings, conclusions and recommendations
5. Background and Project Description
6. Purpose of Evaluation
7. Evaluation Methodology and Evaluation Questions
8. Key evaluation findings (organized by evaluation criteria)
9. A table presenting the key results (i.e. figures and qualitative results) achieved per objective (expected and unexpected)
10. Clearly identified conclusions and recommendations (identifying which stakeholders are responsible and the time and resource implications of the recommendations)
11. Lessons Learned (in prescribed template)
12. Potential good practices (in prescribed template)
13. Annexes (list of interviews, TORs, list of documents consulted, good practices and lessons learned in the ILO format, etc.)

The quality of the report will be assessed against the ILO Evaluation Office (EVAL) Checklists 4.2, 4.3, 4.4. The deliverables will be submitted in the English language and structured according to the templates provided by the ILO. 

6. Debriefing

To close the evaluation, the evaluator will present findings and recommendations to stakeholders.

10. Evaluation Timeframe

The evaluation is to commence in April and complete in August 2022. The table in the attached ToR describe the tentative timeline,

Total payable working days of lead evaluator:  29.5 Days

Total payable working days of each of national enumerators: 6 Days

11. Implications of the COVID crisis on the evaluation

The COVID-19 pandemic may restrict the mobility of consultants. The primary data collection of this evaluation is planned to be mostly conducted face-to-face by the evaluator and national enumerators in both Lebanon and Jordan. But, the situation and national regulations may change  and result in a shift to remote data collection.

When and where relevant, evaluation questions will also be guided by the ILO protocol on collecting evaluative evidence on the ILO’s Covid-19 response measure through project and programme evaluations, available at: https://www.ilo.org/wcmsp5/groups/public/---ed_mas/---eval/documents/publication/wcms_757541.pdf

12. Deliverable

The deliverables for the lead evaluator consist of the following:

• Deliverable 1: Inception report
• Deliverable 2: Draft evaluation report
• Deliverable 3: PowerPoint presentation on debriefing
• Deliverable 4: Final evaluation report with separate template for executive summary and templates for lessons learned and good practices duly filled in (as per ILO’s standard procedure, the report will be considered final after quality review by ILO Evaluation Office)

The deliverables for the national enumerators are,

• Deliverable 1: Transcription of focus group discussions in English for the assigned country, approved by the lead evaluator and evaluation manager

13. Payment Term

For the lead evaluator

1. 10 per cent of the total fee against deliverable 1 above approved by the evaluation manager
2. 40 per cent of the total fee against deliverable 2 above
3. 50 per cent of the total fee against deliverable 3 and 4 above, approved by the ILO Evaluation Office.

For the national enumerators

1. 100 per cent of the total fee against deliverable 1 above approved by the evaluator and evaluation manager

Travel cost will be reimbursed with invoices. DSA payment and transport for travel will be based on ILO Rules and Regulations.

14. Responsibilities

The evaluator will report to the ILO’s evaluation manager and should discuss any technical and methodological matters with him. The national enumerators report to the evaluator and evaluation manager. The project teams and support units will provide administrative and logistical support during the data collection. The evaluation manager will coordinate with ILO Evaluation Office, who approves and signs off the final evaluation report.

The evaluator is responsible for conducting the evaluation according to the terms of reference (ToR). He/she will:

• Review the ToR and propose any refinements to evaluation questions and methodology during the inception phase
• Review project background materials (e.g., project document and progress reports).
• Prepare an inception report
• Develop and implement the evaluation methodology (i.e., conduct interviews, review documents) to answer the evaluation questions
• Conduct preparatory consultations with the evaluation manager prior to the evaluation mission
• Conduct interviews and collect information according to the suggested format
• Present preliminary findings
• Prepare an initial draft of the evaluation report with input from ILO specialists and constituents/stakeholders
• Prepare the final report based on the ILO, donor and other stakeholders’ feedback obtained on the draft report.

National enumerators are responsible for,

• Provide interpretation support for key informant interviews in the assigned country when necessary
• Conduct focus group discussions as per inception report
• Transcribe the focus group discussions and submit the transcriptions to the evaluator and evaluation manager

15. Legal and Ethical Matters

• This evaluation will comply with ILO evaluation guidelines and UN Norms and Standards.
• The ToRs is accompanied by the code of conduct for carrying out the evaluation “Code of conduct for evaluation in the ILO”. The selected consultants will sign the Code of Conduct form along with the contract.
• UNEG ethical guidelines will be followed throughout the evaluation.
• The consultants will not have any links to project management or any other conflict of interest that would interfere with the independence of the evaluation.

16. Qualification

The evaluator is expected to have following qualifications,

• Proven experience in the evaluation of development interventions
• Expertise in labour intensive modality, job creation projects, capacity building and skills development.
• An understanding of the ILO’s projects. Prior experience in the region, particularly in Jordan and Lebanon, is asset.
• High professional standards and principles of integrity in accordance with ILO Evaluation Policy and United Nations Evaluation Group Norms and Standards.
• An advanced degree in a relevant field.
• Proven expertise on evaluation methods and the ILO approach.
• Full command of English. Command of Arabic is an advantage.
• The evaluator should not have any links to project management or any other conflict of interest that would interfere with the independence of the evaluation.
• Previous experience in evaluations for UN agencies is preferred, particularly ILO.

The national enumerators are expected to have following qualifications,

• At least three years of professional experience in qualitative data collection and reporting
• Relevant translation experience between Arabic and English
• Excellent command of English and Arabic.
• Demonstrated experience in qualitative research is an advantage
• Previous experience with the UN agencies is an advantage
• Extensive knowledge of Lebanon/Jordan context
• Excellent drafting skills
• Ability to work on own initiative as well as a member of a team and ability
• Professional facilitation skills preferred
• Understanding of project evaluation is an advantage