Terms of Reference - Synthesis Review of Main Findings, Recommendations, Lessons Learned, and Good Practices from Evaluations conducted in the ILO Arab States Region

Introduction and context

1. Decentralized evaluations are conducted for ILO projects, based on the ILO Evaluation Policy and its guidelines. Decentralized evaluations may include thematic evaluations, project evaluations (internal or independent), RBSA evaluations, impact evaluations, and joint evaluations, as well as other forms of internal evaluations such as self-evaluations. The type of project evaluation is determined by a pre-set criteria including the project’s timeline and budget.
2. The main objectives of these decentralized evaluations are to 1) ensure accountability and 2) extract valuable insights that can improve the organization's effectiveness. These evaluations yield various outcomes, including recommendations, lessons learned, and good practices. By gathering these insights, the evaluations aim to provide a well of knowledge that informs strategic planning, programming, and project implementation.
3. The custodian of the ILO Evaluation Policy is the ILO Evaluation Office (EVAL), which takes charge of a publicly accessible online platform known as "ieval discovery[2]." This platform serves as a comprehensive repository for evaluations, demonstrating a strong commitment to transparency. It contains recommendations from independent evaluations alongside corresponding management responses. Additionally, the repository provides direct access to lessons learned and good practices identified through these evaluations.
4. In 2022, the ILO Regional Office for Arab States, which covers 12 states, assessed the extent to which evaluations have been used in the region. The results highlighted a critical need to enhance the usefulness of evaluations. One of the key recommendations from this assessment was to conduct informative synthesis reviews as needed. As the 2022-23 biennium is ending and to inform the planning of the 2024-25 biennium, a synthesis review of all evaluations conducted in the region during the last three biennia (2018-23) will be commissioned in early November to finalize in late January.

Purpose

• The objective of this synthesis review is to compile, analyze, and synthesize the findings, recommendations, lessons learned, and good practices derived from evaluations conducted in the Arab region. The review aims to extract key insights and identify overarching themes, patterns, and actionable recommendations to enhance the ILO's future programs and initiatives in the region. Its ultimate goal is to create a repository of informative guidance that can enhance the effectiveness of planning and programming efforts.

Scope and Selection of reports for the SR

6. This synthesis review is expected to base upon all extracted evaluations (approximately 45 reports) conducted in the Arab region during the last three biennia (from 2018 to 2023). However, some reports can be excluded based on:
7. Quality of the evaluation report: in order to achieve robust and reliable results, the quality of evaluations should be assessed on the grounds of the reports being comprehensive, evidence based and providing information relevant to the synthesis review questions.
   1. For the specified period a rolling quality appraisal of most evaluation reports was done by an independent quality review team. EVAL will make this database available to the consultants to aid the inclusion/exclusion of reports. This would mean that the consultants would use the QA database to match the evaluations featuring in the long list for their quality ratings. The final list should also indicate how the reports were rated.  
8. Alongside the qualitative analysis of the information contained in the evaluation reports, the reviewer will also rate the projects overall achievements in the selected evaluations along the key evaluation criteria. The rating will be based on the findings and evidence provided in the selected evaluation reports. The rating will follow the Six-point scale used by EVAL (see attached table).

Methodology

The synthesis review will employ a systematic and rigorous approach, encompassing the following steps:

Phase I

• Extraction: The consultant will refer to I-eval discovery and extract all the evaluation reports from ROAS during the designated time. The initial search shows 45 evaluation reports. By the time the consultancy starts, 1 or 2 reports may be added.
• Desk Review: Thoroughly review all extracted evaluation reports, including executive summaries, findings, recommendations, good practices, and lessons learned. The management responses for independent evaluations will be also reviewed. Quality rate the reports to clearly identify those that will be excluded. An inception report is expected at this stage.

Phase II

• Categorization and Analysis: Categorize the extracted information according to 1) ILO’s areas of work in the region (International Labour Standards (ILS) and Labour Law, Occupational Safety and Health (OSH), migration, labour statistics, employment, social protection, crisis and refugee response, skills, enterprise development, Humanitarian Development and Peace (HDP) nexus), 2) ILO’s cross-cutting issues such as gender, social dialogue, disability and non-discrimination, environmental sustainability, etc. and 3) countries concerned with clear linkages to CPOs, 4) other categories depending on the type of information. Analyze the information to identify commonalities, gaps, and trends.

Phase III

• Synthesizing and Reporting: Synthesize the information into two sections under each of the above categories: 1) the current situation and 2) the needed actions. Prepare a comprehensive synthesis report in the ILO style, adhering to established guidelines and ensuring readability, coherence, and clarity. The report should be easily used to feed into different planning exercises of future work.

Key Questions

• Drawing from existing evaluation reports, the review should help answering two main questions: 1) What is the current situation under each of the above-mentioned categories?
• What are the needed action points and recommendations to enhance ILO’s interventions and work in the region?

Some sub questions to guide the synthesis review:

1. Under each category:
   1. What are the main interventions that can be mapped from the different evaluation reports?
   2. What are the main achievements, lessons learned and good practices?
   3. What are the main gaps in interventions?
   4. What are the trends over the years, has the focus been intensified or has it been shifted to other categories?
   5. What are the main conclusions?
2. Under each category:
   1. What are the main recommendations from the different evaluation reports?
   2. What are the trends and how these recommendations have changed/ not changed over the years?
   3. What has been the management response to those of independent evaluations?
   4. What are the synthesized recommendations for the next biennium (based on the recommendations from the evaluation reports and the synthesis of information under this review)? What are the necessary steps to address these recommendations?

• These sub questions should help guide a comprehensive assessment of the current situation and the necessary actions for enhancing ILO's work in the region across the specified categories. It should be noted that the recommendations should comply with EVAL quality checklist to ensure they are clear and actionable.

Timeline

The consultancy is anticipated to start in November with a kick off meeting and end in January.

Deliverables

The consultant is expected to prepare an inception report and a final report. Additionally, a debrief meeting with the Evaluation Manager (EM), on the main takeaways from each phase discussing challenges and the planned way forward is expected at the end of the first two phases. The draft report prepared at the end of phase 3 will be circulated for comments and the consultant should expect some new requests for extra layers of analysis before the final report. Finally, the consultant will prepare a final presentation which content can be used to prepare communication material for dissemination of findings.

Deliverable 1: Inception Report:

Deliverable 2: Draft Report

Deliverable 3: Draft 2 with a comments log explaining how each comment has been addressed.

Deliverable 4: Final Report with a comments log explaining how each comment has been addressed.

Deliverable 5: PowerPoint presentation of the final findings.

• Inception Report: the inception report/methodological outline will include a detailed methodology, the key questions to be addressed; the search strategy and a protocol for the review; possible inclusion/exclusion criteria for the selection of report; the rating of the reports and summary of the overall scores based on the QA and the rating, the coding strategy; the means to address risk of bias and quality control. The approach taken should take into account the specificities of synthesis reviews of international development interventions. EVAL will review and comment on the inception report, and the consultant will respond in writing to all received comments. The inception report must be approved by EVAL before work can proceed.
• Final report (15-25 pages excluding the annex): final report structure should be as follows,
• Title page
• Table of Contents, including List of Appendices, Tables
• List of Acronyms or Abbreviations
• Executive Summary with key findings and recommendations
• Background
• Purpose
• Methodology
• Findings arranged under each category by section.
• Recommendations
• Annexes

Payment Term (please check the attached ToR)

All deliverables must be approved by ILO ROAS M&E Officer.

Legal and Ethical Matters

• This evaluation will comply with ILO evaluation guidelines and UN Norms and Standards.
• The ToR is accompanied by the code of conduct for carrying out the evaluation “Code of conduct for evaluation in the ILO”6. The selected consultant will sign the Code of Conduct form along with the contract.
• UNEG ethical guidelines will be followed throughout the evaluation.

Qualifications

The incumbent is expected to have the following qualifications:

• Holds an advanced degree in a relevant field.
• Proven experience in conducting research and synthesizing complex qualitative data.
• Proven experience in the evaluation of development interventions.
• Previous experience in evaluations for UN agencies is strongly preferred, particularly ILO.
• Understanding of the ILO’s projects and RBM system. Prior experience in the region is an asset.
• Good understanding and experience in application of UN Evaluation Group norms and standards.
• Full command of English with excellent reporting skills.