Call for Application for Final Independent Project Evaluation for “Supporting Ministry of Labour and Social Development in analysis, policy and capacity development” in Kingdom of Saudi Arabia

Background

1. The Kingdom of Saudi Arabia (KSA), guided by the National Transformation Programme (NTP), has been implementing KSA Vision 2030. The Vision aims at:

• diversifying sources of national revenue to build immunity against potential drops in oil prices;
• lowering the rate of unemployment from 11.6 per cent to 7.0 per cent;
• increasing the economic contribution of the private sector from 40 per cent to 65 per cent of GDP;
• increasing the contribution of small and medium-sized enterprises (SMEs) to the economy from 20 per cent to 35 per cent of GDP; and
• Increasing women’s participation in the labour force from 22 per cent to 30 per cent, especially in managerial and leadership positions.

2. Yes, the Saudi government had been facing multiple challenges, including a lack of competitive and fulfilling jobs in the private sector, attractive to Saudi nationals and, thus, an overreliance on expatriate workers in this sector, youth unemployment, inefficient job matching, and low levels of women’s participation in the labour force.

3. In his speech to the International Labour Conference (ILC) in June 2017, the Minister of Labour and Social Development identified the goal of environmental sustainability as part of KSA Vision 2030. The country is transitioning to a green economy, which will develop the renewable energy sector and generate several thousand jobs. The Minister emphasized that the initiatives of the 2030 vision are aligned to a number of the agreed recommendations of the 2017 G20 summit. These include:

• providing decent work opportunities in the private sector, and designing workplaces suitable for Saudi women, making the work environment free of discrimination for them  in order to increase their participation in the labour force;
• preventing and reducing social issues affecting the most vulnerable groups of people (people with disabilities (PWD), divorcees, etc.) through social welfare, thereby encouraging their active participation in the labour market;
• encouraging innovation, increasing the number of SMEs and the participation of productive families in the economy, strengthening social dialogue and industrial relations, and improving the work conditions of expatriates.

4. The first ILO programme of assistance for the KSA was developed in 2011. ILO interventions during the period 2011–2016 focussed on the provision of technical advisory services on employment policy, labour administration, international labour standards, social security, labour inspection and occupational safety and health (LI and OSH), gender issues, social dialogue, skills development, SME development, occupational classification, and labour statistics.

Projects Background

5. This project was developed, following a review of the previous ILO programme, and a meeting between the KSA Minister of Human Resources and Social Development, known previously as Minister of Labour and Social Development, and the ILO Director-General in Geneva during the 106th session of the ILC. Furthermore, it is based on consultations between the ILO and the Ministry of Human Resources and Social Development (MHRSD) in Saudi Arabia during an ILO/ROAS mission to Riyadh from 24–27 July 2017.  6. The ILO assistance was intended to develop the capacity of the MHRSD and its staff in various directorates to formulate and manage national development strategies, and of the social partners where applicable, and to ensure sustainability and effective implementation of the conclusions and recommendations emanating from this proposal’s deliverables.  The three components agreed between the ILO and the MHRSD are:

1. boosting women’s employment and moving towards a more inclusive labour market
2. enhancing social dialogue mechanisms for better policy formulation
3. assessing the child labour situation and developing actions for its elimination

The project’s intended results and activities can be found in the Annex.

7. The main partners and stakeholders of the project have been the Government of KSA represented by the MHRSD, along with the Employers’ representatives and Workers’ committees’ representatives.  These tripartite constituents also represent the project’s target groups, the latter including in specific the Ministry’s Agency of Labour Policies, the elected workers’ committees and the National Committee for Workers’ Committees, the Chambers of Commerce, and the Council of Saudi Chambers. The project was managed by the Senior Employment Policy Specialist at the ILO Regional Office for Arab States with technical support provided by the ILO DWT specialists in Beirut, as well as other specialists from Headquarters, and in partnership with the Ministry of Human Resources and Social Development.

8. Direct beneficiaries of this project have been members of the inter-ministerial committee mandated to formulate the National Policy for the elimination of child labour with special focus on its worst forms. The committee includes representatives of the Ministry of Interior, Ministry of Justice, Ministry of Civil Service, Technical and vocation training Corporation, General corporation for Social Security, and the National Committee for Childhood, the Family Affairs Council whose one of its commitments is to combat child labour, as well as the Chamber of Commerce, and the National Committee for Workers’ committee.

 

9. Social Dialogue beneficiaries are ILO tripartite constituents, in addition to the King Abdul Aziz Centre for National Dialogue. 

10. The project supports national initiatives and priorities as outlined in the NTP, related youth employment strategies and policies, Vision 2030, the Ministry of Human Resources and Social Development vision, and the United Nations Common Country Strategic Framework 2017-2021. The project contributes to Sustainable Development Goals 1, 5, 8 and 16. Moreover, it contributes to Outcome 1. More and better jobs for inclusive growth and improved youth employment prospect and Outcome 5. Decent work in the rural economy of the ILO’s Programme & Budget 2018-2019 and Outcome 3. Economic, social and environmental transitions for full, productive, and freely chosen employment and decent work for all of the Programme & Budget 2020-2021.

11. The project duration was initially 24 months. Yet, the COVID-19 pandemic disrupted the project implementation and resulted in project extension twice. In 2020, ILO carried out a midterm self-evaluation to reflect findings and recommendations in the remaining project period.

Evaluation Background

12. ILO considers evaluation as an integral part of the implementation of development cooperation activities. Provisions are made in all projects in accordance with ILO evaluation policy and based on the nature of the project and the specific requirements agreed upon at the time of the project design and during the project as per established procedures.

13. The project document states that a final evaluation will be conducted, which will be used to assess the progress towards the results, identify the main difficulties/constraints, assess the impact of the programme for the targeted populations, and formulate lessons learned and practical recommendations to improve future similar programmes.

Evaluation Purpose and objectives

14. The final evaluation will be conducted to examine the relevance, coherence, effectiveness, efficiency, sustainability, and potential impact of the project. It will provide recommendations for the future phase of the project and/or future similar projects. This evaluation will also identify strengths and weaknesses in the project design, strategy, and implementation as well as lessons learned and good practices.

15. Specifically, the evaluation will examine the following aspects: 

• Changes in context and review of assumptions (relevance):  Is the projects’ design adequate to address the problems at hand? Were the project objectives and design relevant given the political, economic, and financial context?
• Results in terms of outcomes and outputs achieved (effectiveness): How have the projects contributed towards projects’ goals? To what extent did they contribute to the ILO’s Programme & Budget, Country Programme Outcomes, and more largely SDGs?
• Use of resources in achievement of projected performance (efficiency): How have the resources been used to fulfil the project performance in an efficient manner with respect to cost, time and management staff?
• Assessment of impact (impact): To what extent has the project contributed long-term intended impact? 
• Sustainability: Will the project’s effects remain over time?  To what extent have the projects contributed the sustainable capacity of the constituents?

16. The evaluation will comply with ILO evaluation policy, and the UNEG ethical guidelines will be followed.

Scope of Evaluation

17. The evaluation will look at the project activities, outputs and outcomes to date within the wider context of the country. The geographical coverage is to be across the country, aligned with the scope of the projects. The evaluation should take into consideration the overall project duration (May 2018 – December 2021). As cross-cutting themes, the evaluation will also take specific note of integration of gender mainstreaming, disability inclusion, International Labour Standard, social dialogue, and environmental sustainability as well as contribution to SDGs and COVID-19 response.

Clients of Evaluation

18. The primary clients of this evaluation are KSA tripartite constituents, including government entities, and ILO ROAS. Secondary users include other project stakeholders and units within the ILO that may indirectly benefit from the knowledge generated by the evaluation.

 

Evaluation Criteria and Questions

19. The evaluation utilizes the standard ILO evaluation framework and follows the OECD/DAC evaluation criteria:

Relevance and strategic fit

• Are the project objectives aligned with sectoral national priorities and tripartite constituents’ needs? What measures have been taken to ensure alignment? How does the Project deal with shortcomings of tripartism characteristic of the country?
• How does the project contribute to the ILO’s Programme & Budget objectives, Country Programme Outcomes, and SDGs?

Coherence and validity of the design

• Are the project strategies and structures coherent and logical?
• Does the project make a practical use of a monitoring and evaluation framework? How appropriate and useful are the indicators in assessing the projects’ progress? Are indicators gender sensitive? Are the means of verification for the indicators appropriate? Are the assumptions for each objective and output realistic?
• To what extent did the project design take into account: Specific gender equality and non-discrimination concerns, including inclusion of people with disabilities, relevant to the project context as well as International Labour Standards and Social Dialogue?
• How well does the project design take into account existing local efforts to address issues concerned with women’s employment, social dialogue and child labour? Does the projects’ design fill an existing gap that other ongoing interventions have failed to address?

Project progress and effectiveness

• What progress has the project made towards achieving the overall objectives and outcomes?
• How did outputs and outcomes contribute to ILO’s mainstreamed strategies including gender equality, social dialogue, poverty reduction and labour standards?
• To what extent did the projects contribute to the intended results of ILO Programme & Budget 2018-19 and 2020-21?
• To what extent did the projects respond emerging needs in terms of COVID-19 pandemic? Did the pandemic hinder or reverse the progresses that had been made?

Efficiency of resource use

• To what extent have project activities been cost-efficient? Have resources (funds, human resources, time, expertise etc.) been allocated strategically to achieve outcomes? To what extent can the project results justify the time, financial and human resources invested in the project?
• To what extent have the projects been able to build on other ILO or non-ILO initiatives either nationally or regionally, in particular with regard to the creation of synergies in cost sharing?
• What was the role of the projects in resource mobilization? Given the country’s context, what can ILO do differently in resource mobilization?

 

Effectiveness of management arrangements

• What was the division of work tasks within the project teams and partners? How does the project governance structure facilitate good results and efficient delivery? And if not, why not?
• How effective was communication between the project teams, the regional office and the responsible technical department at headquarters? Have the projects received adequate technical and administrative support/response from the ILO backstopping units?

Impact orientation

• What is the likely contribution of the project initiatives to the intended impact of the intervention?
• What were the interventions long-term effects on more equitable gender relations or reinforcement of existing inequalities?
• How did the project contribute to the realisation of International Labour Standards in KSA?

Sustainability

• Are the results achieved by the project likely to be sustainable? What measures have been considered to ensure that the key components of the project are sustainable beyond the life of the projects?
• To what extent was sustainability of impact taken into account during the design of the project?
• How effectively has the project built national ownership and capacity? In what ways are results anchored in national institutions and to what extent can the local partners maintain them financially at end of project?

Methodology

20. This cluster evaluation is summative and relies on both quantitative and qualitative approaches to respond evaluation questions and fulfil the purpose. It consists of,

• Desk review of existing documents: The evaluator will conduct systematic analysis of existing documents and obtain existing qualitative and quantitative evidence prior to primary data collection. The desk review also facilitate assessment of the situation and available data to plan the evaluation and develop the inception report.
• Key information interviews: Online individual interviews will be conducted with a pre-agreed list of stakeholders who have in-depth exposure and understanding of the projects and their context. Interview guide(s) will be developed during the inception phase to stimulate a discussion on concerned evaluation questions.
• Preliminary finding briefing: Upon completion of primary data collection, the evaluator will present preliminary findings to ILOs and selected stakeholders for validation. The evaluator will also collect further insight from the group to feed them into the final report.

21. Any changes to the methodology should be discussed with and approved by the Regional Evaluation Officer during the inception phase.

Work Assignments

a. Kick-off meeting

22. The evaluator will have an initial consultation with the REO, relevant ILO specialists and support staff in ROAS. The objective of the consultation is to reach a common understanding regarding the status of the project, the priority assessment questions, available data sources and data collection instruments and an outline of the final assessment report. The following topics will be covered: status of logistical arrangements, project background and materials, key evaluation questions and priorities, outline of the inception and final report.

b. Desk Review

23. The evaluator will review project background materials before conducting interviews. Documents to review include, but are not limited to, KSA Vision 2030, National Transformation Programme, ILO Programme and Budget, United Nations Common Country Strategic Framework 2017-2021, project proposal including the results framework, project progress reports, mid-term self-evaluation, and workshop reports.

c. Inception Report

24. The evaluator will draft an Inception Report, which should describe, provide reflection and fine-tuning of the following issues:

• Project background
• Purpose, scope and beneficiaries of the evaluation
• Evaluation matrix, including criteria, questions, indicators, data source, and data collection methods  
• Methodology and instruments
• Main deliverables
• Management arrangements and work plan

d. Primary Data Collection (Key Informant Interviews)

25. Following the inception report, the evaluator will have virtual meetings with constituents/ stakeholders together with an interpreter/enumerator supporting the process. Individual or group interviews will be conducted with the following:

1. Project staff/consultants that have been actively involved;
2. ILO ROAS DWT Director, RPU, and Senior Specialists in Employment and Gender, Workers and Employers’ specialists and HQ backstopping specialists;
3. Interviews with tripartite constituents in KSA (government, public institutions, social partners,  etc.);
4. Interviews with direct and indirect beneficiaries;

e. Preliminary finding presentation

26. Upon completion of data collection, the evaluator will provide a briefing of preliminary findings to the Project teams, ILO DWT, and ROAS to validate findings.

f. Final Report

27. The final report will follow the format below and be in a range of 35-40 pages in length, excluding the annexes:

1. Title page
2. Table of Contents, including List of Appendices, Tables
3. List of Acronyms or Abbreviations
4. Executive Summary with key findings, conclusions and recommendations
5. Background and Project Description
6. Purpose of Evaluation
7. Evaluation Methodology and Evaluation Questions
8. Key evaluation findings (organized by evaluation criteria)
9. A table presenting the key results (i.e. figures and qualitative results) achieved per objective (expected and unexpected)
10. Clearly identified conclusions and recommendations (identifying which stakeholders are responsible and the time and resource implications of the recommendations)
11. Lessons Learned (in prescribed template)
12. Potential good practices (in prescribed template)
13. Annexes (list of interviews, TORs, list of documents consulted, good practices and lessons learned in the ILO format, etc.)

28. The quality of the report will be assessed against the ILO Evaluation Office (EVAL) Checklists 4.2, 4.3, 4.4. The deliverables will be submitted in the English language and structured according to the templates provided by the ILO. 

g. Final presentation

29. Present finding to stakeholders and discuss the way forward.

Evaluation Timeframe

30. The evaluation is to commence in November 2021 and complete in February 2022. The table in the attached docuemnt describe the tentative timeline.

Total estimated working days of consultant: 28 Days

 

Implications of the COVID crisis on the evaluation

32. The current COVID-19 pandemic severely restricts the mobility of staff and consultants. Based on the matrix developed by the ILO EVAL on the constraints and risks as measured against the criticality of the evaluation to the ILO, the evaluator will conduct this evaluation remotely relying on online methods such online surveys, telephone or online interviews, whereas for some country components it will be feasible to use a hybrid face to face/remote approach for collecting data.

33. When and where relevant, evaluation questions will also be guided by the ILO protocol on collecting evaluative evidence on the ILO’s Covid-19 response measure through project and programme evaluations, available at: https://www.ilo.org/wcmsp5/groups/public/---ed_mas/---eval/documents/publication/wcms_757541.pdf

The evaluation manager may propose alternative methodologies to address the data collection that will be reflected in the inception phase of the evaluation developed by the evaluation team. These will be discussed and require detail development in the Inception report and then must be approved from the evaluation manager.

 

Deliverable

34. The main outputs of the evaluation consist of the following:

• Deliverable 1: Inception Report
• Deliverable 2: PowerPoint Presentation on preliminary findings
• Deliverable 3: Draft evaluation report
• Deliverable 4: Final evaluation report with separate template for executive summary and templates for lessons learned and good practices duly filled in (as per ILO’s standard procedure, the report will be considered final after quality review by EVAL)
• Deliverable 5: Final PowerPoint presentation to stakeholders

Payment Term

1. 10 per cent of the total fee against deliverable 1 above approved by the evaluation manager
2. 30 per cent of the total fee against deliverable 2 and 3 above
3. 60 per cent of the total fee against deliverable 4 and 5 above approved by the evaluation manager and EVAL.

Management Arrangement

35. The evaluator will report to the ILO Regional Evaluation Officer in ROAS and should discuss any technical and methodological matters with the REO. The ILO ROAS office will provide administrative and logistical support during the data collection. The ILO ROAS office will coordinate with ILO Evaluation Office in HQ throughout the evaluation process. ILO EVAL approves and signs off on the final evaluation report.

36. The External Evaluator is responsible for conducting the evaluation according to the terms of reference (ToR). He/she will:

• Review the ToR and provide input, propose any refinements to assessment questions, as necessary, during the inception phase;
• Review project background materials (e.g. project document, progress reports).
• Prepare an inception report;
• Develop and implement the evaluation methodology (i.e. conduct interviews, review documents) to answer the evaluation questions;
• Conduct preparatory consultations with the ILO Regional Evaluation Officer prior to the evaluation mission.
• Conduct field research, interviews, as appropriate, and collect information according to the suggested format;
• Present preliminary findings to the constituents; 
• Prepare an initial draft of the evaluation report with input from ILO specialists and constituents/stakeholders;
• Conduct a briefing on the findings, conclusions and recommendation of the evaluation to ILO ROAS;
• Prepare the final report based on the ILO, donor and constituents’ feedback obtained on the draft report.

37. The ILO Evaluation Manager is responsible for:

• Drafting the ToR;
• Finalizing the ToR with input from colleagues;
• Preparing a short list of candidates for submission to the Regional Evaluation Officer, ILO/ROAS and EVAL for final selection;
• Hiring the consultant;
• Providing the consultant with the project background materials;
• Participating in preparatory consultations (briefing) prior to the assessment mission;
• Assisting in the implementation of the assessment methodology, as appropriate (i.e., participate in meetings, review documents);
• Reviewing the inception report, initial draft report, circulating it for comments and providing consolidated feedback to the External Evaluators (for the inception report and the final report);
• Reviewing the final draft of the report, and executive summary;
• Disseminating the final report to all the stakeholders;
• Coordinating follow-up as necessary.

38. The ILO REO:

• Provides support to the planning of the evaluation;
• Approves selection of the evaluation consultant and final versions of the TOR;
• Reviews the draft and final evaluation report and submits it to EVAL;
• Disseminates the report as appropriate.

39. The Project Coordinators are responsible for:

• Reviewing the draft TOR and providing input, as necessary;
• Providing project background materials, including studies, analytical papers, reports, tools, publications produced, and any relevant background notes;
• Providing a list of stakeholders;
• Participating in the preparatory briefing prior to the assessment missions;
• Scheduling all meetings and interviews for the missions;
• Ensuring necessary logistical arrangements for the missions;
• Reviewing and providing comments on the initial draft report;
• Participating in the debriefing on the findings, conclusions, and recommendations;
• Providing translation for any required documents: ToR, PPP, final report, etc.;
• Making sure appropriate follow-up action is taken.

Legal and Ethical Matters

• This evaluation will comply with ILO evaluation guidelines and UN Norms and Standards.
• The ToRs is accompanied by the code of conduct for carrying out the evaluation “Code of conduct for evaluation in the ILO”. The selected consultant will sign the Code of Conduct form along with the contract.
• UNEG ethical guidelines will be followed throughout the evaluation.
• The consultant will not have any links to project management or any other conflict of interest that would interfere with the independence of the evaluation.

Qualification

40. The evaluator is expected to have following qualifications,

• Proven experience in the evaluation of development interventions
• Expertise in employment issues and an understanding of the ILO’s tripartite culture. Prior experience in the region, particularly in KSA, is asset.
• High professional standards and principles of integrity in accordance with ILO Evaluation Policy and United Nations Evaluation Group Norms and Standards.
• An advanced degree in a relevant field.
• Proven expertise on evaluation methods and the ILO approach.
• Full command of English. Command of Arabic is an advantage.
• The consultant should not have any links to project management or any other conflict of interest that would interfere with the independence of the evaluation.
• Previous experience in evaluations for UN agencies is preferred, particularly ILO.

41. Give the travel restriction due to the COVID-19 pandemic, the consultant who implement this evaluation remotely may work with a national interpreter/enumerator, who will provide necessary support for data collection.