Final Evaluation for BPRM funded Value chains for improved Socioeconomic well-being of Syrian Refugees and Lebanese Host Communities in Lebanon project

Qualifications of the Consultant

• Master’s Degree in Agriculture fields, International Development, Humanitarian Action, or related fields with relevant technical knowledge in M&E, research methods, etc.

• 4 to 7 years’ experience in designing, managing and leading assessments and/or evaluation

• Experience of applying evaluation methodologies such as UNEG, OECD-DAC
• Experience of conducting data collection and analysis, including participatory methodologies
• Experience of evaluation of project implementation strategies, budget management, HR and management, and making context appropriate actionable recommendations
• Proficiency in  English is mandatory, Arabic is preferable
• Experience with US government grants is an advantage
• Experience of working in Lebanon is also an advantage

Application

Applications should include the following documents:

• Cover letter;
• CV(s) of the evaluator(s) with details of qualifications, experience, telephone number and names of three referees for similar work conducted;
• Technical proposal that summarizes your understanding of the ToR and the proposed methodology, including the foreseen work plan for the assignment;
• Financial proposal, with costs in US dollars
• Interested Applicants should send the above-requested information to Lebanon.hr@concern.net

TERMS OF REFERENCE

Purpose of the Evaluation

The external evaluation will assess the project’s contribution to improved skills, capacities and access to income generating activities, towards economic and psychosocial wellbeing; for project learning and accountability. The evaluation will use the extended DAC criteria including: project relevance, effectiveness, coverage, efficiency, impact, connectedness and coherence, to assess the degree to which the project has achieved its intended outcomes by the end of the implementation period September 2020 to September 2022.

Description of the Social, Economic and Political context

Ten years into the Syria conflict, Lebanon remains at the forefront of one of the worst humanitarian crises. The Government of Lebanon estimates that the country hosts 1.5 million[1] of the 6.6 million[2] Syrians who have fled the conflict since 2011 (including 855,172 registered with UNHCR as of end of March 2021)[3]. The Syrian refugee population in Lebanon remains one of the largest concentrations of refugees per capita in the world. In addition, Lebanon is facing its third year of a dramatic economic crisis, political paralysis, heightened social tensions, and the ongoing negative effects of COVID-19 and the 2020 Beirut port explosion. The human impact of these crises is sobering. In 2021, 2.1 million Lebanese and 1.3 million Syrian refugees suffered from high levels of poverty and food insecurity, requiring assistance, and/or increasingly resorting to negative coping mechanisms. Government-led interventions have proven unable to address the root causes to mitigate the impact of the ongoing crisis.

Lebanon was without a fully empowered government for over a year since the catastrophic August 4, 2020 explosion at Beirut port, which forced the resignation of then Prime Minister Hassan Diab’s government. Rival political groups were locked in disagreement over the make-up of a new government for months, hastening the country’s economic meltdown. Despite demands from the international community for Lebanon to form a technocratic government of experts, the new cabinet is still balanced along sectarian lines, with 12 Christians and 12 Muslims. The new Cabinet of 24 ministers headed by billionaire businessman Najib Mikati was announced by the president's office, and later by the secretary-general of the Council of Ministers, Mahmoud Makkieh. Ministers were handpicked by the politicians who have ruled the country for decades and several new ministers with little background to suggest they will be successful in their new positions.

Food prices increased at a staggering 400 per cent between January and December 2020. Revised food survival and minimum expenditure basket (SMEB) recorded a 21 per cent increase between March and April 2021. The overall cost is 4 times higher than at the start of the crisis. There has also been a 27% decrease in food imports through the Port of Beirut when comparing June 2021 figures to those of June 2020. This was mainly due to the decreases in the import of edible vegetables (41% decrease), sugar and confectionary (26% decrease), cereals (20% decrease), and live animals (17% decrease). By the end of June 2021, only 12% of WFP contracted shops reported having items from the subsidized food basket (at the rate of LBP 3,900), down from 79% at the beginning of March 2021.[4] Overall, the percentage of households below the SMEB level maintained the same high level as in 2020, and significantly higher than 2019 levels. Akkar, Bekaa, and Baalbek-El Hermel reported the highest share of households below the SMEB at 94%, indicating that these regions host the highest proportions of socio-economically vulnerable households. Women working in the agricultural sector continue to face wage and labour discrimination, limited resources to advance, limited training and working opportunities, lack of access to finance, and limited decision-making power. According to a study published by International Labor Organisation (ILO) entitled: Potential Opportunities for Women's Economic Empowerment - Potato and Leafy Green Vegetable Value Chains - Akkar, Northern Lebanon, findings from the stakeholder analysis indicated that women agricultural workers do not interact with input supplier and agricultural service providers and are always dependent on men for the purchase of agricultural inputs.

The overall security environment is deteriorating with an increasing number of localized incidents, such as theft, reported in the first seven months of 2021 (5,276 security incidents). These are primarily related to intra-Lebanese tensions (political and sectarian conflict, class and economic concerns) but are increasingly manifesting into tensions between communities accessing services (VASyR, 2021 and UNDP, 2019)[5]. According to IRC Protection Monitoring report Syrian refugees reported facing discrimination and verbal harassment. For example, in Akkar, one Syrian man was told he would only be able to fill his motorcycle once all the Lebanese in the queue had received fuel first. This demonstrates how scarcity of resources, and perceived competition, can be a flashpoint for both intra- and inter-communal tensions (IRC, 2021). Both refugee and host communities need to improve their knowledge and skills to enable them to prevent and respond to protection issues within their workplace, concerns including sexual and gender-based violence (SGBV), early marriage, child labor and violence within and between communities.

Description of the Subject of the Evaluation

Title

Value chains for improved socioeconomic well-being of Syrian refugees and Lebanese host communities

Goal

Syrian refugees and vulnerable host communities have skills, capacities, and greater access to income generating activities in a protective environment that supports economic and psychosocial well-being

Objectives

1. Farmers and producers adopt good practices to increase dairy and aromatic herb production that meet high quality standards and are supported by the creation of market linkages.
2. Enhanced psychosocial well-being and protection for all program participants.

Indicators

1. 1. % of households with an adequate food consumption
   2. % Syrian laborers who retain employment with Lebanese herb and dairy farmers 6 months following the conclusion of CFW activities 
   3. # and % program participants, who self-report increased income by end of program period as compared to the pre-program baseline assessment
   4. # individuals trained on sustainable agricultural production (herbs)
   5. # individuals with improved capacity on sustainable agricultural production
   6. # farmers attending training on sustainable livestock production
   7. # farmers with improved capacity on sustainable livestock production
   8. # individuals provided with agriculture inputs
   9. # individuals providing temporary/casual agricultural labour
   10. # farmers business school established
   11. # farmers and producers participating in market linkage events
   12. # farmer and producer linked with relevant business to their value chain
   13. # cooperatives supported through an in-kind or cash grant 
   14. # MSMEs formally registered 

1. 1. % beneficiaries who report an improved sense of safety and well-being
   2. # beneficiaries reporting an increase in decent work conditions
   3. # individuals sensitized on decent work, minimum standards, and labour laws and regulations
   4. # women and men sensitized on SGBV
   5. # persons reached through awareness sessions on legal topics.
   6. # children receiving childcare services

Target

1,320 (520 for Dairy and 800 for Herbs) in Akkar and Dannieh. 61% refugees 57% women, 50% youth (aged 18 to 25), and 7% people living with disabilities and/or their caretakers.

Duration

September 30, 2020 – September 29, 2022

Scope of the Evaluation

To conduct the final evaluation of the project using the extended DAC[6] criteria of development evaluation, i.e. Relevance, Coverage, Connectedness, Coherence, Efficiency, Effectiveness, and Impact, including capturing the extent to which the cross-cutting issues of equality, protection, conflict sensitivity, gender and advocacy were taken into account at project design, planning and implementation stages. The evaluation will provide an independent objective opinion on project performance, and opportunities for improvement in future programming for Concern and the partnership with Environment and Sustainable Development Unit (ESDU).

The evaluator(s) will develop a detailed methodology and work plan that allows for application of the DAC criteria for assessing the project performance against the indicators, including desk review of project documents and baseline and end-line assessments, focus group discussions and key informant interviews, as required; with M&E team supporting on the ground, if needed. Through the tri-angulation of quantitative and qualitative data, project outcomes and consultations with stakeholders (internal and external), the evaluators will provide learnings and practical recommendations.

Objectives of the Evaluation

• Assess and score the project performance in terms of effectiveness, impact, coverage and efficiency both in financial and other resources and against project indicators
• Assess and score the relevance and appropriateness of the project and beneficiary selection criteria, to the social and political context and economic crisis in Lebanon 
• Analyse and score the project design and implementation, against the goal of improved economic and social well-being for Syrians and Lebanese in northern Lebanon
• Assess and score the connectedness and coherence of the project, and adherence to accountability to affected populations during implementation, for both Concern and ESDU
• Capture lessons learned and make practical recommendations for future programming, for both direct implementation and implementation through partners as with ESDU

Research questions:

Scoring

5

Outstanding Performance

4

Very good overall performance with few shortcomings

3

Good overall performance but with some minor shortcomings

2

Generally acceptable performance but with some major shortcomings

1

Barely acceptable performance with many major shortcomings

0

Totally unacceptable performance or insufficient data to make an assessment

Extended DAC criteria

      Research Questions (Findings to be scored 0-5)

Relevance: to what extent was the project aligned with local needs and priorities?

• What was the level of adequacy of the situation analysis and needs assessment that informed the implementation design?
• To what extent was each project component (namely dairy value chain, herbal value chain and support to SMEs and cooperatives) and intervention design responsive to the context and needs of target groups?
• Were the approaches used such as linking Lebanese farmers and Syrian workers socially and culturally appropriate and suitable to achieve the project goal?

Efficiency: to what extent did project inputs contribute to project output?

• To what extent did the scale and scope of the project sectoral, geographical, target beneficiary and intervention design coverage provide value for money?

Effectiveness: to what extent did the project achieve its purpose?

• To what extent were adaptations made to address or prevent inequality, conflict and other potential unintended negative consequences of the interventions? 
• How well did Concern and ESDU perform against the project indicators? Identify any justifications, challenges and learning; and make recommendation for future reference
• What was the implementation timeliness, towards successful project delivery on improved economic and social well-being of vulnerable Syrians and Lebanese by the end of the project? What could have been done differently?

Coverage: Who was supported by the project, and why?

• To what extent were the villages selected for the project interventions most appropriate for the project objective of reaching the most vulnerable people?
• To what extent was the project beneficiaries’ selection criteria clear and fair, and project participants informed about the inclusion and exclusion criteria?
• What was the level of adequacy of the methodology of beneficiary selection and verification during trainings, payments, distributions and other project interventions?

Impact: What were the social, economic, technical, and environmental effects on individuals, communities and institutions?

• What mid to long term changes, including intended and unintended, positive and negative, have been achieved in the project locations as a direct result of the project?
• Were these changes the same for men, women, boys, girls, people living with disabilities and other vulnerable groups?

Connectedness: Did activities take longer-term and interconnected problems into account?

• To what extent did the project link with other interventions in the same villages towards common impact?
• To what extent did the linkage of Syrians and Lebanese contribute to reduced social tension?
• To what extent did the project interventions contribute towards self-sufficiency of vulnerable farmers, businesses, cooperatives and Syrians to survive the economic crisis, without further dependence on aid?

Coherence: Ensuring that policies take into account humanitarian and human-rights considerations.

• To what extent did the project adhere to and align with national policies and guidelines in herbal and dairy value chains, agribusiness and cooperatives?
• To what extent did the project link, improve, and contribute to the food security and livelihoods sector guidelines, priorities, and targets?
• To what extent did the project engage with the local authorities and municipalities and ensure adherence to local strategic plans in herbal and dairy value chains, agribusiness and cooperatives?

Methodology

The evaluation will be a conducted by a Consultant. Concern M&E team will only support the evaluation exercise within the execution plan prepared by evaluator(s) in the field locations.

Proposed basic steps:

• Desk review: Concern will provide the relevant project documents, and findings from project baseline and end-line assessments, and as well as market analysis conducted by ESDU.
• Data collection: The Consultant is expected to collect primary data, for largely qualitative information, to respond to research questions and the evaluation objectives not adequately covered by existing quantitative data, through focus group discussions, key informant interviews, and any other qualitative techniques as appropriate. The Consultant will lead in the development of the detailed evaluation methodology, tools and questionnaires, and collection of required data to answer the DAC criteria research questions, against the evaluation objectives. Data collection will take into account gender, protection and conflict sensitivity and other cross-cutting issues (inclusion of men, women, people living with disabilities, people from different ethnic groups etc.). All data provided or collected during the evaluation will be disaggregated by gender, age and any other relevant criteria.
• Data analysis: The Consultant will conduct data analysis, including triangulation of primary and secondary data qualitative and quantitative data provided by Concern or collected during the evaluation; in order to collate responses to the DAC criteria research questions and score the project performance accordingly. An overall score will be required for each of the 07 DAC criteria, with a summary of findings and recommendations against the score provided.
• Validation of findings: The Consultant will validated findings with Concern management, through a meeting, before submitting the final report.

Roles and responsibilities

Concern Lebanon will be responsible for:

• Approval of the final evaluation plan, tools and methodology
• Providing relevant documentation (including baseline and end line surveys) and briefing on evaluation expectations
• Supporting the Consultant in organising the focus group discussions and key informant interviews, if needed
• Organising and covering the costs of transportation between Concern offices and the field for data collection
• Provision of timely feedback to the Consultant on the findings and the report
• Dissemination of results and report of the evaluation to key stakeholders and BPRM

The Consultant will be responsible for:

• Developing the detailed methodology and evaluation plan for approval by Concern
• Conducting focus group discussions and key informant interview with beneficiaries, stakeholders, Concern staff, with support from Concern if needed
• Evaluation data compilation and analysis, and validation meeting with Concern
• Report writing including finalisation according to feedback provided
• The report will be the intellectual property of Concern. Its content should be shared with third parties only with Concern’s consent. Sensitivities to confidentiality will need to be respected. Names and identifiable data associated with any particular quotes or experiences should not be cited.

Lines of communication

The Consultant will report to the MEAL Manager and work closely with the Livelihoods Project Manager.

Timeline

Task

Timeline

No. of days

Signing of contract

11.10.2022

Briefing, desk review and preparation of tools and methodology

12-15 Oct

3

Collection of primary data

16-23 Oct

7

Analysis and compiling of preliminary findings

23-25 Oct

2

Debrief/presentation of findings to Concern for validation

25 Oct

1

Complete final report

26-31Oct

6

Total

19 days

This is a 19 calendar day project. working on weekends is a MUST.

Expected outputs and deliverables

• Survey tools: FGD guides, KII guides, Templates for DAC scoring, etc
• Presentations/workshop of the evaluation findings to Concern for validation
• A complete final report, integrating the feedback received during the validation meeting, within seven working days. The report, in English, should be 15-20 pages long without counting annexes, and should be submitted in electronic format (Word and PDF) to Concern. The report should include the following sections:
  • Executive Summary (maximum 2 pages)
  • Introduction and Project Overview
  • Evaluation methodology and limitations
  • Findings and Discussion on Evaluation Questions under each DAC Criteria
  • Conclusions and Lessons Learnt
  • Recommendations
  • Updated Results Framework/Logframe with End line values and analysis
  • Report Annexes: including ToR, list of sites and respondent categories, interview questionnaires and tools, Evaluation schedule, Photos (following Concern’s Code of Conduct on Images and Messages, including consent forms

Ethical considerations

Concern has an organisational Code of Conduct (CCoC) with three Associated Policies; the Programme Participant Protection Policy (P4), the Child Safeguarding Policy and the Anti-Trafficking in Persons Policy. These have been developed to ensure the maximum protection of programme participants from exploitation, and to clarify the responsibilities of Concern staff, consultants, visitors to the programme and partner organisation, and the standards of behaviour expected of them. In this context, staff have a responsibility to the organisation to strive for, and maintain, the highest standards in the day-to-day conduct in their workplace in accordance with Concern’s core values and mission. Any candidate offered a consultancy opportunity with Concern Worldwide will be expected to sign the Concern Code of Conduct and Associated Policies as an appendix to their consultancy contract. By signing the Concern Code of Conduct, candidates acknowledge that they have understood the content of both the Concern Code of Conduct and the Associated Policies and agree to conduct themselves in accordance with the provisions of these policies. Additionally, Concern is committed to the safeguarding and protection of vulnerable adults and children in our work. We will do everything possible to ensure that only those who are suitable to work or volunteer with vulnerable adults and children are recruited by us for such roles. Subsequently, being engaged as a consultant with Concern is subject to a range of vetting checks, including criminal background checking.

The evaluation process will involve the following ethical considerations and measures:

• The Consultant will adhere to Concern’s Code of Conduct and associated policies
• The purpose of the data collection and the use of the collected data will be explained to all respondents, with the option to opt out at any point
• The respondents will be informed that the survey will have no impact on the extent of support they receive (as to avoid raising false expectations) from Concern or other organizations
• Verbal informed consent will be requested from all participants using a consent statement included on the FGD Guides, questionnaires, observation checklists and any other tools used
• Photos will be taken only after receiving a relevant person’s signed consent form, and any minors will have the form signed by their guardian
• Interview and FGDs will be conducted in a safe environment where no other people can listen to what the respondent(s) say, and avoid interrupting participants’ daily schedule
• Respondents’ names will be collected only in the case of individual interviews with Concern and its partners’ staff; other respondents’ names will not be collected
• The Consultant will apply the principles of age, culture and gender-sensitive interviewing