Endline Evaluation-Youth With Disabilities Project

Asmae’s Background

Asmae-Association Soeur Emmanuelle ("Asmae") is a French international NGO and registered charity, specializing in child development. Independent, secular, and apolitical, it is open to all. Founded in 1980 by Sister Emmanuelle, based on her experience with rag pickers in the slums of Cairo, Asmae continues its actions in line with the values and methods inherited from its founder: listening and proximity, pragmatism, taking account of differences, professionalism, and reciprocity. Asmae's actions aim to support vulnerable children, youth and their families through personalized and continuing support for local actors working in the fields of education and child protection.

Asmae has been operating in Lebanon through local partners since 1986. The organization is currently collaborating with five Lebanese NGOs, working in and around Beirut, Naher El Bared, Mount Lebanon (Shouf), and Ain El Helwe camps, with some activities also taking place in other parts of Lebanon.

NRDC’s Background - SCALE

NRDC (National Rehabilitation & Development Center) is the main partner for this project. It has provided people with disabilities with a range of medical, social, and educational services since its inception in 1986. The Technical Institute was launched in 2018 to offer technical education in an inclusive environment. SCALE by NRDC's main areas of interventions are: Education, Healthcare, Child Protection, Social Integration, Disabilities, Training, Capacity Building, Case Management, Technical and Vocational Training, Learning Disabilities, Rehabilitation, and Therapies (Occupational Therapy, Psycho-motor Therapy, Speech Therapy, Physical Therapy, and Psycho Therapy). On the 8th of November, 2022 NRDC changed the name of the Center to SCALE (Specialize Center for Accessible Learning and Education)

The department of International Cooperation - Monaco (DCI)

The Government of the Principality of Monaco is involved in development aid through the Department for International Cooperation (DCI). The DCI supports around 150 projects in 11 partner countries, mainly LDCs (Least Developed Countries) and fragile states and in 4 main areas of intervention:

- Health

- Food and nutritional security

- Education and child protection

- Access to decent work

The Government of Monaco has been providing development aid to Lebanon for over 30 years through its Department of International Cooperation (DCI), focusing on four main areas: health, education and child protection, food security and access to decent work for the most vulnerable. Particular attention is paid to the inclusion of people with disabilities.

Project location:

Country: Lebanon

Town, region: Shouf Area

Project Calendar:

Project duration: 36 months

• Start date: March 2022
• Ending date: February 2025- with 2 months extension (End of April)

Exit Strategy implementation: March 2025 – June 2025 (4 months)

The project targeted  135 youth mostly aged between 14 and  25 years, of which about 40% Syrian – 50-75 % 0f them are with disabilities – and 60% Lebanese with a ratio of 50/50 male/female. 250 youths’ family members and caregivers were also be targeted. During project implementation, youths and families were encouraged to channel their feedback, share their suggestions through regular contacts with SCALE team. Besides, through the component of capacity building (SO3), SCALE team members (special educators, therapists, trainers, social workers, support staff) attained increased knowledge and skills (technical, operational and organizational). Moreover representatives of associations, companies, local authorities are also part of indirect beneficiaries. Also more than 340 youth enrolled in current programs implemented by SCALE and their parents in addition to more than 600 community members of the project areas were indirectly benefited from the project.

The overall objective: Promote socio-economic inclusion of most vulnerable youth, in particular Lebanese and Syrian refugees with disabilities in Shouf region.

Purpose of the Endline Study

The study aims to evaluate the project’s impact, effectiveness, efficiency, relevance, coherence, and sustainability. Findings will inform future programming and provide evidence-based recommendations.

Key Objectives:

• Assess the project’s contribution to socio-economic inclusion.
• Identify key lessons learned and best practices.
• Measure long-term impact on beneficiaries and partners.

Evaluation Criteria

The OECD has defined six evaluation criteria – relevance, coherence, effectiveness, efficiency, impact and sustainability – and two principles for their use. These criteria provide a normative framework used to determine the merit or worth of a development intervention (policy, strategy, project or activity). They serve as the basis upon which evaluative judgements are made. 4 out of 6 criteria were selected to focus on while evaluate this project. This is given to the project size and scope, and given the project's focus on equipping marginalized youth with market-relevant skills and enhancing their employability, these criteria were the most suitable for assessing whether the training met labor market needs and community needs (relevance), delivered tangible results (effectiveness), contributed to participants' long-term economic prospects (impact), and ensured lasting benefits beyond the project’s duration (sustainability). While coherence and efficiency remain important, they were not the primary focus of this evaluation, as the assessment aimed to concentrate on the most critical factors influencing the project's success and long-term viability

Key Evaluation Criteria selected for this project

1. Relevance:

Information to be assessed: The extent to which the project addressed the identified needs of youth and partners, ensuring inclusivity across gender, disability, and nationality dimensions with identified needs, considering inclusivity factors such as gender, disability, and nationality.

• Are the training models offered at SCALE adapted to the needs of potential employers ?
• Do the courses offered at SCALE meet the needs of employers, the career plans of young beneficiaries ?
• How well did the project address the needs of youth and partners? - Is the training content adapted to the expectations of professionals and young beneficiaries?
• Was it inclusive (gender, disability, nationality) - Are gender, disability and nationality issues well integrated

2. Sustainability:

The likelihood of the project’s benefits continuing beyond its completion, including the strategies and mechanisms in place to sustain results over the long term among SCALE.  As well as the ability of project outcomes to remain effective after completion, including strategies for long-term sustainability.

• Are the project benefits likely to continue?
• How can results be sustained beyond the project?
• What was the role of Exit Strategy in ensuring the sustainability of SCALE?
• Have the teams of SCALE (management, administration, pedagogy) developed their skills sufficiently to ensure the continuity and reinforcement of the Project's positive effects? What skill-building needs persist?
• Do existing or planned income-generating activities contribute to the financial sustainability of the training courses? What adjustments need to be made?
• What is the degree of ownership of the project by SCALE?
• Is SCALE in phase with this exit strategy? Are there any areas for improvement to ensure that the quality of training continues after ASMAE's withdrawal?

3. Impact

The socio-economic changes brought about by the project, including intended and unintended consequences, and their effects on beneficiaries. Also, the extent of positive or negative changes experienced by beneficiaries, including unintended consequences.

• What socio-economic changes did the project bring?
• Were there any unintended outcomes?
• how many young people have had an internship/job opportunity/income-generating activity and the quality of this opportunity (sustainable employment, level of remuneration, etc.)
• How were youth integrated into the workplace, and what obstacles have they encountered? Were they integrated on a long-term basis?
• What was the impact of the training provided at SCALE on the personal development of beneficiaries?
• Are there disparities between men and women? Youth with and youth without disabilities? Lebanese and Syrian Refugees? What obstacles and challenges still remain?

4. Efficiency:

The effectiveness of resource allocation, ensuring that financial and human resources were utilized optimally to achieve project objectives, with considerations for cost-efficiency improvements. Moreover, how well financial and human resources were allocated to achieve project objectives and whether improvements in cost-efficiency were possible.

• Were resources used effectively?
• Could the implementation have been more cost-efficient?

NB: the above questions are not exhaustive and may be supplemented/updated by the selected consultant. For each criterion, SMART (specific, measurable, acceptable, realistic and time-bound) actions and recommendations could be proposed.

Methodology

The endline study will adopt a mixed-methods approach combining both qualitative and quantitative research techniques to ensure a comprehensive assessment. The methodology will include a desk review of project documents and reports to provide context and background analysis. Additionally, key informant interviews (KIIs) will be conducted with beneficiaries, project staff, and relevant stakeholders (Business owners, employers, parents…etc) to gain insights into the project’s implementation and impact. Focus group discussions (FGDs) will be organized with youth, parents, and project partners to collect qualitative data on personal experiences and perceptions. Surveys will be carried out to quantify key indicators and measure project effectiveness. The evaluation will also incorporate observations and case studies, allowing for a deeper understanding of best practices and challenges. All data collection tools will be designed to ensure inclusivity, considering gender, age, and disability aspects. The findings will be triangulated to ensure reliability and validity before being synthesized into the final report.

The main sources and methods of data collection will be through:

• Desk review
• Reviewing global action plan documents and secondary data
• Assessment and revision of the project's logical framework
• In-depth Individual interviews with Asmae’s partner and beneficiaries
• Focus Group Discussions with youth and, Partner’s Staff.
• Representatives if needed and other local stakeholders.
• Interviews with key informants or other relevant stakeholders
• Collect, revise, and plan the data matrix approach
• Collect quantitatively Case referral database
• Coordination meetings with Asmae and SCALE

Deliverables

Having considered any observations made by the field (Asmae and partners) and Head Office (Asmae country office and head office – internal coordination mechanism), and DCI steering committees, and requests for additional information and changes made by Asmae, the final report will be presented to Asmae.

It will be 25-40 pages maximum in length (excluding annexes) and will include:

• A cover page
• An executive summary (maximum 2 to 3 pages)
• The Endline evaluation purpose
• The methodology used for the endline evaluation (including constraints and limitations on the evaluation conducted)
• The findings and analysis
• A conclusion
• Annexes: Draft and final questionnaires for FGDs, KII, etc., developed tools for quantitative and qualitative data; report/notes of meetings with staff and Youth, analytical frameworks for both qualitative and quantitative data analysis.

Qualifications:

• Advanced degree in relevant field (Social Sciences, international development, project management)
• A minimum of 7 years relevant experience undertaking baseline and/or evaluation is required
• Good understanding of child protection and Education in the Lebanese context
• Deep knowledge about Capacity Building approaches and CSOs work in Lebanon
• Knowledge in crosscutting areas such as gender, inclusion (disability), child/youth participation would be an asset
• Good analytical, critical thinking, facilitation, and writing skills
• Proven knowledge and experience in applying participatory research methods and tools
• Experience conducting baselines/evaluations that have provided a participatory and meaningful experience for children
• Official authorization to work in Lebanon
• Native Arabic and fluency in English