External Final Evaluation of SPECIAL project financed by AFD

1. Asmae’s Background & partners

Asmae-Association Soeur Emmanuelle ("Asmae") is a French international NGO and registered charity, specialising in child development. Independent, secular and apolitical, it is open to all. Founded in 1980 by Sister Emmanuelle, based on her experience with rag pickers in the slums of Cairo, Asmae continues its actions in line with the values and methods inherited from its founder: listening and proximity, pragmatism, taking account of differences, professionalism and reciprocity. Asmae's actions aims to support vulnerable children and their families through personalised and continuing support for local actors working in the fields of education and child protection.

Asmae has been working in Lebanon, among others, in partnership with Imam Al- Sader Foundation (ISF) organisation which has established the “Strengthening Protective Early Childhood and Inclusion Approaches in Lebanon” (SPECIAL) project since August 2022.

Imam Al-Sader Foundations (ISF) is a Lebanese, non-governmental, non-profit association, founded by Mr. Musa Al-Sadr, and began its activities in 1963 with the mission of striving towards an advanced society that believes in social justice, free from ignorance, poverty, disease and violence, and where opportunities are equal for all. The Imam Al-Sader Foundation strives, through its vision, to empower women, youth, and children to achieve self-reliance, and to build a safe and dignified life, based on the values of tolerance, justice, and mutual trust, through educational, rehabilitative, cultural, health, social, and economic programs.

Education is one of the cornerstones of the Foundation's ongoing development programmes: pre-school, primary education and specialized curricula designed for children with special needs. Moreover, the School of Nursing at Imam Al-Sader Institutions has enabled thousands of graduates to find gainful employment and thus become economically independent. One of its specialized projects is the Women's Economic Empowerment Program with Intensive Vocational Training Programme, as an approach to empowering women and improving the economic and social conditions of disadvantaged populations. Additionally, the organization's pioneering curriculum for mental health professionals is the first of its kind nationally and trains workers to effectively recognize mental disorders and provide mental health services in rural Southern communities.

The project is funded by the Agence française de développement (AFD). The Agence française de développement (AFD) Group funds, supports and accelerates the transitions to a fairer and more sustainable world. Focusing on climate, biodiversity, peace, education, urban development, health and governance, the AFD carries out more than 4,200 projects in France’s overseas departments and territories and another 150 countries and contributes to support the sustainable development goals. 

This external evaluation, sponsored by Asmae Lebanon, reflects a concern to determine if the project design addresses the needs that were identified and to assess how well the project has been implemented to meet these needs, while taking into point the criteria of validity and efficiency of the project towards its beneficiaries.

2. Project Description

The project aims to enable children to reach their full developmental potential, which in the long run will have many impacts at the individual and collective levels, particularly in terms of social cohesion. The project is also part of a continuum between the different educational levels – early childhood development and education (nursery and pre-primary school) to primary school entry (acquisition of prerequisite skills, support for children and parents for the enrolment in primary school). Furthermore, this project proposes a holistic collaborative approach, involving public and private service providers, relevant ministries, civil society and communities. This project is based on a strong partnership with the partner CSO, Imam Al-Sader Foundation, which is responsible for the direct implementation, with the support of Asmae. Imam Al-Sader Foundation is therefore in charge of capacity building activities in 10 early childhood centres (nurseries and pre-primary schools) in the Tyre region, as well as parent support and community mobilisation. In a child rights-based approach, another key component has been to foster inclusive services, at the level of nurseries and pre-primary schools: through the review of educational programmes, through the training of the staff of the supported centres as well as through the provision of specialised support to children identified with special needs (related to disability, protection risks, gender discrimination, origin for refugees, etc). In addition, the project was designed to promote sustainability by institutionalising a network of actors (directly involved in the project as well as other local and national actors) in order to encourage quality inclusive initiatives in the long term.

• Duration of Project: August 2022- July 2025
• Geographical area of the project: Tyre- South of Lebanon
• Overall objective of the project: To improve access to inclusive, quality early childhood services for girls and boys aged 0-6 years in order to improve their overall development, in South Lebanon

Main Specific objectives and expected results are listed in the attached TOR

3. Justification of the Endline Evaluation:

The evaluation will measure the extent to which the project’s key objectives were met according to their Specific Objectives and the indicators:

• 80% of children demonstrating improvement across five key developmental areas.
• 80% of parents showing enhanced parenting skills and well-being.
• 75% of ISF staff adopting recommended organizational practices.

•   3.1 Measure Impact: The primary justification for conducting the end-line evaluation is to measure the actual impact of the SPECIAL project by   comparing the outcomes against the project's original objectives.
• Primary Impact: The primary, long-term effect is the improvement in the development of children aged 0-6 through quality early childhood services, as well as the enhancement of parents' skills to support their children. This includes changes in cognitive, social, emotional, and physical development for the children served by the project.
• Secondary Impact: Secondary impacts may include increased community awareness and support for inclusive education, improved collaboration among local stakeholders (e.g., nurseries, social services, and ISF), and strengthened community networks that promote child development and well-being.
• Intended and Unintended Consequences: The project will also be evaluated for any unintended outcomes. For instance, a positive unintended consequence might be the improvement of social cohesion in communities due to inclusive programming. On the other hand, potential negative impacts, such as overburdening local resources, will also be considered.

1. 3.2 Accountability: NGOs often need to provide transparent evidence of how resources were used. The SPECIAL project, especially with its specific focus on early childhood services, requires a clear, accountable evaluation to demonstrate resource allocation.

      3.3 The evaluation will assess whether the project was executed within the allocated resources efficiently,

      3.4 Learning & Knowledge Sharing: The end-line evaluation should provide key insights into what worked well and what          didn’t, allowing for learning from both successes and challenges

Inform Future Projects: The SPECIAL project’s end-line evaluation should guide future interventions & inform stakeholders by identifying what was effective and what needs revision and providing recommendations

    3.5 Evidence-Based Decision Making: The data collected from the end-line evaluation will provide robust evidence to inform both operational decisions and policy recommendations

    3.6 Sustainability Assessment: The sustainability of the SPECIAL project’s impact is crucial to its long-term success. The long-term sustainability of the project's impact will be assessed by looking at how well the community, partners, and ISF have integrated and institutionalized the project’s approach. For example, whether the training and awareness-raising activities lead to continued improvements in child development services or whether the partnerships and networks established are maintained beyond the project duration. The end-line evaluation will focus on whether the project’s benefits will continue after its completion

• 3.7 Questions related to crosscutting issues:
• Resource Allocation: The project will be evaluated on whether the allocated resources (financial, human, and material) are being used in the most efficient way. This includes assessing the cost-effectiveness of the interventions, such as the number of children served per dollar spent, or the cost per training session delivered to staff and caregivers.
• Timely Execution: The evaluation will look at whether the activities and milestones were completed within the proposed timeline, or if delays occurred, and the reasons behind them. Delays in implementing key activities, such as the provision of educational materials or capacity-building workshops, could affect efficiency.
• Human and Material Resources: The efficiency of training and coaching sessions for staff, the provision of resources to nurseries and kindergartens, and the delivery of parenting skills sessions will be assessed in terms of the resources used relative to the impact achieved.
• Utilization of Expertise: The evaluation will also assess whether the project has effectively used the expertise of local partners, trainers, and experts in the field to achieve its goals.

1. 3.8 Identifying Successes and Challenges

The evaluation will analyze factors that contributed to project success, such as:

• Effective training programs for caregivers and staff.
• The successful implementation of inclusive education in nurseries and kindergartens.
• Strong partnerships with CSOs and local authorities. The added value that Asmae has given to ISF based on Asmae’s support through Phase 1 for technical support and also in Phase 2 through the OCA.
• The ability of the Mobile Unit to reach more beneficiaries after being displaced due to war
  • • • To what extent does the project align with the evolving needs of children (0-6 years), parents, and communities in the Tyre region, particularly in the context of social, economic, and political challenges?
      • What lessons learned about adapting to external challenges should inform the design and implementation of Phase 2 to maintain long-term relevance?

It will also identify challenges that affected implementation, including:

• Difficulties in reaching marginalized or at-risk children.
• Resource constraints and logistical challenges.
• Resistance to inclusive practices in some communities.
• The war in South Lebanon, which disrupted project activities during the end of Cycle 2 and the beginning of Cycle 3.

4.Scope of the Evaluation

The consultant will be responsible for conducting the final evaluation of the SPECIAL Project implemented in South Lebanon, with a focus on measuring the project’s achievements against its planned objectives, results, and indicators. The consultant is expected to apply both qualitative and quantitative methodologies, including desk review, surveys, key informant interviews, and focus group discussions. The evaluation will capture the perspectives of key stakeholders, including children, caregivers, nursery and kindergarten staff, community actors, and project partners. The sampling strategy should ensure representation across geographic areas, vulnerability profiles (including children at risk), and types of stakeholders involved. Below is a table detailing the project activities, sampling targets, and proposed modalities for data collection

5.Recommendation

Based on the findings of the final evaluation, the evaluator is expected to formulate realistic and actionable recommendations that will feed future project’s design in a way that contributes to improving the program’s overall performance, sustainability, and long-term impact. These recommendations should be grounded in a thorough analysis of the program’s results, challenges, and opportunities within the broader operational and contextual framework.

The recommendations should be clear, practical, and prioritized, distinguishing between strategic recommendations, which focus on long-term sustainability and institutional strengthening, and operational recommendations, which address immediate, actionable improvements that can be implemented within the project's scope. Each recommendation should be accompanied by a rationale, highlighting how it addresses key gaps or opportunities identified in the evaluation.

Furthermore, the evaluator should specify relevant stakeholders responsible for implementing each recommendation, ensuring alignment with the capacities and roles of key actors such as ISF management, local authorities, CSOs, education institutions, and child protection specialists. The final evaluation report should provide a structured and coherent set of recommendations that guide programmatic and operational decision-making, supporting the organization in enhancing its effectiveness and long-term impact. 

The recommendations should consider potential evolutions in the political and security context in Lebanon. Recommendations should for that purpose include areas and activities suitable for emergency projects.

These recommendations should be actionable and include:

• Enhancing training programs for staff and caregivers.
• Improving outreach strategies to engage beneficiaries effectively.
• Strengthening stakeholder collaboration and network-building.
• Optimizing resource allocation and logistical planning for improved project delivery.
• Foster closer partnerships with local government bodies and emergency response organizations.
• Invest in mobile unit infrastructure to ensure continuity during emergencies, thus establishing an alternative service delivery method (ex: community hubs) if mobility is restricted.
• Develop a flexible resource distribution model to respond to evolving community needs and Design adaptive session formats (group or individual) to continue psychosocial support during disruptions.
• What are the key lessons learned from supporting 1,000 parents and guardians in their educational role, and how have these interventions improved parenting practices and home environments?
• What additional resources or strategies would enhance the quality and reach of early childhood services in Phase 2?
• How can inclusive practices be further institutionalized and expanded to reach more vulnerable children in Phase 2?
• How has the organizational capacity of the Imam Al-Sader Foundation (ISF) evolved throughout the project, and what areas need further strengthening for Phase 2?
• Are there emerging needs or gaps in early childhood development services that Phase 2 should prioritize?

6. Methodology and Process

The endline evaluation must provide evidence-based information that is credible, reliable, as well as useful and actionable in designing a future phase.

The main sources of information must be secondary, through review of all relevant documentation linked to the project, and primary via interviews of a sample of stake holders which includes direct and indirect beneficiaries as well as representatives if needed and other local stakeholders.

.

Example of secondary sources of information are:

• Asmae Lebanon’s AFD projects Literature
• Global action plan documents
• Project's logical framework
• Project’s mid-term evaluation report

As for primary sources of information, approaches for collecting them must include

• In-depth Individual interviews with Asmae partners and beneficiaries
• Focus Group Discussions with youth and, Partners Staffs.
• Interviews with key informants or other relevant stakeholders
• Coordination meetings with Asmae and Partners personnel.

In the technical offer, the consultant is expected to clarify how he or she will proceed for gathering primary information as well as the reason for the choice (online meetings/interviews, in person, a mix of both approaches, observation…). Information shall also be shared on the sampling of project’s target and stakeholders to be involved in the primary sources of information collection process.

7. Report format:

It will be 40 pages maximum in length (excluding annexes) and will include:

• A cover pages
• An executive summary (maximum 2 to 3 pages)
• The Endline evaluation purpose
• The methodology used for the endline evaluation (including constraints and limitations on the evaluation conducted)
• The findings and analysis
• A conclusion
• Annexes: Draft and final questionnaires for FGDs, KII, etc., developed tools for quantitative and qualitative data; report/notes of meetings with staff and Youth, analytical frameworks for both qualitative and quantitative data analysis.

8.Timeframe

The conditions for carrying out the study: Contract and Implementation of the study duration (30 days) as per the above schedule

     9. Property of the report and confidentiality

The Endline evaluation report is the property of Asmae.

The consultant is subject to a confidentiality obligation concerning all the information and documents that may be brought to his or her attention in the exercise of his or her duty. She/He must protect the information collected and must not use it to gain any profit whatsoever.

   10.Research ethics plan.

Endline evaluator is required to set out their approach to ensuring complete compliance with international good practice about research ethics and protocols, particularly with regards to safeguarding children, vulnerable groups (including people with disabilities), and those in fragile and conflict-affected states. Consideration should be given to:

• administrative, technical, and physical safeguards to protect the confidentiality of those participating in research;
• physical safeguards for those conducting research;
• data confidentiality for personal information;
• parental consent concerning data collection from children or collection of data about children;
• assent/consent processes as per age and cognitive understanding of children involved in the evaluation need to be taken into consideration
• age-appropriate participation of children, including in the development of data collection tools.

In addition, Asmae possesses a SCPP (Safeguarding and Child Protection Policy) as well as a PHESA (Policy for Prevention and Protection against Harassment, Exploitation and Sexual Abuse) to which terms consultant(s) must comply

Required qualifications and procedures

Qualification of consultant:

• Advanced degree in relevant field (Social Sciences, international development, project management)
• A minimum of 7 years relevant experience undertaking baseline and/or evaluation is required
• Good understanding of child protection and Education in the Lebanese context
• Deep knowledge about Capacity Building approaches and CSOs work in Lebanon
• Knowledge in crosscutting areas such as gender, inclusion (disability), child/youth participation would be an asset
• Good analytical, critical thinking, facilitation, and writing skills
• Proven knowledge and experience in applying participatory research methods and tools
• Experience conducting baselines/evaluations that have provided a participatory and meaningful experience for children
• Get the official authorization to work in Lebanon
• Experience in Tyr region and awareness of the cultural sensitivity is a plus

Language requirement:

• Arabic and English writing and communication skills are required.