Home Based Skills Development Programme External Evaluation

1.                   BACKGROUND – SC In response to the growing needs of the Syrian refugee community and host community in Lebanon, Save the Children established its Food Security and Livelihoods program in Lebanon in late 2012. SC Lebanon’s program currently supports household immediate food and non-food needs through food parcels for new arrivals; unconditional cash assistance; and WFP e-vouchers. In parallel we are strengthening the ability of Lebanese and Syrian households to improve their self-reliance and livelihoods.  One of our principal target groups is youth aged 18-24 whom we have been supporting to date with placements in humanitarian organizations as trainees; youth-led community initiatives and vocational training.  We are also supporting a sub set of particularly vulnerable households to get the opportunity to participate in more home based work or in home based livelihood skills training to help them support their families through a programme known as Home-Based Skills Development (HBSD), which is the focus of this evaluation. The main focus is on developing skills and services that meet the needs of the refugee population (such as food processing, soap making, and dairy production), complemented with business management and entrepreneurship training and linked to distinct market opportunities. At present Save the Children’s livelihoods programming has been focused on Central Bekaa and Akkar, as well as some initial youth programming in Tripoli and in Saida. SC is keen to build on its existing programming experiences to build a layered portfolio of interventions of food assistance and livelihoods interventions in targeted communities, with social cohesion at the core of programme design. We also plan to expand and adapt current programming to urban contexts, with a particular focus on strengthening livelihoods dependent on the informal economy.   Programme approach and methodology The Home based skills development programme was designed to target vulnerable household not usually targeted in cash for work programs because of the nature of the work (tough, physical, outdoor work) or the conditions (long distances, long hours not compatible with family requirements). The project is innovative within SCI as it combines emergency support to basic needs through cash transfers (200 USD/month during 3 months), and recovery programming through the introduction of a livelihoods component. Combining livelihood skills (both technical and entrepreneurship) offers the advantage of preparing beneficiaries for more sustainable livelihood opportunities though business creation or access to employment in Lebanon or (eventually) Syria. Beneficiaries were provided with the following package: Vocational training (from 3 to 5 days) provided by local training providers[1] with whom SC has partnered (soap, bread, food processing, women accessories, dairy production, beauty care, etc.); Entrepreneurship skills training; Kit distribution; Tailor-made coaching from the FSL team during 3 months based on group management and Business Management Training received from SC partners   Home-based goods or services production were identified as having low supply combined with high local consumer demand as highlighted in the SC Lebanon Skills Gap Assessment (e.g. soap making, food production, dairy products, women accessories, etc.). Goods and services were eventually identified through a field assessment conducted between November and December 2013 in both areas.   Beneficiaries were encouraged to form groups (average of five people per group) based on geographical self-selection and skills. The project mainly targeted women and group approach has the benefit of providing women with a space to meet with other women who are enduring similar hardships, where they may decide to share personal issues and coping strategies between themselves. Given the nature of displacement, the establishment of groups will also contribute to rebuilding and establishing linkages and support networks within neighbourhoods in Lebanon. The group approach also has benefits in terms of its economic effectiveness as it encourages women with different existing skillsets to be matched, and responsibilities to be shared. They are supported with group management training (to be delivered by SC staff) and the group work skills they acquire will also lend themselves well to accessing work in cooperatives in future. 3. SCOPE AND PURPOSE OF THE EVALUATION The objectives of this evaluation are:   To make an informed assessment on the performance of the project in the last month of its implementation (May 2014). This will provide the donors (DFiD and CIDA), SCI and its partners with sufficient information regarding the relevance, the effectiveness, the efficiency, the impact and the sustainability of the project. To provide SCI and its partners with practical recommendations for improving existing tools and the approach for the next phase of the project (and for new funding, i.e. from BPRM for a similar approach) To provide SCI and its partners with documented information to be used for advocacy purposes towards potential donors and Lebanese national authorities.   The evaluation should include: An assessment of the overall project results and approach is carried out taking into account the context. Practical recommendations regarding the methodology, tools and approaches used by the project to 1) meet the needs of the target beneficiaries, 2) improve the quality of the response, 3) increase the FSL staff and beneficiaries’ capacities. Recommendations on internal SCI systems and management structure (staffing structure etc.) specific to this programme Strengths and weaknesses of the project’s methodology and implementation procedures, with a view to increasing impact and sustainability in the next phase of the project.   1.       Evaluation questions and criteria The following table outlines criteria and questions which   Criteria Evaluation sub   questions   Indicators to be   proposed by the consultant Relevance   Did the project address   needs of the target population?   Was the project   design appropriate to the specific contexts of Bekaa and Akkar?   To what extent   beneficiaries were consulted in project design and implementation?   Are the   mechanisms and approaches developed in coherence with the priorities SCI   strategy, of the Government as defined in the National Poverty programme and   the standards set by the Humanitarian actors (working groups)?   Are the livelihood   needs of vulnerable Syrian and Lebanese households similar enough that the   same approach works for both groups, or should separate livelihood strategies   be developed?   Efficiency   Can the results   achieved be considered adequate for the budget spent?   Was the   organization of the FSL team efficient for implementing the project in the   area covered?   Was the   allocation of resources among different activities of the project (Training   support, staff capacity building, business support (kit), and groups coaching),   appropriate to achieve results?   Was the   allocation of the resources between SCI and partners optimum to achieve   results?   Effectiveness (NB:   specific questions on partnerships are below)   To what extent, did   the project achieve expected results? Did the project team implement all   initially planned activities?   How did the project   team adjust the project and its methodology to any constraints faced during   the implementation phase?   Were the criteria   set for selecting the beneficiaries effective?   Impact/Outcomes   To what extent   did the project contribute to create livelihoods opportunities for vulnerable   HH amongst refugees and host communities in Akkar and Bekaa through skills   development and training?   Are the created business   groups likely to be maintained after the end of the project support?   What key   determinants of group success can be identified?   Which types of   skills are more likely to lead to long-term livelihoods outcomes and why?   Do beneficiaries and   their families report increased confidence in accessing livelihoods   opportunities?   Did the group   approach contribute to enhance livelihoods and self-confidence of   beneficiaries?   Did the project   have any unplanned for negative effects and/or is this likely in future?   Sustainability   What elements of   sustainability have been identified in the programme?   What   recommendations could ensure increased sustainability of the programme at   progressively reduced cost?   Analyse the   degree of commitment of all parties involved, such as partners, Local   authority, related stakeholders (CDC, etc.), etc.   Cross cutting :     Gender/Protection   Was gender   balanced integrated in project design and implementation?   Analyse the   relevance and effectiveness of referral mechanisms within this programme   (GBV, child protection and disability in particular)   Partnerships (Atayeb and AlMajmoua)   Was partner   selection conducted in a systematic and objective way?   Were the training   curriculums delivered by the partners clear and competency-based? How can we   improve the quality of curriculums in future?   Analyse the   degree of commitment of the partners at national level (communication with SC   Beirut team) and at local level (communication with SC FSL programme teams)         2.       EVALUATION METHODOLOGY & Timeframe The expert (or the team of experts) shall propose a participatory evaluation methodology (including a gender approach), addressing main project stakeholders as well as beneficiaries and their families. Activities are expected to include: Desk review of the project’s documentation to date (and documentation from similar initiatives implemented by other organisations) Meetings and discussions with relevant SCI staff in country office (Project’s staff, FSL technical advisor, partnership manager, etc.) Key Informant Interviews at national and local level (i.e. Atayeb el Rif and AlMajmoua; municipalities; local traders etc) Individual interviews and/or Focus Group Discussions with project beneficiaries through fields visits in the 2 areas of intervention;   3.       EXPECTED OUTPUTS All deliverables are expected to be in English. Inception Presentation The Inception presentation will be due before the commencement of activities, and this presentation will provide the refined methodological approach to the assignment, the Work Plan and any issues related to the ToRs that may require discussions and adjustments.   Evaluation Tools, Instruments and Protocol The consultant shall: -          Recommend a list of relevant indicators related to the criteria described above -          Present the tools and instruments to be used (questionnaires, focus group discussion guide, interview questions, desk review, etc.) -          Outline the protocol to be used for sampling and FGD technique. Draft report   Presentation of findings (workshop): The presentation is expected to be finalized within 5 days after the completion of the last assessment activity. The presentation is to be delivered to selected key stakeholders. It should include the preliminary findings of the evaluation, conclusions and recommendations. Final report: The final report will be presented within 5 days after the delivery of the presentation of findings, and should include the clarifications that were required during the presentation. It should include: Summary of findings; Introduction to the context; Detailed findings (including clear responses to the questions outlined in the ToR); Recommendations; Tools developed for evaluation purposes; List of persons met during the evaluation process and salient points of the meetings   4.       TIMEFRAME The total duration of the evaluation should not exceed 25 days, broken down approximately as follows: -          Desk review: 2 days -          Development of tools, instruments and protocol: 2 days -          Meetings with key informants and FGDs[2]: 14 days -          Draft report: 4 days -          Presentation of findings & feedback: 1 day -          Final report: 2 days   5.       CONSULTANT SPECIFICATION The following is a list of desired characteristics for the lead consultant on this evaluation. Proven      experience in livelihoods programme design, implementation and monitoring Experience      of livelihoods programming for vulnerable groups in Lebanon and in the      rest of the MENA region (experience outside Lebanon is preferable) Excellent      written and spoken English language skills; and excellent spoken Arabic Evidence      of experience leading participative evaluations and designing evaluation      tools Experience      including  gender analysis in      evaluation protocol Experience      working with Save the Children   The consultant will report to SC’s FSL Advisor, based in Beirut. It is proposed that the consultant also engages additional support to conduct field work, particularly FGDs. [1] Atayeb el Rif and AlMajmoua [2] Local authorities; NGOs and UN; ILO; Ministry of Labour (central and local level); Employers’ Organizations; Workers’ organizations; Employment offices and recruitment companies; Vocational training providers; Job seekers (women); Job seekers (men); Job seekers (youth, disaggregated by gender)