Consultant

    i.    Introduction World Vision Lebanon (WVL) has been implementing an EU-funded project entitled “New Education Perspectives for Children with Special Learning Needs” since February 2014 in 9 Palestinian camps across Beirut (Mar El-Yas Camp, Shatila camp, Borj Al-Barajneh Camp, Sabra Camp as well as other Palestinian gatherings) North (Naher Al-Bared Camp, Badawi Camp, and other unofficial gatherings) and South of Lebanon (Al Buss Camp, Al Rashidiyeh Camp and Burj Al Shemali Camp).   The project is a 2-year project which seeks to contribute to creating an inclusive environment for Children with Disabilities (CWDs) in the educational programs within the Palestinian community by mobilizing and equipping relevant stakeholders in the refugee community (parents, youth, and kindergartens) to improve the learning environment. The project is being implemented in partnership with Community Based Rehabilitation Association (CBRA) organization in the North and with Mousawat organization in Beirut and South.   This document is a reference to the end-of-project evaluation which will cover the two years project’s implementation. WVL’s purpose is to evaluate the project’s relevance, effectiveness, efficiency, impact and sustainability for the 2 years of project implementation. 1.    Evaluation Summary Project New Education Perspectives for Children with Special Learning Needs Project Phase End of project Evaluation Type End of Project Evaluation Evaluation Purpose The purpose of the evaluation is to assess the project’s relevance, effectiveness, efficiency and accountability of the project’s implementation. The current evaluation will highlight the project indicators (specific and result level objectives) with an intentional focus on WVL’s structure and managerial approach. Primary Methodologies Qualitative and quantitative methodologies Evaluation Start and End Dates May 2016 - July 2016 Anticipated Evaluation Report Release Date July 29, 2016   2. Description of Program or Project Being Evaluated As described in the concept note, over the duration of 2 years, the overall objective of this action is to contribute to creating an inclusive environment for Children with Disabilities (CWD) in the educational programs within the Palestinian community. With the framework and resources of the action, the specific objective is to mobilize and equip relevant stakeholders in the refugee community (parents, youth, and kindergartens) to improve the learning environment of CWD. Initially, the action sought to adopt an early intervention approach targeting age groups between 0 and 6 years old in order to facilitate their inclusion into Kindergartens and later in life into schools. However, after an initial round of identification and screening of children with disabilities by the co-applicants ensuring that all 0 to 6 year olds were reached, older children were targeted too.   Overall Objective: To contribute to creating an inclusive environment for Children with Disabilities in educational programs within the Palestinian community   Specific Objective: Relevant stakeholders in the refugee community (parents, youth, and kindergartens) are mobilized and equipped to improve the learning environment of CWD   The action will achieve four key results that were slightly changed in the process of re-design. Concept Note Re-design Result 1: 850 CWD are equipped so that they are better able to access education opportunities   Result 2: 600 Parents of CWD are equipped so that they support their children’s education   Result 3: 25 Kindergartens are provided with technical and material support to create an inclusive learning environment   Result 4: 9 Parent Action Groups and Youth Groups are empowered and equipped to lobby for inclusion of CWDs in UNRWA schools Result 1: 850 CWD are equipped so that they are better able to access education opportunities   Result 2: 600 Parents of CWD are equipped so that they support their children’s education   Result 3: 25 Kindergartens are provided with technical and/or material support to create an inclusive learning environment   Result 4: 3 Parent Action Groups and 2 Youth with Disability Groups are empowered and equipped to lobby for the education rights and acceptance of CWDs in their communities   As described in the concept note, the achievement of Result 1 will show that CWDs are supported with key skills to facilitate their integration in educational institutions through the provision of trainings and consultations tailored to their needs delivered by a qualified cadre of staff trained on the various theoretical and pedagogical methods pertaining to inclusive education. The staff will be performing outreach and home visits and collecting data to establish a baseline in order to track the progress of CWDs and their caregivers. Result 2 will directly support result 1 by working with the caregivers and parents of CWDs through trainings that enhance their understanding of their children’s disability and their potential and their ability to support their children’s education. This will be done through group trainings and home follow-up. Result 3 will move to work with educational institutions that work with the targeted ages, mostly Kindergartens, to improve their accessibility, train their teachers, and provide on the job-coaching. Material rehabilitation or adapted equipment is not sufficient, teachers and staff need to be confident and empowered to deliver inclusive education, the action will thus combine the two approaches. Finally, Result 4 will work at the community level through mobilizing, creating, and training groups composed of parents and youth that will work together to produce a Lobbying and Advocacy Plan aimed at working with the community and key decision makers to change the perception of CWDs and implement sustainable policy changes that improve the learning environment ls improving the sustainability and allowing for the monitoring of improvement in the CWDs learning environments from one year to the other. The action will continue to work with the children already targeted in the first academic year and will, when possible, add additional beneficiaries. However from experience, the large number of beneficiaries and the corresponding outreach activities will ensure that most CWDs in the Palestinian communities are targeted from the beginning of the action.    This action will target three main areas in Lebanon where Palestinians reside: Beirut, South, and North. In the North, co-applicant Community Based Rehabilitation Association (CBRA) will be running its operation from its centres in Naher Al-Bared and Badawi Camp while supporting other unofficial gatherings where Palestinians reside. Co-applicant Mousawat will be running its operations from its center in Beirut at Mar El-Yas Camp. Important Note: It is important to mention that the project faced implementation and budget-related challenges during the first year of implementation which resulted in many delays in program implementation. In addition to that, the project underwent a redesign which also delayed the implementation of certain activities due to the time (8months) needed to receive approval from EU on the redesign documents. All in all, the impact of the challenges faced is that the implementation of activities related to Results 3 and 4 were delayed to Year 2. However, the project received an extension of 3 months. 3. Evaluation Target Audiences Children with Disabilities and learning difficulties, Parents of Children with disabilities and learning difficulties: They are the main target audience, beneficiaries of the project and participants in the end-of l8project evaluation. The evaluation findings will identify the changes brought upon their lives and well-being.   WVL: The evaluation will inform WVL on the successes and challenges faced throughout the project implementation. It will also highlight the effectiveness and sustainability. Documented information will further contribute to generating lessons learned and recommendations that would guide future interventions implemented by WVL.   CBRA and Mousawat (WVL Partners): Partners of the program will play an important role in the evaluation as their documentation of project activities is the main source of secondary data for the end-of-project evaluation. Their input will be for logistic purposes related to data collection with project beneficiaries as well as during the validation process of the results. Partners of the program will benefit from the results of the evaluation as this will help guide their future actions for the improvement of their work with Children with Disabilities and learning difficulties.   Support Office - WV France: The results of the evaluation will present information pertaining to the relevance, effectiveness, efficiency, impact and sustainability of the project. It offers WVL the opportunity to be accountable towards the support office as the process will be documented in details and the data will be generated with the participation of the community.   European Union: The results of the evaluation will present information pertaining to the relevance, effectiveness, efficiency, impact and sustainability of the project. It offers WVL the opportunity to be accountable towards the European Union, funder of the grant, as the process will be documented in details and the data will be generated with the participation of the community.   4. Evaluation Type The current evaluation is an end of project evaluation. It is a mixed method evaluation including qualitative and quantitative components and it will be conducted to determine what has been the project’s effectiveness. This evaluation will allow the comparison of the findings with the baseline results to some extent and it will mostly serve for comparison purposes with the mid-term evaluation.   5. Evaluation Purpose and Objectives The purpose of the evaluation is to assess the project’s relevance, effectiveness, efficiency and accountability for the 2 years of the project implementation. The current evaluation will highlight the project indicators (specific and result level objectives) as stated in the project’s Monitoring and Evaluation (M&E) plan with an intentional focus on WVL’s structure and managerial approach.   The objectives of the evaluation are as follows: To identify if the program was properly designed to meet targeted needs through evaluating and assessing program theory, logic, conceptual components and assumptions To assess the progress made towards achieving the project’s specific objective and results (mainly 1&2) based on the logframe and the M&E plan (re-design documents) To investigate whether the resources (financial, human, and materials) have been used efficiently and effectively for the well-being of the target community. To assess the sustainability of the project, particularly focusing on the partnering approach and the capacity and willingness of partners to sustain the program’s achievements This Evaluation Terms of Reference specifies five primary dimensions, namely to assess the a) relevance, b) effectiveness, c) efficiency, d) impact and e) sustainability of the project. It is expected that the evaluator will divide these objectives into three broad stages of the project cycle: project design, implementation and management, and achievements. The evaluation matrix is attached as Appendix A and it is required to be completed by the consultant as part of developing the evaluation methodology.   6. Evaluation Methodology The evaluation methodology is informed by the M&E plan specified during project re-design. It is aligned with WV’s guidelines and standards for ensuring good quality evaluation process. The assessment methods should be a combination of quantitative and qualitative methods with the relevant project’s stakeholders, partners and beneficiaries. Data collection methods may include but are not limited to secondary data information, general observations, surveys, Key Informant Interviews (KII) with main stakeholders/partners, and focus group discussions (FGD). In order to be able to compare with the mid-point evaluation, it is expected from the consultant to use the same questions that were used during the mid-point evaluation. In addition to that, the consultant is expected to develop the appropriate methodology and tools that would best capture the objectives of the evaluation.   Proposed Methodology The evaluation will be following but not limited to the below methodologies: Quantitative evaluation targeting children with disabilities enrolled in KGs and UNRWA schools, parents of children with disabilities and KG teachers based on the M&E plan. Qualitative evaluation with parents of children with disabilities benefitting from the program activities, teachers of the KG benefitting from the presence of shadow teachers, project partners, project manager, shadow teachers, specialists, KG administration, UNRWA contact person.   Data collection and analysis The meetings and the qualitative data collection will be completed by the consultant as a facilitator, and a note taker (someone from the consultant’s team) shall be present during the FGDs and the KIIs. For the FGDs and the KIIs, thematic analysis of the data will be completed in order to discern patterns and come up with conclusions that would reflect the views and the perceptions of the participants. The information collected during the FGD will be used to triangulate the quantitative data for a holistic understanding of the evaluation results. For the quantitative data collection, it will be completed in each of the areas by selected data collectors hired by the consultant and supervised by him/her. Appropriate statistical tests will be employed depending on the type and measurement scale of the outcomes and independent variables (e.g., continuous, nominal, or ordinal). For instance, ANOVA will be used for comparisons between means of scores (probably knowledge and attitudes); chi-square test will be used to determine the significance of differences in percentages;  spearman rank correlation coefficients will be used to examine the relationship between different scores used; non-parametric equivalent tests may be used in case the data is not normally distributed etc. Data will be analyzed with statistical significance set at 0.05 and confidence intervals should be calculated and added. The details of the data collection are shared below in section 8 “Authority and Responsibility”.   7. Limitations Baseline measurements were mostly qualitative which limits the ability of comparison with end of project evaluation. However, the mid-term evaluation contains a quantitative part that shall be used for comparison with the end-of-project evaluation.                     8. Authority and Responsibility 8.1 Team Members and Roles   Role Responsibilities and tasks Consultant Develop and submit the evaluation methodology (sample selection, quantitative & qualitative methodology) Submit evaluation matrix Develop data collection tools Pilot testing of the tools Recruit and train data collectors Recruit and train data entry clerk Supervise and assure the quality of quantitative data collection Develop data analysis plan Conduct in depth data analysis Provide weekly feedback to WVL team Prepare a power point presentation of the main findings Present findings to WVL staff and partners during a stakeholder interpretation and validation workshop Draft the Evaluation report Finalize the Evaluation report based on feedback from WVL team Submit a 2 pager summary fact sheet (English and Arabic). DME officer Finalize the evaluation ToR Provide feedback on evaluation methodology  and matrix Provide feedback on data collection tools Follow up with the consultant on data collection and analysis Provide feedback on data analysis plan Provide feedback on data analysis Provide feedback on the power point of the main findings Provide feedback on the draft evaluation report Share draft evaluation report with WV France and EU and other stakeholders for feedback Share final evaluation report with WV France and EU and other stakeholders DME Manager Review and approve TOR Review evaluation methodology and matrix Ensure quality of tools developed Ensure quality of data collection Review data analysis plan Provide critical feedback on draft evaluation report Ensure good quality of the final evaluation report Project Manager Provide technical support in finalizing data collection tools Provide logistical support in planning and organizing data collection Provide technical input for the analysis, synthesis of data, and drafting of recommendations Provide critical feedback on the draft and final evaluation report   8.2 Partners (i.e., Who will be involved in the evaluation?) The partners involved in the evaluation process will be CBRA and Mousawat project staffs whose feedback and input is essential in the end-of-project evaluation process as well as in the interpretation of the results. Partners will also be supporting logistically to organize the data collection from beneficiaries.   9. Logistics                                  The recruited consultant will be in charge of finalizing logistical issues related to the transportation to the different areas where data collection will be completed. As for the location for completing the different meetings (KIIs, FGDs, validation meetings, etc.), it will be in the partners’ offices in the different areas. The paragraph below will explain in details how the data collection and validation will take place:   Data collection: The DME Officer, Project Manager in WVL and partners will be supporting with the coordination of the different meetings/surveys with beneficiaries and project’s stakeholders.   The final presentation of results will take place in one meeting at WVL where Project Manager, project staffs, partners, Operations Directors, PDQA Manager, DME Manager, DME and RAM teams and education specialist will be invited to participate in the discussion and learn about the evaluation results.   10. Products The end product of the evaluation will be the evaluation report. In the preparation of the final report, the consultant is expected to provide the reader with, as much as possible, accurate sources of information and conclusions.  In fact, all findings and statements must be backed by existing data and/or data based on approved methodology for sampling, data collection, and analysis. As explained in the previous sections, the process will integrate intentionally the feedback and the input from the partners and it will be shared with them in different formats, based on their preference and involvement. In fact, the LEAP evaluation report will be shared with NO point people (DME manager, PDQA manager, Operations Directors, Technical Lead and Education Specialist), Support Office point people (WV France) as well as the EU. The 2 pager fact sheet summary will be translated to Arabic and will be disseminated among partners who participated during the implementation and/or evaluation of the end-of-project evaluation. In addition, an interpretation and validation workshop will be held in order to share the findings with partners who were part of the evaluation process.   Other expected deliverables throughout the process of evaluation include: Evaluation Methodology (including Evaluation Matrix) Validated and tested data collection tools Thematic in-depth analysis of qualitative data Data sets (excel or SPSS formats) Data analysis plan and SPSS outputs Evaluation report (based on LEAP Evaluation Report Template) Power Point presentation of the findings (in English) 2 pager fact sheet summary (English and Arabic).   11. Budget The funding available for this end of project evaluation should include all fees (transportation, logistics, accommodation, enumerator and note taker fees, etc.) as identified by the consultant.   12. Documents  Major documents that need to be reviewed during the research phase of the evaluation will be: Project description and detailed implementation plan (DIP) Project logframe, M& E plan and Indicator Tracking Table (ITT) Project baseline assessment report and tools Mid-term evaluation report and tools Evaluation report guideline Evaluation report template   13. Lessons Learned Reflection events will take place with the project manager, Education specialist, RAM team, DME team, partners in order to discuss the challenges, lessons learned and recommendations identified in final report. 14. Qualifications of the consultant The consultant should have the following competencies and experience: At least 5 years of progressive proven experience in similar studies and in using both qualitative research methodologies and data analysis; Previous evaluation experience of education related projects (to be provided within the CV); Extensive professional experience in the design and implementation of outcome and impact evaluations; Good analytical and critical thinking; Proven knowledge and experience in applying participatory research methods and tools; Advanced degree in relevant field (e.g. Public Health, Epidemiology or any related technical field); Good understanding of the Humanitarian work; Familiarity with civil society and Non-Governmental Organization (NGO) engagement; Ability to work on tight schedules with minimal supervision; Good English and Arabic speaking and writing skills. 15. Duration of consultancy WVL has set aside 2 months (Mid May, 2016 till Mid July, 2016) for this assignment. Bids should include a detailed proposed work-schedule with specific tasks and a budget should also be incorporated that outlines all relevant costs that will be associated with this evaluation. Candidates are expected to state how much time they will need to start the assignment and how much time they need to conduct every task. Selected candidates are also expected to abide by the deadlines and the conditions for deadlines specified within the contract/agreement. Interested individuals and consultancy firms should send in their applications no later than 16.00 hours GMT on May 5th, 2016 including a detailed Technical and Financial Bids with the following documents: proposed methodology (including a revised timeline and budget), curriculum vitae and/or resume, The names and addresses (including telephone and e-mail) of two non-related referees, and sample of previous work (evaluation reports related to the topic). Technical and Financial Bids should be in English, should not exceed 5 pages and should be submitted in two separate Sealed Envelopes.   Important: The top right-hand side of the envelopes (Technical and Financial) must be clearly marked with the name of the consultant and the subject line “New Education Perspectives for Children with Special Learning Needs End-of-Project Evaluation” and sent to the below address: “Main National Office address: Villa Siniyora, Mountazah, Mansourieh; Office Phone: 961-4-401-980 Mailing address: World Vision Int'l Lebanon P.O.Box 55355, Sin el Fil, Lebanon”   Technical bids should also be sent by email with the name of the consultant and the subject line “New Education Perspectives for Children with Special Learning Needs End-of-Project Evaluation” to the following email address consultant_lbn@wvi.org. Only short-listed candidates will be contacted. In case the Consultant is not registered with Ministry of Finance, a deduction of 7.5 % should always be considered. N.B: Selected consultant will be provided with all additional documents required to carry out the work.   16. Appendices Appendix A: Evaluation Matrix Appendix B: Timeline   Appendix A Evaluation Matrix   Evaluation Objectives Issues/Questions to be asked Methods Sources To identify if the project was properly designed to meet targeted needs (Relevance) Was the project designed in a way to meet the needs of the community? To what extent was the project successful in addressing the needs of the community? How appropriate was the project from the communities’ points of views including partner? How appropriate was the project from WVL staff points of view? How appropriate was the selection of the partner for the project? How did the challenges faced in the management affect the achievement of the project’s objectives?     To assess the progress made towards achieving the project specific objective and results based on the log frame and M&E plan (Effectiveness)   How well were the projects’ objectives achieved? What were the projects’ strengths in terms of the method used and the design? How properly was the project executed and how effective was the monitoring system?  How did implementation variations (from program design affect results? What were the projects’ weaknesses in terms of the method used and the design)? What alternative methods would have led to a better achievement of program’s objectives? How well did the project adapt to unexpected problems? How flexible was the project design?     To investigate whether the resources (financial, human, and materials) have been used efficiently and effectively for the well-being of the target community. (Efficiency ) Was the methodology of implementation adequate under the circumstances? Were most of the outputs achieved to an acceptable standard? Were most of the outcomes achieved to an acceptable standard?       To assess the potential impact of the program on the targeted communities (Impact) To what extent the community benefited from implemented activities? To what extent did the partner benefit from the activities and outputs? Have there been any unintended or negative changes that can be attributed to the project? What differences have the project made to people’s lives?     To investigate whether the design and implementation of the project tackles sustainability (Sustainability) To what extent do community members and partners own the project? What are the individual, organizational, and community level capacities? Does the project have a transition strategy? Are there mechanisms in the community to sustain the project? What was the added value of the partnership established?         Appendix B Responsibilities Week 1 Week 2 Week 3 Week 4 Week 5 Week 6 Week 7 Week 8 Review material and documents provided  X               Develop the evaluation methodology  X               Receive feedback from WV team on the methodology    X             Finalize evaluation methodology    X             Develop data collection tools     X            Receive feedback from WV on tools      X           Finalize and pilot test the tools       X          Qualitative & quantitative data collection conducted        X  X       Conduct data cleaning & analysis         X   X     Meet with WV team to provide input on analysis              X   Draft the Evaluation report              X   Prepare a power point presentation of the main findings                  X Present finding to WV staff and partner                X Finalize the Evaluation report based on feedback from WVL team                  X Timeline