Support For Water And Sanitation In The Bekaa

1. Evaluation Summary Project Title Support for Water and Sanitation in the Bekaa Evaluation Type End of Project Evaluation Evaluation Purpose • To assess the effectiveness of the project’s activities and results. • To assess the factors of sustainability within the project. • To measure the impact of the project and change at the level of the community by comparing the findings with the available baseline results. Primary Methodologies Quantitative and qualitative methods Evaluation Start and End Dates July till September 2017 Anticipated Evaluation Report Release Date September 30, 2017 2. Description of Project Project is funded by European Union (EU) Name of affiliated entity in the Action Bekaa Water Establishment (BWE) is the main partner Project Start date and end date September 2014- March 2017 (extended till July 2017) Project Location Bekaa (Ablah & Taalabaya) Beneficiaries & target groups • 1,600 households (HHs) which will have access to water network in Ablah and Taalabaya as follows:  1,100 will have access to the water network in the town of Taalabaya  500 will have access to the water network in the town of Ablah  A total of 3,932 HHs in Taalabaya and Ablah as well as 4,448 HH in further Bekaa villages (the latter through an added activity) will be provided with the opportunity to be connected to the water network, with pipes leading to their house. • 9,987 individuals (mainly students) participating in hygiene promotion activities. Project objectives • Overall Objective: Mitigate the risk of WASH-related mortality and morbidity through provision of and access to safe water, and hygiene services among the target population of the Lebanese host community and Syrian refugees • Specific Objective 1: Increased access to basic water supply infrastructure among target population • Result 1.1: Expanded municipal water network in target communities • Specific Objective 2: Improved water, sanitation, and hygiene practices among target population • Result 2.1: Expanded hygiene and water conservation awareness in target communities In this project, World Vision Lebanon (WVL), funded by the European Union (EU), aimed at providing support to enhance basic infrastructure and economic recovery in Lebanon. The overall objective of this action was to mitigate the risk of WASH-related mortality and morbidity through provision of and access to safe water, and hygiene services among the target population of the Lebanese host community and Syrian refugees. The first specific objective under this project was to increase water supply for both vulnerable Lebanese host communities and Syrian refugees in the targeted areas in Bekaa. The rationale behind this intervention is the need of significant improvement in the water network in many villages as a result of an outdated network that was established over 50 years ago. Moreover, a feasibility study conducted by World Vision together with the Bekaa Water Establishment (BWE) showed that parts of the network needing replacement were laid as early as the 1950s. Therefore, the intention of the proposed action is to rehabilitate and further extend the water network (and household connections) across two municipalities (Ablah and Taalabaya), for a total of 43.395km. For this purpose, an engineering firm was recruited for the design of the network and contractors were responsible for the execution under the supervision of the WVL’s project engineer and the design consultant engineer. Throughout the implementation of the project, a critical issue came up affecting the design of the project and the expansion of the water network to Taalabaya. In the village of Jdita there is an existing 16” Ductile Iron Pipe that springs from the wells of Jdita in order to reach several surrounding villages such as Taalabaya and Saadnayel. This pipe was constructed in 1996 and has never been put in service because the Jdita community does not want to share the water sources with other cities. The Jdita community even refused that other cities are provided with the water from the old asbestos line due to political tensions which none of the concerned stakeholders in the past have managed to address. However, Oxfam and WVL managed to receive positive feedback from authorities and communities on their approach to jointly construct a new water line as both organization’s projects are individually depending on appropriate water flow from Jdita wells. Consequently, both organizations have agreed to share both the costs and efforts and proceeded in implementation accordingly. It is very important to note that the project in all its phases was implemented in partnership and continuous coordination with the municipalities and local Bekaa Water Establishment (BWE), who is the ultimate owner of the project. The second specific objective focused on improving water conservation and healthy hygiene practices in parallel to the infrastructure work, especially focusing on the key challenges in the region and in Bekaa (according to the assessment done by WV) is that potable water is being lost as a result of poor infrastructure and poor conservation practices. This component was achieved by raising awareness about hygiene related topics to the targeted population through media campaigns and hygiene promotion sessions in 15 schools in the area. Moreover, the project built on the existing Mother Action Group (MAG) who was established and trained by WVL under its development program, to provide these awareness sessions. It is also important to note that the work in schools also included rehabilitation. Finally, in relation to Monitoring and Evaluation (M&E), an M&E plan was set at the beginning of the project to track the implementation of the project and define standards for measuring progress towards objectives. As part of this plan, a baseline was conducted, internally by WVL, to measure the initial status of the indicators in the 2 villages: Ablah and Taalabaya. Throughout the project, indicators were tracked on a monthly basis through the Indicator Tracking Table (ITT) and the number of beneficiaries (especially those participating in hygiene awareness sessions) were entered in a separate database. As the project comes to an end in March 2017, WVL is initiating the process of planning for an evaluation to assess the effectiveness of this project and its impact in the targeted villages in Bekaa. 3. Evaluation Purpose and Objectives This is an end-of-project evaluation covering the five standard dimensions of this evaluation including: relevance, effectiveness, impact, sustainability and efficiency. The main purpose is to determine what was the program effectiveness and to assess whether there are indications of sustainability. This evaluation will also allow the comparison of the findings with the available baseline results in order to highlight changes across time. The key questions in each of the evaluation dimensions are detailed below in the evaluation matrix: Relevance Key objectives: i. To identify if the program was properly designed to meet targeted needs through evaluating and assessing proposal, conceptual components and assumptions ii. To assess the relevance of the tools developed throughout the project to the area context and to the needs of the communities Key questions to be asked: i. Was the program designed in a participatory and adequate way to identify and meet the needs of the community? ii. Was the hygiene promotion component delivered in child-friendly and effective manner? iii. Were the established partnerships appropriate for the scope of work of the project? To be developed by consultant To be identified by consultant Effectiveness: Key objectives: i. To assess the progress made towards achieving the project’s goal and objectives based on the logframe, design and monitoring data Key questions to be asked: i. To which extent were each of the projects’ objectives achieved? ii. What were the main the reasons why the intervention did or did not achieve particular objectives? iii. How properly was the project executed in terms of quality and timeliness? iv. Was the frequency and duration of the trainings provided to MAGs enough? Were they able to transmit acquired information and train others? v. Did the schools and local community groups benefit from WVL’s support during this project? How? vi. What were the challenges faced during implementation and how did it affect the project’s design and execution? vii. How well did the program adapt to unexpected problems? How flexible was the program’s design? viii. How were environmental and financial risks managed? Impact: Key Objectives: i. To assess the potential impact of the program on the targeted communities Key questions to be asked: i. To what extent the community benefited from implemented activities? ii. What are the longer term consequences of achieving (or not achieving) the project’s outcomes? iii. What were the unintended positive or negative effects (direct or indirect) of the project? Sustainability Key Objectives: i. To assess the partnering approach and the participation of various partners and stakeholders in design, planning, implementation, monitoring and evaluation, and their capacity and willingness to sustain the program’s achievements ii. To assess local ownership of the project by partners (specially Bekaa Water Establishment) and to identify key factors to ensure replicability of the project components iii. To extract and analyze the lessons learnt and to provide specific, practical and actionable recommendations for expansion and further development of the program Key questions to be asked: i. To what extent is the Bekaa Water establishment able and willing to sustain the efforts after the program ends? ii. To what extent do local partner own the project? iii. Which capacities were intended to be developed and which capacities of local partners and communities were built? iv. To what extent are the partners able to advocate and influence local government in relation to WASH services? v. How involved were partners in design, planning, implementation, monitoring and evaluation of the program? vi. Was WVL’s approach successful in implementation and building towards sustainability? vii. What are the essential lessons learnt and recommendations for the further development of WVL area programming and similar projects in collaboration with the Water Establishment? Efficiency: Key questions: i. To investigate whether the resources (financial, human, and materials) have been used efficiently and effectively for the well-being of the target community. Key questions to be asked: i. Have resources (financial, human, and materials) been used efficiently for the well-being of the target community? ii. Is the staffing structure sufficient to address program objectives? 4. Evaluation Methodology The evaluation methodology is informed by the M&E plan (attached as Appendix A) specified during program design and it is aligned with EU standards and WV standards for ensuring good quality evaluation process. The key indicators to be measured are included in the M&E plan. The evaluation methods should be a combination of quantitative and qualitative methods involving the relevant project stakeholders, mainly Bekaa Water Establishment, relevant municipalities, schools, MAGs and beneficiaries at the community level. Data collection methods may include but are not limited to secondary data information (project documents will be provided by WVL), surveys with target population in villages, Key Informant Interviews (KII) and focus group discussions (FGD) with main stakeholders. For quantitative methodology specifically, sample size will be calculated based on industry standards with a confidence level of 95% and a margin of error of 5%, statistical significance will be set at 0.05 and confidence intervals should be calculated and added. It is also important to note that at baseline, the sample size was calculated based on these criteria to be 400 households assuming that the project was targeting 7,444 Households between Ablah (Lebanese: 1,579 HH, Refugees: 250 HH) and Talabaya (Lebanese: 4,095 HH, Refugees: 1,527 HH). These targets were higher than the current targets stated in page 3; this change in target numbers have been made following several assessments by the partner engineers and after discussions with stakeholders. When it comes to baseline data collection, it was done via door to door visits in a systematic manner by leaving a distance (around 200m or skipping between 7 and 10 houses) between the interviewed HH and the next HH to be interviewed. The consultant is expected to develop the appropriate methodology and tools that would best capture the objectives of the evaluation. The evaluation process also includes a stakeholder interpretation workshop which will be carried out towards the end after analysis of results to further interpret and validate the findings and extract actionable recommendations. The evaluation matrix in the section above is required to be completed by the consultant as part of developing the evaluation methodology. 5. Logistics The recruited consultant will be in charge of finalizing logistical issues related to the transportation to the different areas where data collection will be completed. As for the location for completing the different meetings (KIIs, FGDs), it will be agreed upon based on location of beneficiaries and partners (WV’s offices are an available option). WVL will support in coordinating the meetings with local partners. 6. Products The expected deliverables throughout the process of evaluation include: 1. Evaluation Methodology (including Evaluation Matrix) 2. Budget Breakdown 3. Validated and tested data collection tools 4. Thematic in-depth analysis of qualitative data 5. Data sets (software outputs) 6. Notes and transcription of KII and FGDs 7. Tabulation of results including descriptive analysis, associations and comparison with baseline data 8. Evaluation Report (based on Template provided by WVL) 9. Power Point presentation of the findings for the purpose of the interpretation workshop and the WASH sector coordination meeting. 10. 2 pager fact sheet summary (in English and Arabic) The consultant is also expected to present and co-facilitate a stakeholder interpretation workshop which will join together WV staff and partners to present the main findings, reflect on the lessons learned and recommendations and develop key action points for future programming. This process will feed into the final evaluation report. Finally, the results of the evaluation will be also presented by the consultant in WASH sector coordination meeting in Bekaa. All collected data is legally owned by WVL and the consultant is expected to hand over all data sets and notes of the interviews or FGD to the organization. The Consultant shall maintain in confidence and protect all information provided to him/her by WVL, its employees, and beneficiaries. The consultant may only disclose the extent necessary to perform the evaluation. 7. Qualifications of the consultant The consultant should have the following competencies and experience: • At least 5 years of progressive proven experience in similar studies and in using both qualitative and quantitative research methodologies and data analysis; • Previous evaluation experience in relation to WASH projects (to be provided within the CV); • Extensive professional experience in the design and implementation of outcome and impact evaluations; • Good analytical and critical thinking; • Proven knowledge and experience in applying participatory research methods and tools; • Advanced degree in relevant field (e.g. Public Health, Epidemiology or any related technical field); • Good understanding of the Humanitarian work especially the development field; • Familiarity with civil society and Non-Governmental Organization (NGO) engagement; • Ability to work on tight schedules with minimal supervision; • Good English and Arabic speaking and writing skills. 8. Budget The consultations fees available for this end of program evaluation will include transportation, logistics and accommodation fees identified by the consultant, as well as recruitment of data collectors/data entry clerks and data collection, data entry, translation fees and report writing. The accommodation fees will include venue rental, meals and coffee breaks for FGDs. A budget breakdown needs to be submitted by the consultant. 9. Duration of consultancy and application WVL has set aside 3 months (July until September 30, 2017) for this assignment. Bids should include a detailed proposed work-schedule with specific tasks and should also incorporate a budget that outlines all relevant costs that will be associated with this evaluation. Candidates are expected to state how much time they will need to start the assignment and how much time they need to conduct every task. Selected candidates are also expected to abide by the deadlines and the conditions for deadlines specified within the contract/agreement. Interested individuals and consultancy firms should send in their applications no later than 16.00 hours GMT on June 26, 2017 including a detailed Technical and Financial Bids with the following documents: • Proposed methodology (including evaluation matrixm, timeline and budget), • Curriculum vitae and/or resume, • The names and addresses (including telephone and e-mail) of two non-related referees, • Sample of previous work (evaluation reports related to the topic). Technical and Financial Bids should be in English and should not exceed 5 pages and should be submitted in two separate Sealed Envelopes. The top right-hand side of the envelopes must be clearly marked with the name of the consultant and the subject line “EU WASH Evaluation” and sent to the below address: “Main National Office address: Villa Siniyora, Mountazah, Mansourieh; Office Phone: 961-4-401-980 Mailing address: World Vision Int'l Lebanon P.O.Box 55355, Sin el Fil, Lebanon” Technical bids should also be sent by email with the name of the consultant and the subject line “EU WASH Evaluation” to the following email address consultant_lbn@wvi.org. Only short-listed candidates will be contacted. In case the Consultant is not registered with Ministry of Finance, a deduction of 7.5 % should always be considered. N.B: Selected consultant will be provided with all additional documents required to carry out the work