Health Programme- Consultancy

Introduction   World Vision Lebanon (WVL) has been implementing “Maternal and Child Health and Nutrition” (MCHN) Program since 2010 at the level of Area Development Programmes (ADPs) and at the national level. It aims to improve the health of the mothers and children (under 5) in Lebanon by addressing the root causes negatively affecting their health. The programme was implemented with full participation from the community which was mobilized to disseminate health messages and conduct outreach activities. Our purpose is to evaluate this programme from the beginning of implementation until the end of 2014. 1.   Evaluation Summary   Programme/Project Health Program Programme Phase N/A Evaluation Type End of Program Evaluation Evaluation Purpose The purpose of the evaluation is to assess the change brought upon in the different Lebanese regions by WVL’s MCHN programming. The current evaluation will highlight the MCHN indicators with an intentional focus on WVL’s structure and managerial approach. Primary Methodologies Qualitative and quantitative methodologies Evaluation Start and End Dates April 2015- June 2015 Anticipated Evaluation Report Release Date June 30, 2015     2.   Description of Programme or Project Being Evaluated   During Fiscal Year (FY) 2010, a health assessment (which also served as a baseline) was completed by WVL in 4 Area Development Programs (ADPs) North ADP , Central Bekaa ADP, Zahle ADP, and West Bekaa ADP as part of a project entitled “Health Jump Start Initiative” (HJSI) in order to identify the issues affecting MCHN in Lebanon. HJSI was launched in 2009 as a global partnership initiative, and in Middle East and Eastern Europe Region (MEER) it was implemented in 7 National Offices (NO). The main purpose was to build WV internal and partner capacity for MCHN quality programming and scale-up. It provided countries with the key leadership required to develop technically sound and appropriate strategic plans, as well as provide the champion required to work with operational and technical staff in the NO and ADPs to ensure that MCHN issues are effectively addressed.   The results of the assessment revealed that women of childbearing age have low MCHN knowledge resulting in poor practices negatively affecting their health and the health of their children: poor antenatal care, poor feeding practices, poor nutrition and inappropriate care for sick children and more. The assessment also highlighted the weakness of the Primary Healthcare Centers and their inability to have an influential role on the MCHN indicators in the areas they cover. Adding to that, review of MCHN policies in Lebanon indicated that the mechanisms for monitoring are not functional, and the policy for Baby Friendly Hospital Initiative (BFHI) is not activated. Based on the above, the Health program in WVL designed an intervention in order to improve MCHN for the mother and children (under 5) of Lebanon by targeting the root causes negatively affecting their health. The 4-year program was divided into activities at the local ADPs level and at the national level.   At the level of ADPs, the program was initiated in 4 ADPs North, Central Bekaa, West Bekaa, and Zahle where a series of capacity building workshops in relation to MCHN were conducted for WVL Staff and health workers from different partner organizations.  The Health Project Coordinator in each ADP is responsible for the implementation of the project; the coordinator reports directly to the ADP manager and consults technically with the Health Specialist based in the NO. Within the ADP programme, groups of mothers forming “Mother Action Groups” (MAGs) were mobilized in order to conduct outreach activities in their communities for the enhancement of MCHN status. In parallel, the services of Primary Health Care Centres (PHCs) within the ADPs and the capacities of healthcare providers were strengthened through MCHN capacity building sessions. PHCs were also linked to the MAGs for increased collaboration and sustainability of the MCHN outreach activities beyond the lifetime of the project. Additional funding was acquired and the same project model was rolled out in 3 additional ADPs, thus increasing the coverage of the health program to include most of the regions in Lebanon: West Bekaa ADP, Zahle ADP and Central Bekaa ADP in Bekaa (East of Lebanon), Becharreh ADP and El Sahel ADP in the North of Lebanon, Bent Jbeil ADP in the South of Lebanon, and Beirut Urban ADP in the capital Beirut.   All of the ADPs that were included in the health program followed the same log frame during implementation (similar at the goal and outcome level and slightly different at activities’ level) which allowed more consistent programming and condensed WVL’s influence at the national level.   Goal: Children of North /El Sahel /Central Bekaa /West Bekaa /Zahle /Beirut /Bent Jbeil ADPs enjoy good health   Outcome 1: WVL staff and partners facilitate the implementation of contextualized and sustained evidence-based mother and child health and nutrition programming   Outcome 2: Mothers adopt healthy lifestyle practices to improve maternal and child health and nutrition   Below is an overview of the operational dates of the programme in different ADPs and the number of targeted population:   Area Development Programme Date of Programme Initiation Number of Mother Action Groups (MAGs) Number of targeted mothers Number of targeted children Central Bekaa 2009 2 640 2,390 West Bekaa 2009 2 690 3,133 Zahle 2009 1 465 1,830 North (Besharreh) 2009 2 600 2,000 Al Sahel 2011 1 600 2,000 Bent Jbeil 2011 1 400 1,000 Beirut 2011 1 1,195 3,690   Finally, at the national level WVL built a partnership with national health sector stakeholders (i.e. Ministry of Public Health (MOPH) and World Health Organization (WHO)) and facilitated the establishment of Infant and Young Child Feeding (IYCF) coalition which had the following objectives:   Enhancing collaboration among national health stakeholders Re-activation of the BFHI policy and ensuring adequate implementation and monitoring For more information about MCHN programme, a concept note, Detailed Implementation Plans (DIP), summary of main activities implemented, outcome level indicators, Indicator Tracking Table (ITT) as well as semi-annual and annual reports are all embedded in the “Documents” section in this TOR. 3.   Evaluation Type   The current evaluation is an end of program evaluation. This evaluation will allow the comparison of the findings with the baseline results in order to highlight effectiveness and changes across time. It would also measure if the initial assumptions held true and if the approach is worth replicating in other areas where WVL is operating. The evaluation aims to analyze the health program structure and implementation approach (which would include but will not be limited to the analysis of the WV partnership approach) and to identify best practices and lessons learned that would serve as a practical example for the development of the Technical Approach (TA) which is the sub-strategy of the Health Programme. 4.   Evaluation Purpose and Objectives   The purpose of the evaluation is to assess the change in MCHN outcomes brought upon in the different Lebanese regions by WVL’s MCHN programming. The current evaluation will highlight the MCHN indicators stated in the program’s Monitoring and Evaluation (M&E) plan with an intentional focus on WVL’s structure and managerial approach.   The objectives of the evaluation are as follows: To identify if the program was properly designed to meet targeted needs through evaluating and assessing program theory, logic, conceptual components and assumptions To assess the progress made towards achieving the Health program’s goal and objectives based on the log frame and design To assess the potential impact of the program on the targeted communities To investigate whether the resources (financial, human, and materials) have been used efficiently and effectively for the well-being of the target community. To assess the sustainability of the program, particularly focusing on the partnering approach and the capacity and willingness of partners to sustain the program’s achievements To provide specific, practical and actionable recommendations for health program integration within the ADPs across the sectors.   These Evaluation Terms of Reference specify five primary objectives, namely to assess the a) relevance, b) effectiveness, c) impact, d) efficiency, and e) sustainability of the project. It is expected that the evaluator will divide these objectives into three broad stages of the project cycle: project design, implementation and management, and achievements. 5.   Evaluation Methodology   The evaluation methodology is informed by the M&E plan specified during program design and it is aligned with WV’s guidelines and standards for ensuring good quality evaluation process. The assessment methods should be a combination of quantitative and qualitative methods with the relevant Health program’s stakeholders, partners and beneficiaries. Data collection methods may include but are not limited to secondary data information, general observations, surveys, Key Informant Interviews (KII) with main stakeholders/partners, and focus group discussions (FGD). The consultant is expected to develop the appropriate methodology and tools that would best capture the objectives of the evaluation, based on the following assessments:   The baseline assessment was conducted in 3 ADPS in 2010: West Bekaa, Central Bekaa and the North (Besharreh). The tools used consisted 2 surveys administered among pregnant women and mothers of children under 2. The convenient sample included 216 pregnant women and mothers of 276 children aged 0-24 months selected from thirty villages in West Bekaa, Central Bekaa and the North. Furthermore, interviews were conducted with 16 heads of health care centers from the North, Central Bekaa and West Bekaa ADPs; 6 obstetricians/gynecologists; and 10 pediatricians/ generalists practicing in health care centers.   Baselines were also completed in Bent Jbeil and Nabaa ADPs as well. In Nabaa ADP, quantitative data collection was completed in 2011, using questionnaires administered to 279 students (RCs and non-RCs), 279 parents (of RCs and non-RCs) and 129 teachers to measure indicators related to the 3 projects, Education, Health, and Sponsorship. The sample was selected from 6 schools in the area.   In Bent Jbeil ADP, a baseline was completed in 2013, in order to measure the initial status of the projects’ indicators prior to the intervention. Primary data was collected using survey tools that were administered to two target groups: 271  mothers and 230 youth aged 12-18 years old) from 8 villages in BJ ADP using the self-administration methods. In addition, health Facility Evaluation tool was administered to 9 PHCs in the area. Finally, 56 FGDs were conducted with seven target groups (Men, Women, Girls, Boys, Health committee, cared for committee and sponsorship committee) representing the 9 villages of BJ ADP  to collect qualitative data. Out of the 56, 6 FGD were used to measure the health indicators.     Mid-term evaluation: In late 2010 the MEERO conducted mid-project progress reviews in all JSI implementing countries in the region (seven countries).  A consultant was hired to participate in two country reviews, and to consolidate a regional report as closure to the process. In Lebanon the review was organized by the Health Team Manager, and assisted by the WVI Health team Operations Director.  The review was conducted over a five day period, from 30 May through 3 June 2011.  The review process entailed project and related document review, key informant and focus group interviews.  Nearly 20 interviews were conducted, with WV/L staff, MOPH, IOCC, MAGs, PHC and Hospital staff.  The final report was based primarily on qualitative information, with few quantitative points (achievement against activities, budget, etc.).    Another qualitative assessment was conducted earlier in March 2014, using the following methods:   FGDs with MAGs members to discuss the impact of the health program on their lives and on their communities; FGD with the Health coordinators to analyze the activities’ implementation and the structure of the team; KIIs with Health program manager, previous Ministry Quality manager, WVL’s Operations director, ADP managers where MCHN was implemented. The purpose of the KII was to identify the impact of the health programming on the community and on WVL’s processes based on the approach adopted by the health program; KIIs with the national level partners (MoPH, WHO, International Orthodox Christian Charities (IOCC), Lebanese Association for Early Childhood Development (LAECD), Sagesse and Balamand Universities, etc.) , based on the DPA (Development Programme Approach) partnership health check tool.    Proposed Methodology The evaluation will be following but not limited to the below methodologies:   Quantitative evaluation targeting pregnant women and mother of children under 2 years based on the baseline tools which are adapted to capture all the changes in the programme Qualitative evaluation with MAGs, health coordinators and partners (at the local/ADP level and the national level)   Data collection and analysis For the validation meetings and qualitative data collection, it will be completed by the consultant as a facilitator, and a note taker will be present during the FGDs and the KIIs. For the FGDs and the KIIs, thematic analysis of the data will be completed in order to discern patterns and come up with conclusions that would reflect the views and the perceptions of the participants. The information collected during the FGD will be used to triangulate the quantitative data for a holistic understanding of the evaluation results. The details of the data collection are shared below in section 8 “Authority and Responsibility”.   For the quantitative data collection, it will be completed in each of the areas by selected data collectors hired by the consultant and supervised by him/her. Appropriate statistical tests will be employed pending the type and measurement scale of the outcomes and independent variables (e.g., continuous, nominal, or ordinal). For instance, ANOVA will be used for comparisons between means of scores (probably knowledge and attitudes); chi-square test will be used to determine the significance of differences in percentages;  spearman rank correlation coefficients will be used to examine the relationship between different scores used; non-parametric equivalent tests may be used in case the data is not normally distributed etc. Data will be analyzed with a confidence level of 95% and a statistical significance set at 0.05 and confidence intervals should be calculated and added. 6.   Limitations   Baseline measurements in the different ADPs were completed at different timings which limits the ability to compare the data between the regions. The selected methodology for baseline was convenient sampling and there were no control sites to be able to measure program effectiveness. Different methodologies and sampling procedures were used for each of the baselines, thus affecting comparability between the regions, and affecting the methodology to be used for the evaluation. MAGs in ADPs were established progressively and throughout different years as the project expanded, so the progress and impact of the project differs between ADPs. Another limitation is that project monitoring was not very strong; there was inconsistent data collection through the life of the project. 7.   Logistics The recruited consultant will be in charge of finalizing logistical issues related to the transportation to the different areas where data collection will be completed. As for the location for completing the different meetings (KIIs, FGDs, validation meetings, etc.), it will be in WV’s offices in the ADPs and at the national office, based on the needs. The paragraph below will explain in details how the data collection and validation will take place:   Data collection: The health team in WVL will be supporting with the coordination of the different meetings/surveys with beneficiaries, partners and project’s stakeholders.   Validation meetings: the validation meetings at the ADP levels will be completed at the respective ADP offices. For the national partners, the validation meeting will be held at the National Office.   The final presentation of results will take place in one meeting where staff and partners will be invited to participate in the discussion and learn about the evaluation results. 8.   Products   Expected deliverables throughout the process of evaluation include: Evaluation Methodology (including Evaluation Matrix) Validated and tested data collection tools Thematic in-depth analysis of qualitative data Copy of raw data – both qualitative and qualitative raw data Tabulation of results including descriptive analysis and associations Draft of the Evaluation report Power Point presentation of the findings 2 pager summary fact sheet – showing, amongst other things, the progress against all indicators in the project design 9.   Budget Financial bids should be submitted by the Consultant, and specific details will be worked out based on discussions and plans between the Consultant and the DME Officer. The funding available for this end of program evaluation should include transportation and accommodation fees identified by the consultant. 10.       Qualifications of the consultant   The consultant should have the following competencies and experience: At least 5 years of progressive proven experience in similar studies and in using both qualitative and quantitative research methodologies and data analysis; Previous evaluation experience of health related projects (to be provided within the CV); Extensive professional experience in the design and implementation of outcome and impact evaluations; Good analytical and critical thinking; Proven knowledge and experience in applying participatory research methods and tools; Advanced degree in relevant field (e.g. Public Health, Epidemiology or any related technical field); Good understanding of the Humanitarian work especially the development field; Familiarity with civil society and Non-Governmental Organization (NGO) engagement; Ability to work on tight schedules with minimal supervision; Good English and Arabic speaking and writing skills. 11.       Duration of consultancy   WVL has set aside 2 months and a half (April 22nd, 2015 till June 30th, 2015) for this assignment (Taking into consideration all public holidays during these months). Bids should include a detailed proposed work-schedule with specific tasks and should also incorporate a budget that outlines all relevant costs that will be associated with this evaluation. Candidates are expected to state how much time they will need to start the assignment and how much time they need to conduct every task. Selected candidates are also expected to abide by the deadlines (specified in Appendix A) and the conditions for deadlines specified within the contract/agreement.   Interested individuals and consultancy firms should send in their applications no later than 16.00 hours GMT 10th  of April, 2015 including a detailed Technical and Financial Bids with the following documents: proposed methodology (including a revised timeline and budget), curriculum vitae and/or resume, the names and addresses (including telephone and e-mail) of two non-related referees, sample of previous work (evaluation reports related to the topic). Technical and Financial Bids should be in English and not more than 3 pages and should be submitted in two separate sealed envelopes.   Important: If application is sent by email, the subject line should clearly indicate “Health programme evaluation”. If sent by post or delivered by hand, the top right-hand side of the envelope must be clearly marked “Health programme evaluation”. Only short-listed candidates will be contacted. N.B: Selected consultant will be provided with all additional documents required to carry out the work.   Timeline     Responsibilities Week 1 Week 2 Week 3 Week 4 Week 5 Week 6 Week 7 Week 8 Review material and documents provided X               Develop the evaluation methodology X               Receive feedback from WV team on the methodology   X             Finalize evaluation methodology   X             Develop data collection tools     X           Receive feedback from WV on tools     X           Finalize and pilot test the tools         X         Qualitative & quantitative data collection conducted       X X       Conduct data cleaning & analysis         X X     Meet with WV team to provide input on analysis             X   Draft the Evaluation report             X   Prepare a power point presentation of the main findings               X Present finding to WV staff and partner                 X Finalize the Evaluation report based on feedback from WVL team                 X