Baseline Evaluation

1. Project and Evaluation Summary Program/ Project Promoting non-violent communication in Marjeyoun ADP Evaluation type Baseline Evaluation Evaluation Purpose To provide baseline data for the indicators of the suggested project goal and outcomes Primary Methodologies - Surveys with parents, teachers, children, youth and local stakeholders - Focus Group Discussions (FGDs) with children and adults Evaluation duration 8 Weeks: July, 06, 2015 until August, 24, 2015 Anticipated evaluation final report release date August 24, 2015 2. Description of project being evaluated Title of Project Promoting non-violent communication Operational Period for the project May 15, 2015- September 30, 2015 (Mid FY15-FY16) Project Target Population Direct target population: 2,800 children aged 9-14 years old 50 youth aged 15-24, 100 parents 50 teachers.    The goal of the project is to improve the protective environment around children by engaging children, youth, parents, teachers, school management and community stakeholders of Marjeyoun Area Development Program (ADP). WVL will work with all the segments of Marjeyoun society to promote non violent communication. Parents will be provided with opportunities to develop and disseminate knowledge about positive ways of discipline. Teachers will be provided with adequate resources to use alternative ways of discipline with students at schools. Children will be provided with opportunities to participate in relevant interactive activities that promote non violent communication and local stakeholders will be supported to reactivate/create and equip children/ youth clubs to enable children/ youth to spend time together and participate in active discussions. The project will be directly targeting around 2,800 children aged 9-14 years old, 50 youth aged 15-24, 100 parents and 50 teachers.   3. Evaluation target audiences The following parties are involved and have a major interest in the baseline evaluation process:   WVL: The data generated from the baseline document will allow WVL and Marjeyoun ADP team to have an in depth assessment of the indicators’ status prior to the intervention so as to measure against at project-end evaluation and assess impact of the project.   Community members: Being the key players in this project, the baseline data is as important for the community members as it is for WVL, as communities will have a view of the current situation of their children’s wellbeing. The data generated from the baseline document will highlight the status of the indicators prior to the intervention.   Support Office (SO) – World Vision Malaysia: SO has high interest in the results of the baseline, as these results, when compared to the project evaluation results later on, would show the impact of the project and thus the impact of the donors’ contribution. 4. Evaluation type The evaluation conducted is a baseline evaluation which will guide the implementation of the project. According to Learning through Evaluation with Accountability & Planning (LEAP), all relevant indicators in the M&E plan must have a baseline established within the first year of implementation, from which changes over time can be tracked. 5. Evaluation purposes and objectives As previously stated, the purpose of the baseline data collection is to provide relevant quantitative information that informs the project’s implementation through establishing baseline data for indicators. The set indicators will be measured against the baseline figures at different time intervals throughout the implementation of the program depending on the frequency of measurement set in the M&E plan. 5.1 Baseline Evaluation Matrix This Baseline evaluation matrix is required to be completed by the consultant as part of developing the evaluation methodology. Goal/ Outcome Indicators Tool Data Source Sample Size Goal: Children, youth, parents, teachers, school management and community stakeholders of Marjeyoun ADP are engaged to improve the protective environment around children Proportion of parents or caregivers who feel that their community is a safe place for children Caregiver survey: Child protection and participation module     Community and family behaviour creates a protective environment for children FGDs with community members     Outcome: Children, youth, parents, teachers and local stakeholders of Marjeyoun ADP have the knowledge and tools to promote non violent communication among them Children feel supported within families and communities FGDs with children     Children, youth, parents, teachers and local stakeholders' knowledge in regards to non violent communication is improved Surveys children, youth, parents, teachers and local stakeholders       6. Evaluation methodology The baseline evaluation methodology is informed by the M&E plan specified during project’s design and it is aligned with WV’s guidelines and standards for ensuring good quality evaluation process. Establishing the baseline for the goal & outcome indicators will rely on primary data collection through surveys and FGDs with relevant stakeholders and target beneficiaries. The collected primary data will be validated after which it will be analyzed and presented in a formal report to be later disseminated to the partners and concerned WVL staff. The consultant is expected to develop the appropriate methodology and tools that would best capture the objectives of the evaluation baseline. 6.1 Proposed Methodology The evaluation will be following but not limited to the below methodologies: -       Quantitative evaluation targeting children, youth, parents, teachers and local stakeholders -       Qualitative evaluation with children and adults 6.2 Data collection and analysis For the validation meetings and qualitative data collection, it will be completed by the consultant as a facilitator, and a note taker will be present during the FGDs. For the FGDs, thematic analysis of the data will be completed in order to discern patterns and come up with conclusions that would reflect the views and the perceptions of the participants. The information collected during the FGD will be used to triangulate the quantitative data for a holistic understanding of the evaluation results. The details of the data collection are shared below in section 7 “Authority and Responsibility”. For the quantitative data collection, it will be completed by selected data collectors hired by the consultant and supervised by him/her (covered within the budget). Appropriate statistical tests will be employed pending the type and measurement scale of the outcomes and independent variables (e.g., continuous, nominal, or ordinal). For instance, ANOVA will be used for comparisons between means of scores (probably knowledge and attitudes); chi-square test will be used to determine the significance of differences in percentages;  spearman rank correlation coefficients will be used to examine the relationship between different scores used; non-parametric equivalent tests may be used in case the data is not normally distributed etc. In depth statistical analysis is required; data should be analyzed with with a confidence level of 95% and statistical significance set at 0.05 and confidence interval should always be calculated and added to all the analysis done. 7. Authority and responsibility Baseline phase Role Responsibilities and tasks Planning   DME Officer Develop baseline evaluation TOR based on LEAP standards Develop baseline work plan Follow- up on recruitment of consultant Provide feedback on baseline evaluation methodology  and matrix Follow up with WV Canada for review of baseline evaluation TOR Consultant Develop and submit the Baseline evaluation methodology (sample selection, quantitative & qualitative methodology) Finalize baseline evaluation matrix Develop qualitative data collection tools based on WVL’s standards Gather feedback on tools from baseline advisors Conduct pilot testing of the tools Area Manager Provide input on TOR Provide financial approval for consultant recruitment Support with the coordination of the baseline evaluation if needed Project Coordinator Provide input on TOR Coordinate and communicate with local implementing partners as needed Protection specialist Provide technical input and feedback on TOR Provide guidance on tools developed by the consultant Data collection and analysis Consultant Recruit and train data collectors Conduct quantitative and qualitative data collection (supported with data collectors where relevant) Supervise and assure the quality of quantitative data collection Recruit and train data entry clerk (data analyst) Set up an analysis plan for quantitative data analysis Conduct in depth data analysis and interpretation of results provided by data analyst Provide weekly feedback to WVL team Data analyst Conduct data entry for quantitative data Conduct data analysis as per the quantitative data analysis plan Report to evaluation consultant Data collectors Carry out quantitative and/ or qualitative data collection as relevant Report to evaluation consultant DME officer Provide feedback on data collection tools Follow up with the consultant on data collection and analysis Provide feedback on data analysis Protection Specialist Technical support in finalizing data collection tools Technical input for the analysis, synthesis of data and drafting of recommendations Project coordinator Facilitate access to community members, schools and local stakeholders/ partners Reporting, validation, interpretation and follow up Consultant Draft the Baseline Evaluation report using LEAP template Prepare 2 power point presentations of the main findings (2 versions: highly technical and lay terms in Arabic) Facilitate community validation workshop and support WVL team in validating the results with children and their families Co-facilitate stakeholder interpretation workshop with DME officer and discuss findings with relevant WV staff and partners. Finalize the Baseline Evaluation report based on feedback from WVL team, stakeholders and community members Submit a 2 pager summary fact sheet to be disseminated to relevant stakeholders   DME officer Provide feedback on draft evaluation report Share internally and with WV Canada draft version of the report and follow up on the feedback received. Share internally and with WV Canada final version of the report and the 2 pager summary fact sheet.   Protection Specialist Provide feedback on draft evaluation report and recommendations   Area Manager Provide feedback on draft evaluation report and recommendations   Project Coordinator Provide feedback on draft evaluation report and recommendations Facilitate access and dissemination to community members and different stakeholders 8. Team advisors Role Responsibilities and Tasks DME Manager Ensure quality of Baseline Evaluation ToR and data collection tools Ensure quality of the data collection and analysis Review the baseline evaluation report Urban Learning Hub Leader Review the evaluation ToR document Support technically in finalizing data collection tools Support technical in data analysis Review the baseline evaluation report and support in drafting recommendations. WV Malaysia Review the Baseline evaluation ToR Ensure compliance with WV Canada standards and requirements Review the Baseline evaluation report and support in drafting recommendations 9. Logistics The recruited consultant will be in charge of finalizing logistical issues related to the transportation to the different areas where data collection will take place. As for the location for completing the different meetings (FGDs, validation meetings, etc.), it will be in WVL office in Marjeyoun Area Development Program (ADP). The paragraph below will explain in details how the data collection and validation will take place: Data collection: WVL staff of Marjeyoun ADP will be supporting with the coordination of the different meetings/surveys with beneficiaries and project’s stakeholders. Validation meetings: The validation meetings will be completed at Marjeyoun ADP office. The final presentation of results will take place in one meeting where WVL and partner’s staff will be invited to participate in the discussion and learn about the evaluation results. 10. Products The end product of the baseline will be the baseline evaluation report. In the preparation of the final report, the consultant is expected to provide the reader with, as much as possible, accurate sources of information and conclusions.  In fact, all findings and statements must be backed by existing data and/or data based on approved methodology for sampling, data collection, and analysis. Expected deliverables throughout the process of evaluation include: Copy of training package and materials used for the baseline evaluation team; Baseline evaluation Methodology (including Baseline Evaluation Matrix) Survey guideline - including questionnaires, qualitative tools, sampling methodology, sample selection criteria, data collection and analysis protocols; Copies of all quantitative data (datasets, output analysis, tabulation of results including descriptive analysis and associations...) and qualitative data (thematic in-depth analysis...) Final Baseline Evaluation Report in English (as per the World Vision (WV) Evaluation Report Guidelines to be provided); and in hard copy and electronic forms;   Final Baseline Evaluation Report Structure:       i.        Cover page      ii.        Table of contents     iii.        Acknowledgements     iv.        Affirmation      v.        Glossary     vi.        Introduction Executive summary Evaluation Background Methodology Limitations Findings Conclusions and Recommendations Lessons Learned from the baseline evaluation process Appendices: -       To include copies of all tools, list of enumerators, survey timeline including all KII and FGD participants and discussion transcripts (as many pages as necessary- please reference the annexes in the report, but include them in a zip file as separate documents). -       A 2-page evaluation summary fact sheet. This is to include relevant findings from the baseline evaluation, key points and recommendations. -       An Indicator Summary sheet, giving status of all indicators measured in the surveys -       A power point presentation in English including methodology, main findings, recommendations, best practices and lessons learnt. 11. Budget The baseline evaluation process will be conducted by an external consultant.  A financial bid detailing the funding needed to conduct this study should include transportation, accommodation and data collectors/data entry clerks fees identified by the consultant. 12. Qualifications of the consultant The consultant should have the following competencies and experience: ü  At least 5 years of progressive proven experience in similar studies and in using both qualitative and quantitative research methodologies and data analysis; ü  Previous evaluation experience of protection related projects (to be provided within the CV); ü  Extensive professional experience in the design and implementation of outcome and impact evaluations; ü  Good analytical and critical thinking; ü  Proven knowledge and experience in applying participatory research methods and tools; ü  Advanced degree in relevant field (e.g. Social Sciences, Public Health, Epidemiology or any related technical field); ü  Good understanding of the Humanitarian work especially development field; ü  Familiarity with civil society and Non Governmental Organization (NGO) engagement; ü  Ability to work on tight schedules with minimal supervision; ü  Good English and Arabic speaking (preferable) and writing skills. 13. Duration of consultancy WVL has set aside 8 Weeks (July 06, 2015 till August 24, 2015) for this assignment. Selected candidates are also expected to abide by the deadlines and the conditions for deadlines specified within the contract/agreement. TASKS TIMELINE Review of TOR 1st week Design of the evaluation methodology 1st week Develop all necessary data collection tools including KIIs, FGDs Surveys... (These tools will be reviewed by the Design, Monitoring and Evaluation Team, as well as Technical Specialist for approval) 2nd week Use the appropriate method to calculate the sample size and carry out sampling 3rd week Recruit and train enumerators, supervisors and facilitators and oversee pilot testing 3rd week Supervise the quantitative and qualitative data collection while insuring the quality of data 4th - 5th  week Design the database for data entry 5th week Supervise data entry and quality data management 6th week Carry out data analysis as well as report writing 6th- 7th week Present main findings and a debriefing of the evaluation to the Design, Monitoring and Evaluation team as well as key stakeholders 7th week Present the draft report in a review meeting and incorporate the feedback and compile a final report 8th week   In the preparation of the final report, the consultant is requested to provide the reader with, as much as possible, accurate sources of its information and conclusions.  In fact, all findings and statements must be backed by existing data and/or data based on approved methodology for sampling, data collection, and analysis. Interested individuals should send in their applications no later than 16.00 hours GMT 24th of June, 2015 including a detailed Technical and Financial Bids with the following documents: -       Proposed methodology (including a revised timeline and budget) -       Curriculum vitae and/or resume -       The names and addresses (including telephone and e-mail) of two non-related referees -       Sample of previous work (evaluation reports related to the topic) (Technical and Financial Bids should be in English and not more than 3 pages and should be submitted in two separate sealed envelopes). Important: If application is sent by email, the subject line should clearly indicate “Promoting non violent communication in MR ADP baseline” If sent by post or delivered by hand, the top right-hand side of the envelope must be clearly marked “Promoting non violent communication in MR ADP baselime”. Only short-listed candidates will be contacted. N.B.: Selected consultants will be provided with all additional documents required to carry out the work.