These terms of reference (TOR) describe the methodological requirement for the baseline, midterm and final evaluations and to outline the conditions and responsibilities of the consultant(s) who will undertake in Guinea-Bissau these evaluations for a newly awarded 4-year McGovern-Dole project. CRS will engage an independent consultant, following a competitive international bidding process. Assuming a satisfactory work product, the same consultant will be hired for the midterm and final evaluations, thus CRS requests bids for all three evaluations, with a separate budget broken out for each.
Please note this ToR and its annexes are subject to donor approval, and thus may change before contract signing. Please also note this is a partial posting only of the ToR. For the full ToR, and relevant annexes, please send an email to SN_HR@crs.org.
The MeREECE program aims to strengthen the education system in Guinea-Bissau and improve literacy of school-aged children in the regions of Oio, Cacheu,, Quinara,, Bafata and Gabu. CRS will work with its partners, Caritas Guinea-Bissau and Plan International to fully implement the project in 350 elementary schools to reach 199.539 individuals in the five proposed regions.
4.1.2. Data Collection Sources and Ideal Sample Sizes
Please see section 3, sampling sub-section, in Annex 1.
4.2. Data Processing and Analysis Procedures
To meet expectations as to how evaluation data can be useful, CRS will engage the recruited evaluation team to determine how to ensure data quality through a quality control system. Data analysis should be descriptive in that it will provide trends (central and dispersion trends, rate, percentage) in the achievement of results at each measurement period. Because these evaluations will employ representative samples, the significance of the estimators (indicators) will be verified using inferential statistical methods.
Selection of the Evaluation Team
All evaluations will be conducted by an external independent consulting firm or individual evaluator in coordination with CRS’s regional and national MEAL technical advisors and the CRS Program Quality Department. CRS will advertise the ToR for the baseline, midterm and final evaluations together and recruit one consultant or firm to conduct all three studies. The firm will be selected following a competitive, transparent and independent procurement process conducted by CRS procurement team.
The proposal will be assessed using the following criteria:
- Soundness of the technical approach;
- Practicality of the methodologies proposed;
- Cost Efficiency and;
- Evaluation consultant qualifications (see below)
The expected consultants and/or firm should have strong experience with education programming and evaluations including, in the domains of health and nutrition and school feeding programs. The team should at least be composed of a lead consultant and an associate consultant with the profile below:
- Advanced degree in social sciences or any related background
- A minimum of 5 years of experience in conducting quantitative and qualitative impact and performance evaluations in similar complex international development programs.
- Experience in conducting research and evaluation of US government international development programs. Preference will be given to those who have experience in USDA McGovern-Dole Food for Education programs.
- Experience in designing or evaluating education, literacy and school feeding programs.
- Experience in designing, using and analyzing international literacy assessments such as PASEC and/or ASER.
- Experience in qualitative evaluation techniques such as key informant interviews, focus group discussions, observations, and case studies.
- Experience in quantitative data collection, statistics/econometrics such as randomized control trials, propensity score matching, regression discontinuity, sample size selection, design effects, questionnaire design, etc.
- Experience evaluating programs in West Africa, preferably Guinea-Bissau.
- Ability to communicate, read, and write fluently in English, Portuguese and other languages as appropriate.
- Willingness to work in remote areas without electricity and running water.
- MSC in statistics, Program Evaluation and Measure, international development or related background.
- Experience and knowledge in the use of electronic data collection tools in evaluations
- Background in statistics and evaluation methods that use counterfactual and experimental/quasi-experimental approach, cohort analysis experience will also appreciate.
- Experience in data processing, analysis and reporting
- Strong proficiencies in English and Portuguese are required
All deliverables should be completed in English (and data collection tools must also be in Portuguese).
Deliverables include the following:
- Work plan (including evaluator responsibilities for identifying, interviewing, contracting, training and overseeing enumerators).
- Sampling plan, including if the sample sizes will differ from Annex 1.
- Instruments, data collection manual, and training materials for enumerators (i.e., focus group guides, key informant interview guide, observation checklist).
- Quality Assurance Plan (including training of enumerators and weekly check-ins during data collection.
- Conduct interview with USDA (it is expected USDA will facilitate this exercise by providing the contact person and the means of interview)
- Data sets with accompanying codebook/data dictionary (original paper and/or electronic as well as final, clean electronic data sets with syntax).
- At baseline only, a 10-page preliminary report, suitable for presentation to USDA, 6 weeks after the end of data collection. The report will only contain:
- Draft Report with one round of edits from CRS and another subsequent round from USDA
- Final Report with the following sections:
- A two to four-page summary document, with easily accessible graphics, highlighting the project’s key successes, for sharing with a larger audience
- Presentation of final evaluation to stakeholders
- A webinar of key findings and lessons learned for CRS globally and USDA (if requested).
CRS and implementing partners will provide to consultant team preparatory, logistical assistance and the following documents.
- MEAL documents and tools such as the project’s: results framework, evaluation plan, key performance indicators list, theory of change, learning agenda, existing evaluation reports and case studies (and other available documents as needed)
- Access to a database that includes all 350 schools targeted with demographic and geographical information
- Secondary data available to further understand educational context in Guinea-Bissau;
- Compilation of reference documents (project proposal, periodic reports, etc.)
- Contact details of stakeholders in the implementing zones
- Submitting protocol and compliance information to relevant local and administrative authorities (MoE, MoH, etc.) as needed
- Use of CRS CommCare software license, if desired
- Tablets for data collection
Structure of Proposal and Submission Guidelines
Consultants or consulting firms wishing to apply to conduct these evaluations should send their CVs, along with a technical proposal that includes at least the following specifications:
- A description of the firm’s expertise (maximum 5 pages)
- The different tasks they are planning to undertake in order to fulfill the evaluation’s purpose, scope and objectives (2 pages)
- Detailed explanation of the selected methodology (maximum 5 pages)
- A detailed budget with explanatory notes (maximum 5 pages). Bidders must submit a detailed financial proposal for the baseline, midline, and final evaluation, and special study, not exceeding $400,000 for the three data collection points.
- A sample of similar work undertaken as lead consultant(s) (maximum 5 pages)
The proposal should contain no more than a total of 25 pages of which; technical proposal 20 pages and financial proposal 5 pages
The proposals must be submitted no later 22 October, 2019 at midnight GMT to SN_HR@.crs.org