Final Project Evaluation Analyst and Writer

Job Description

Final Evaluation Terms of Reference

Evaluation Terms of Reference: Revenues through Cotton Livelihoods, Trade, and Equity (RECOLTE) Project

Catholic Relief Services: Burkina Faso Country Program

June 10, 2019

DRAFT

1. INTRODUCTION

The USDA-funded Food For Progress Revenue through Cotton Livelihoods, Trade, and Equity (RECOLTE) project works to upgrade the organic cotton value chain and improve the livelihoods of vulnerable smallholder producers in Burkina Faso. Organic and fair-trade cotton commercialization represents a niche market for smallholder producers, especially women. RECOLTE targets all cotton production zones in the country in the regions of Comoe, Loba, Ziro, Boulgou, Nayala, Oubritenga, Kossi, and Gourma. CRS has lead project implementation since April 2014 in partnership with the National Union of Burkina Faso Cotton Producers (UNPCB) and Texas A&M – AgriLife Research. RECOLTE also works in collaboration with the Institute of Environmental and Agricultural Research (INERA), as well as the Ministry of Agriculture and Food Security and the Ministry of Commerce and Trade, to support the government of Burkina Faso to foster long term support for the organic cotton industry.

The RECOLTE project is valued at $11.8 million and seeks to benefit 10,000 producers. CRS has monetized 16,600 MT of milled rice valued at $9.4 million. The proceeds are used to implement the program. Additionally, CRS has used approximately $2 million to administer and monitor the program.

RECOLTE works with UNPCB to increase production of organic cotton in response to high demand through two Strategic Objectives (SO).

1. SO1 seeks to increase agricultural production by a) improving the quality of land and water resources; b) increasing the use of improved agricultural techniques and technologies; and c) by improving farm management.
2. SO2 seeks to expand the trade of agricultural products by a) increasing value added to post-production agricultural products; b) increasing access to markets to sell agricultural products; and c) improving transaction efficiency.

Project activities focused on strengthening 255 farmer groups, increasing organic cotton production, and increasing the capacity of the National Cotton Producer’s Union of Burkina in management and linking the union to the international market for increased organic cotton trade. As the organic cotton production involves rotation crops, CRS has also been promoting soy, sesame, peanut, and shea production. CRS has been linking these producers with service providers to provide intensive training and advisory support for organic cotton production. The program offers microloans to women for income-generating activities via a revolving fund. Implementing partners, the Ministry of Agriculture, and technical partners provided extension support through demonstration plots and structures, and input supply networks. A detailed activity description is in section 1.C.

The Results Framework in Annex 1 presents RECOLTE’s objectives, results streams, intermediate results, and related activities.

1.A. Final Evaluation Overview

The final evaluation data collection was completed from January – March 2019. Per USDA regulations, a final evaluation report must be completed by late August 2019.

The purpose of this final evaluation is to assess and capture outcomes of RECOLTE project strategies and interventions and document best practices to inform future programming and wider learning. Specific evaluation questions are presented in Section 3. EVALUATION QUESTIONS / OBJECTIVES below.

The data collection was performed using industry standard mixed-methods of both quantitative and qualitative data collections, including: quantitative beneficiary household survey; document review; interviews, discussions and post-evaluation workshop; and field observations. The evaluation report will use pre-post performance evaluation design, and will employ the data collected, including document review.

The evaluation report will be led by an international consultant as explained in section 5. EVALUATION TEAM.

The USDA approved and complete version of the Performance Management Plan, Baseline study, Midterm Evaluation, and Evaluation Plan will be provided to the selected evaluator. They will also receive the quantitative and qualitative final evaluation raw datasets, as well as a drafted evaluation document that will inform the final evaluation.

1.B. Background: CRS and Implementing Partners

CRS works in partnership with the National Union of Burkina Faso Cotton Producers (UNPCB) as the main implementing partner for activities. The University of Texas A&M – AgriLife Research Center and the Cotton Research Program of the Institute of Environmental and Agricultural Research (INERA) provide technical support on a contractual basis to assist RECOLTE and UNPCB on the seed systems component of the project.

USDA Food for Progress is the funder of the evaluation, and will review, provide comment, and approve the final report.

Other key stakeholders RECOLTE worked with include the Ministry of Agriculture and The Permanent Secretariat for the Monitoring of the Cotton Value Chain of the Ministry of Commerce and Trade, Organic and Fairtrade certification organizations (ECOCERT, GOTS, and Fairtrade International), the SOFITEX cotton company, and USDA. CRS also maintained a strong relationship with the USAID C-4 cotton project implemented by IFDC. Final evaluation information from the C-4 project will be among the resources provided for document review.

In addition to organic cotton farmers, the above-mentioned entities are the main stakeholders the evaluation team will consider as key informants at this final evaluation, especially for the qualitative portion of the study, process and context analysis. The evaluator will review interviews with these entities, as they will be instrumental in assessing the outcomes of the RECOLTE intervention as well as potential sustainability with respect to the strength of local ownership and sustainable partnerships.

1.C. Background: Project Activities

The project has ten interventions/activities:

1. Provide training on production of organic cotton for extension agents and producers

CRS has worked to improve and increase the production of organic cotton by developing and conducting trainings of producers in organic cotton production. These trainings have touched 8,425 people and include topics such as the proper placement of organic fields, including the importance of maintaining the required distance from conventional or Bacillus Thuringiensis (Bt) fields; plot selection and seed preparation; managing internal controls to ensure organic standards are upheld; composting; integrated pest management; and income diversion through rotational crops other than cotton.

CRS has set up 229 demonstration field plots throughout the project zone to reinforce these trainings. CRS selected lead producers and has arranged for them to host training visits for new producers of organic cotton and follow-up training for existing producers, showing best practices in organic production. CRS has recruited additional zone managers and field agents in order to mobilize and train new producers in the organic program.

1. Provide training on improved organic cotton farm management for extension agents and producers

CRS has trained 8,467 individuals in improved farm management practices, counting field agents and zone managers, who have then trained producers, in improved farm management practices. These trainings include the collection and management of production data, such as cotton and rotational crop production, yield, area planted, and global positioning system (GPS) usage. The trainings also include instruction on how to report the data required for organic and FairTrade certification.

1. Facilitate organic and FairTrade cotton certification process for UNPCB

CRS has trained its staff and the organic cotton producers in topics such as the organic and FairTrade certification process, rotational crop standards to maintain the organic cotton certification, Global Organic Textile Standards, and best practices. At project initiation, CRS provided organic cotton certifiers, such as EcoCert, with a list of new and existing organic cotton fields in the project area to establish a schedule for production and processing inspections. CRS has also assisted the producers in preparing and maintaining the internal control documents that are necessary for certification. CRS has trained agents in each project zone in the proper procedures for conducting the Quick Test for Bt at the point of cotton collection and weighing before the cotton is shipped to the gin.

1. Develop cooperative business capacity for UNPCB, extension agents, and producers

CRS hired a Senior Business and Cooperative Management Technical Advisor, who was be based in Bobo Dioulassou, for the first three years of the project. This advisor strengthened the management, finance and operations systems of the National Union of Burkina Faso Cotton Producers’ Organic Program Management Team, as well as developed a new five-year business plan for increased programming investments during the project and beyond. Over the course of the project, 42 management staff have received training in management, finance, and operations.

1. Develop business services providers for UNPCB, extension agents, and producers

CRS has strengthened the input supply for organic seeds by working closely with seed farms and seed input providers. CRS has established three organic cotton seed multiplication farms, in the regions around Banfora, Fada, and Ziro, that plant cotton and rotational crops. The farms are owned and managed by the National Union of Burkina Faso Cotton Producers (UNPCB). CRS has provided funds for required inputs for the farms, including one borehole per site, and have used GPS mapping to ensure that the seed farms adhere to organic farm zoning regulations. CRS has also provided technical guidance for the seed farms and trained the seed farm managers in organic cotton seed multiplication systems, seed farm management, and testing to detect and prevent genetic drift.

CRS is working with the Institute of Environmental and Agricultural Research (INERA) to establish a test plot and laboratory to multiply organic cotton foundation seed in order to ensure an adequate supply of certified seed annually. CRS has facilitated organic certification of seed farms through EcoCert and trained seed farm staff regarding the internal controls necessary to ensure organic certification of the seed farms. CRS has linked cotton producers to organic cotton seed producers each season and supply seed for rotational crops, which include ground nuts, sesame, and a cereal, typically sorghum, maize, or

millet.

1. Improve post-production, processing, and handling for UNPCB, extension agents, and producers

Under this activity, CRS will procure and install a dedicated organic cotton gin, which will facilitate processing of organic cotton in a timely fashion. CRS has trained field agents and zone managers, who will then train producers, in improved post-production processing and handling for rotational crops. CRS has trained 209 people involved in ginning, such as: the head of the Sofitex cotton ginning factory, shift managers, and workers. The themes focused on improved cotton ginning and storage standards and provide them with updated Sustainable Compliance Initiative tools as they become available and as new standards take effect. CRS has created a standards team within Sofitex to ensure that the standards are followed throughout the entire ginning process. CRS has also provided the gin with the locally-produced FASOPLAST bags for the cotton bales. In addition, CRS has selected rotational crops for promotion after considering their contribution to soil quality, cultural acceptability, and hardiness against pests, and the participation of women in their cultivation.

1. Financial services: facilitate agricultural lending for UNPCB and producers

CRS has established a loan guarantee fund with an anticipated average of $250 per loan to facilitate producers’ access to micro-finance services. CRS works with the National Professional Association of Decentralized Financial Systems of Burkina Faso to identify a financial institution to manage the fund.

CRS works with this institution to establish training on administration of the new fund and management parameters for new agricultural loan products; to determine how to manage the application and assessment process; and to facilitate linkages with micro-finance institutions (MFI)s with branches in the target zones. CRS continues to assist producers’ groups with loan applications, and the MFIs in the loan assessment process, throughout the life of the project.

CRS developed improved agricultural loan products by working with MFIs to assess current financial products and design appropriate financial products for organic cotton farmers, both as individuals and as cooperative groups. CRS emphasizes the importance of savings as part of an organized financial approach, followed by appropriately sized loan principals, reasonable loan terms, documentation and collateral, and installment payment requirements that match the cash flow of the investment. CRS has also trained MFI staff regarding the products and customer service practices specific to the needs of organic cotton producers. CRS links producers to existing micro-insurance options.

Additionally, the Columbus Foundation has contributed $140k to a revolving fund that is available to women organic cotton producers. For purposes of fund management, women’s associations have been created out of organic cotton producer groups that have more than 10 women participating, resulting in 115 women’s associations. There are 2,272 loans currently, at a value of 77 million FCFA, the average loan is 33,890 FCFA, or about $64. The fund is earmarked relative to the number of women in each association to ensure equitable access. The loans are granted after a participating woman successfully completes an application process facilitated by UNPCB and CRS.

1. Improve market linkage for increased trade in organic cotton for UNPCB

CRS works to ensure that producers are aware of available systems for obtaining market information and will encourage producers to access and use this information. These include established internet information and Information and Communication Technology (ICT) platforms, such as the Global Organic Cotton Community of Practice and the government food security information system. CRS has informed the producers that these systems are able to receive requests for market information from beneficiary producers in all project zones and respond to them by SMS and voice messaging in the local language.

1. Facilitate policy and regulatory dialogue on organic cotton for UNPCB, government of Burkina Faso, private cotton sector

CRS works to facilitate and participate in national and regional policy dialogues among producers, CRS staff, and government officials regarding organic agriculture and fair trade, focused on promoting the sector and gaining government recognition. The dialogues will include subjects such as genetically­ modified organism (GMO) contamination and zoning; sector promotion, both internally and externally; certification options and cost efficiency of the current structure; land tenure and use; pricing transparency; scientific research; soil fertility; and land and water quality. CRS has organized four one­ day roundtable sessions to facilitate discussion and strategic planning on how to contribute to positive policy outcomes for the organic cotton sector. The round tables have facilitated discussion on soil fertility, research findings on organic cotton production, certification, and local cotton processing.

1. Gender and child labor prevention awareness for UNPCB, extension agents, and producers

CRS has conducted a three-day training session to promote increased women’s participation in the organic cotton sector. The training included discussion of barriers to women’s participation, household decision-making, and access to and control of household resources. In addition, CRS will review and update existing training materials on prevention of child labor and then train zone managers, field agents and farmers on this topic.

1.D. Background: Monitoring, Evaluation, Accountability and Learning (MEAL) and Evaluation Plan

For the RECOLTE Project, CRS established a MEAL system that yields timely information on implementation progress against annual work plans, progress towards achieving performance indicators’ results targets, changes in program opportunities and risks, and on the impact of program activities on farmers’ groups and entrepreneurial women’s status and success. It was also used for communicating and reporting progress to USDA and Global Organic Textile Standards (GOTS) as appropriate.

Through the RECOLTE Evaluation Plan, CRS conducted baseline and annual outcome surveys. An independent evaluator conducted the midterm evaluation, and the final evaluation will follow suit. Special studies such as a soil analysis at baseline, and a cost-efficiency and political feasibility study on a small organic gin were conducted.

Project monitoring included collecting data on USDA-required standardized indicators. Data collected on defined indicators tracked progress towards achieving final results and strategic objectives to determine the effectiveness of program activities.

1.E. Background: Context and implementation-related issues

Major issues that influence project implementation and have occurred after the start of the project (overall political situation, management issues and changes in UNPCB’s business model) will require specific attention in this evaluation.

Timeline of RECOLTE Challenges

Although the RECOLTE agreement was officially signed on September 27, 2013, actual implementation started April/May 2014, after negotiations on the monetization plan were completed. This effectively cut seven to eight months out of the project’s first fiscal year.

In October of that same year (2014), a popular uprising resulted in the resignation of the President, followed by a transitional government. An attempted coup d’etat the following September was thwarted and the transitional government reinstated, but not before several weeks of civil disobedience, limited civilian movements, and a general strike. Peaceful presidential and parliamentary elections were held in November 2015 and the newly elected President took office at the end of December 2015. Two weeks later, Burkina Faso experienced its first major terrorist incident in the capital city, accompanied by the kidnapping of two foreigners in the north of the country and a strike on a border patrol. All of these events have impacted the operational environment for CRS and its partners at various points.

Following the former President’s resignation in October 2014, workers in several state-owned companies started protesting to request the removal of the leaders, accusing them of mismanagement or being too close to the former regime. The UNPCB did not come away unscathed from this development, and saw a number of internal protests.

Critical Assumptions to Consider for Final Evaluation

It is important that this study carry out a thorough context analysis to assess how many of the project’s critical assumptions have been called into question, and what the potential impact may be on the project.

Although the hypothesis used during project design supposed that farmers enrolling in the organic program will permanently stay, it appears that this is not the case, but rather farmers decide whether to produce organic cotton and engage in the project or not on an annual basis, meaning that in any given year there have been farmers joining, leaving, or returning to the program. This hypothesis shaped the design of service delivery (number of trainings planned, distribution of equipment). This situation also impacts project targets in terms of the number of farmers who will be ultimately reached, and for how long. The final evaluation will assess and determine the effects of this important change in farmers’ enrollment on project implementation and relevant services. This will be part of the process analysis.

Another important assumption was that new farmers enrolling in the project will create new groups that will stand alone and work on their own. Standalone groups of new farmers are important for the cohort study that are part of the RECOLTE evaluation methodology. The evaluation team will not only assess the effect of this assumption on service delivery but will take it into account in their methodology for this evaluation.

Additional Questions to Consider

The final evaluation will need to take certain questions related to UNPCB’s situation in 2014 into account, for example: How was UNPCB’s overall management affected? Where does the issue stand; has it been resolved or is it still ongoing? Was the RECOLTE project affected, and if so, to what extent?

Another important question that will stand out in this evaluation is the transition to a different business model that UNPCB started during the 2016 planting season. After losing its sole buyer for organic cotton, UNPCB must now attract new buyers and secure new deals. It has been clear that the previous arrangement is unlikely to be replicated, which raises additional questions concerning the sustainability of the project with fewer buyers and a lower price.

Feedback on the mid-term evaluation requested that the final evaluation evaluate the use of FairTrade premiums by producer groups. As a rule, FairTrade premiums must be reinvested into community development, and each group receives an amount relative to their production. If possible, it would be interesting to systematically document and report on these community benefits across the project area.

1. PURPOSE OF THE EVALUATION

2.A. Purpose of the Evaluation

The overall purpose of the final evaluation is to assess and capture outcomes of RECOLTE project strategies and interventions and document best practices to inform future programming and wider learning. It will assess whether the project has achieved the expected results as outlined in the results framework. The final evaluation will assess areas of project design, implementation, management, and lessons learned. It will provide lessons learned and recommendations for program participants, USDA and other key stakeholders for future programs. The final evaluation will focus on relevance, effectiveness, efficiency, impact and sustainability.

2.B. Key Audiences and Uses

CRS will ensure that key stakeholders who have interests in the project are involved in the evaluation process. They will also participate in the discussions related to corrective actions to be taken to address evaluation findings and recommendations.

Stakeholder

Interest

Interviewed for Final Evaluation

Government of Burkina Faso- Ministry of Agriculture and Water, Ministry of Commerce and Trade

Contribution of the project towards the government’s priority to increase the volume of export cotton products

Yes

INERA

Contribution of the project towards increasing expertise in seed production and multiplication, and capacity to test organic products

Yes

UNPCB

Contribution of the project towards strengthening managerial and financial capacity, and their members’ production capacities

Yes

Producers

Contribution of the project in increasing their revenue

Yes

USDA

Major accomplishments and challenges; accountability to US Government and American people

No – needs to be interviewed

USAID

Accomplishments and challenges, lesson learning, coordination with USG investments

No

United States Department of Labor (USDOL)

Findings relating to child labor issues

No

Texas A&M – AgriLife Research

Experience sharing and learning

Yes

CRS

Appreciating accomplishments and challenges; learning lessons; improving linkages with the Government of Burkina Faso and relevant Ministries; compliance to USDA reporting and accountability requirements

No – needs to be interviewed

1. EVALUATION QUESTIONS

Broader evaluation questions approved by USDA under the Evaluation Plan are:

This evaluation will measure progress against stated goals, objectives and targets against baseline values following these criteria: 1) the relevance of all interventions, 2) the effectiveness of implementation strategies and activities, 3) the efficiency of the project and, 4) the likely sustainability of project outcomes and 5) the impact (induced effects) of initiated actions.

Relevance is determined by the extent to which project interventions meet the needs of the organic cotton producers, respect Burkina Faso’s organic cotton sector development strategies, align to USDA’s objectives, and consider economic, cultural and political contexts. Key questions to be addressed are the following:

• Are the activities and outputs of the project consistent with the overall goals and the attainment of its objective?
• Are the activities and outputs of the project consistent with the intended impacts and effects?
• Does RECOLTE meet the organic cotton producers’ and Burkina Faso’s organic cotton sector development strategies’ and government priorities?
• Is the RECOLTE project relevant as is given the current economic, cultural and political context?
• Are stakeholders (organic cotton producers, local authorities) satisfied with their participation in the project? Why or why not?
• Does the project align with government policies and programs (local, national)? Does the project align and complement other donor, other NGO and/or local organization-managed programs?
• How have the management structure and buyer transitions for UNPCB affected the RECOLTE project design, implementation and assumptions? What are the additional activities that would need to be created/strengthened to support this transition? Is UNPCB is staffed to take care of the additional work they have to in the organic cotton supply chain? What are the needs in terms of additional funding/investment to allow for smooth transition? What role can the RECOLTE project play to further support UNPCB?

Effectiveness is determined by the extent to which the project has achieved its objectives. Key questions to be addressed are the following:

• To what extent were the objectives of the project and the yearly benchmark indicators achieved/are likely to be achieved? (the table listing all of the questions related to project indicators is attached to the TOR in annex 3)
• Were the implementation strategies effective enough to improve income, economic prospects, especially for women, increase agriculture productivity and expand trade of agriculture products?
• What are the project’s major limitations?
• Is the staffing structure and capacity sufficient and appropriate? Is the coordination mechanism effective?
• Has program implementation been effectively monitored? How well did the monitoring and evaluation mechanism in place help the implementation of the project?

Efficiency is determined by the extent to which the project’s resources (inputs) have led to the achieved results, and if the same results could have been achieved with fewer resources or alternative approaches. Key questions to be addressed are the following:

• Were activities cost efficient?
• Were objectives achieved on time?
• Was the project implemented in the most efficient way compared to alternatives (Value for Money)?
• What risks has the project faced and what has been done to mitigate their impact?

Sustainability is determined by the continuation of the benefits of an activity after funding stops, and how much the project has developed local ownership and sustainable partnerships. Key questions to be addressed are the following:

• What activities and/or outcomes (both expected and unexpected) of the program are likely to be sustained? What evidence is there to suggest this?
• What is the level of ownership acquired by the stakeholders? Is it conducive to proper project implementation? Are the benefits likely to be sustained after the end of the intervention?
• What are the major factors that can influence the sustainability of the project intervention?
• What strategies used will obtain long-lasting support from communities and local/central administration that goes beyond the duration of the project?
• How have producers used their FairTrade premiums to invest in community development?
• What are the relevant systems at the village and commune levels for protecting against catastrophic events? How functional are they?

Impact is determined by the intended and unintended medium and long-term effects that can be attributed to the project’s intervention. Effects can be direct or indirect, positive or negative. Key questions to be addressed are the following:

• What are the major changes attributable to the RECOLTE project in terms of UNPCB’s management and cotton production?
• How are these changes affecting the well-being of the project participants and their families?
• Are these changes likely to last after the end of the RECOLTE project?

An assessment of events and coping strategies related to disaster risk reduction is also included as follow-up to data collected at the baseline. This is to determine CRS’s contribution to resilience initiatives in the Sahel:

• What are the natural, political and climactic events that have impacted the well-being of households and communities? (How long ago, what was the impact?)
• What are the current resilience capacities of RECOLTE households and communities (in terms of existing protections and buffers against shocks, including mitigations and adaptive strategies) (by cohort)?
• How did the household cope, how well did the household recover, who in the household was most affected (by cohort)?

Based on the findings of the midterm evaluation, the consultant and CRS will validate or modify the above questions to ensure the most relevant questions are evaluated. Additional questions that may be included are:

• What is the change in beneficiary revenues and in their access to markets (by cohort)?
• What is the change in food security (by cohort) in target areas?
• What was the government’s support to the project?
• What is the degree of ownership by the project’s beneficiaries and by UNPCB?

Draft topical questions specific to project objectives and results are presented in Annex 3, which would be discussed and finalized with selected external consultant and approved by USDA.

1. EVALUATION METHODOLOGY

4.A. Evaluation Approach

The final evaluation report will need to use quantitative and qualitative data analysis to report on the outcomes and lessons learned for the RECOLTE project. The evaluation needs to use mixed methods to assess relevance, effectiveness, efficiency, sustainability and impact. The evaluation is both summative and formative: it will use performance and cohort comparison methods to measure changes in the project’s key indicators, and analyze process and context. All data needs to be disaggregated by gender, as the RECOLTE project serves predominately women in organic cotton. Cohort comparison will allow a coherent analysis of results between cohort 1 (farmers enrolled at baseline), cohort 2 (farmers enrolled after baseline), and cohort 3 (farmers enrolled after midterm) in alignment with the monitoring process of the project and performance evaluation. The performance evaluation will measure project performance among similar groups of farmers, such as rate of changing production, revenue, adoption of techniques, among other indicators.

The purpose of the process analysis is to assess how well the project was implemented and how this has affected its outcomes in terms of quality of services and accessibility of these services to project beneficiaries. This includes project efficiency and challenges in delivering these services. The context analysis will need to capture changes in the project environment (political, organizational, and socioeconomic) and their effect on project outcomes. Most of this information is available in the raw qualitative data that will be provided to the selected consultant.

The quantitative endline data that will be provided measures the extent to which planned targets were achieved. The methodology for the mid-term was used for the final evaluation with the exception that an additional stratum has been added to capture the newest group of farmers that joined since the mid-term. The sample size was determined based on 95% confidence interval and 6% of error margin.

4.C. Sources of Data and Data Collection Methods

As per the RECOLTE project evaluation plan, both quantitative and qualitative data collection methods have been employed to collect data on RECOLTE interventions on organic production and trade. The final evaluation data collection involved formal and informal surveys, interviews of key people, community meetings, participant observation and reviews of project records and documents. Quantitative household surveys provided data to measure the changes in the results (outcome and impact) of project activities, the success of implementation strategies and the extent to which planned targets were achieved.

1. Document review: Program proposal and Results Framework; work plans; baseline, mid-term and outcome survey reports; semiannual donor reports; reports of assessments and special studies including soil analysis, organic cotton gin feasibility study; routine monitoring system generated periodic reports; training modules, and implementation manuals or standard operating guidelines; routine monitoring system tools; performance indicators, performance monitoring plan, and approved evaluation plan.
2. Quantitative beneficiary household surveys to estimate outcome and output performance indicators and to compare with baseline benchmarks. Annex 3 for indicators that have been measured by this method.

• Key informant interviews, and focus group discussions, including implementing partner staff (ATBs, CZs, and seed farm workers), University of Texas A&M – AgriLife Research Center, the Cotton Research Program of the Institute of Environmental and Agricultural Research (INERA), Ministry of Agriculture, Permanent Secretariat for the Monitoring of the Cotton Value Chain of the Ministry of Commerce and Trade, Organic and FairTrade certification organizations (ECOCERT and FairTrade International), and the SOFITEX cotton company, donor representatives, organic cotton producer groups (GPCBs) and community leaders and members. Key informant interviews and Focus group discussions should specifically include women, youths, the elderly and people with disabilities.
  1. The consultant will need to develop surveys and conduct interviews with relevant CRS and USDA staff to complete the qualitative data collection for this evaluation.

1. Field Observations: warehouses, cotton planting and ginning (if not completed by the time of the final evaluation), IGA group activities, processing plants.

4.D. Sampling Strategy

The sampling method used in final evaluation data collection was probability proportional to size, where 500 organic cotton producers were randomly selected. This strategy is consistent with the baseline and midterm sampling strategy to ensure comparability. Table 1 shows the distribution of sampled producers.

The following formula was used to determine sample size:

The primary unit selected was the Organic Cotton Producer Group (GPCB). A GPCB is made up of several producers. Each primary unit was created in proportion to their size in number of members, using the Probability Proportional to Size sampling method. This method was selected based on the assumption that the size of the GPCB is strongly correlated to the production and yield of its group members. The secondary units, representing the participating producers in the RECOLTE project, were selected among the primary units.

Qualitative data collection involved group interviews, individual interviews and key informant interviews. Qualitative data collection included various beneficiary groups and other stakeholders including men, women, youths who are members of organic cotton production and IGA groups.

Table 1: Distribution of the sample and replacement by cohort

(N_i) Number of members of GPCB

Cohort weight

Sample size

Cohort 1

N₁=2279

F₁=N₁/N=0.486

n₁=F₁Xn=241

(117women + 124 men)

Cohort 2

N₂=1327

F₂=N₂/N=0.283

n₂=F₂Xn=142

(121women + 21 men)

Cohort 3

N₃=1087

F₃=N₃/N=0.232

n₃=F₃Xn=117

(56women + 61 men)

TOTAL

N=å N_i=4693

1

n=500

4.E. Data Analysis Procedures

The consultant will need to perform data cleaning and analysis for the final evaluation report. Quantitative beneficiary survey data was collected using standard ICT4D solutions (iFormBuilder on iPads). Qualitative data was collected in questionnaires, available in Word documents. The cleaned database needs to be imported into a standard survey data analysis package such as SPSS or stata for tabulating the indicator estimates, and their confidence intervals and p-values, and additional statistical analysis is needed to answer evaluation questions. The quantitative data analysis plan will need to be developed by the external consultant and finalized with CRS. For qualitative data analysis, the consultant will use industry standard protocols, tools and software, and will draft a data analysis plan that will be finalized with CRS.

1. EVALUATION TEAM

• A selection of the evaluator will be made in collaboration with the USDA representatives. An independent firm will lead the final evaluation. The external consultant team will be an International firm.

The team should at least comprise lead consultant, with the profile below:

Lead consultant

· Advanced university degree in agricultural economics, agribusiness management, natural resources management; agriculture and rural development; international development or any related background

· Excellent knowledge and experience in gender sensitive rural sector economy value chain development (gender sensitive agribusiness development)

· Demonstrated experience and success in evaluation report writing

· Team management experience

· Demonstrated experience conducting project evaluations, including experience with cohort comparison analysis, and possession of strong research skills

· Demonstrated skills in statistics, including experience in analysing qualitative data

· Sound knowledge and experience working in the West African region with knowledge of Burkina Faso and its agriculture and rural sector development domain

· A good understanding of the problems of market development and access to agricultural credit

· Experience in the formulation, monitoring and evaluation of USG-funded projects (USDA Food for Progress preferred)

· A demonstrated high level of professionalism and ability to work within tight deadlines

· Strong interpersonal and communication skills

· Good computing skills

· Native English speaker and writer

· Strong proficiency in French is required

Associate consultant (optional)

· Master’s in Statistics or sufficiently strong quantitative analysis background

· Experience and knowledge in the use of electronic data collection tools in evaluations

· Background in statistics and evaluation methods that use counterfactual, experience with cohort comparison analysis preferred

· Experience in data processing and analysis and comfortable with complex household survey sampling design and implementation

Consultancy Contact Person: The RECOLTE Chief of Party will be the consultant contact person.

Final Evaluation Review Committee: The Deliverables of this Consultancy service shall be reviewed by a committee from CRS and partners. This will include the Head of Programs, the RECOLTE Chief of Party, the RECOLTE MEAL Officer and project staff, CP MEAL Coordinator/Officer, CRS Regional Technical Advisors (MEAL, Agriculture & Livelihoods, Health & Nutrition, and Microfinance), and USDA representative.

CRS will provide support to the evaluation team, like facilitating contact with project staff, partners, and local stakeholders at the field level. Furthermore, the evaluation team is expected to provide their own laptop computers, and any software necessary to plan for and implement the final evaluation.

1. Evaluation Management

The RECOLTE project final evaluation report will be carried out by an independent consultant with oversight from Catholic Relief Services. The independent consultant will have the autonomy to conduct the evaluation process and draw conclusions free from political influence and organizational pressure. The following will be major roles and responsibilities of each part:

Responsibilities/Tasks

Primary responsible body

Remark

· Develop TOR

RECOLTE Chief of Party/Team

With technical support from CRS Regional and HQ MEAL and Technical Advisors, and USDA

· Send out invitation to potential Consultant

Chief of Party

· Submit proposal to CRS

Consultant

· Review proposal, select and notify the Consultant

Chief of Party/Team

With support from CRS regional and HQ MEAL and TAs, and USDA

· Conduct data processing and analysis (utilizing appropriate software/s to manage multi-variant analysis)

Consultant

With oversight from CRS regional and HQ MEAL and TAs

· Produce tables (outputs)

Consultant

· Submit a draft report for review

Consultant

· Review and provide feedback

Chief of Party/Team

With support from the CRS MEAL and TAs from region and HQ

· Incorporate comments and submit the final report.

Consultant

· Submit all the survey/study documents (e.g., completed questionnaires, electronic version of data, training manual, fieldwork logs).

Consultant

· Review and approve the entire report

Chief of Party/Team

With support from the CRS Regional and HQ TAs and MEAL, and USDA

7.REPORTING AND DISSEMINATION PLAN

7.A. Evaluation Report

Report shall be presented in the following template maximum in 40 pages, excluding appendices.

• A title page
• A list of acronyms and abbreviations
• A table of contents, including a list of annexes
• An executive summary
• An introduction describing the program’s background and context
• A description of the program, including the results framework or theory of change
• A statement of the purpose of the evaluation
• Key evaluation questions or objectives and a statement of the scope of the evaluation, with information on limitations and delimitations
• An overview of the evaluation approach and methodology, including evaluation design limitations, and data sources
• A description of the evaluation findings
• Recommendations based on the evaluation findings
• Lessons learned and good practices based on the evaluation findings
• Appendices

Additionally, the report will be made public, per USDA’s M&E policy. The selected evaluator is required to provide a version of the report that is free from personal identifying information.

7.B. Dissemination Plan

STAKEHOLDER

KEY FINDINGS

CHANNEL(S) OF COMMUNICATION

PRODUCT(S) TO SHARE

USDA

· Quality of service

· Project outcomes

· Sustainability

· Lessons learned

· FE report

· FE Report

· PowerPoint slide

· 3-4 page summary

USAID

· Project outcomes

· Sustainability

· Lessons learned

· FE report

· Abstract

USDOL

· Lessons learned on child labor

· FE report

· PowerPoint slide

RECOLTE implementing partners

· Quality of service

· Project outcomes

· Sustainability

· Lessons learned

• Dissemination meeting

· FE report

· PowerPoint slide

· Abstract

· 3-4 page summary

Government of Burkina Faso, private sectors, Ministry of Agriculture and Water, Ministry of Commerce and Trade, Financial Institutions

· Quality of service

· Project outcomes

· Sustainability

· Lessons learned

· Dissemination meeting

· FE Report

· Abstract

· PowerPoint slide

· 3-4 page summary

Communities/main beneficiaries

· Quality of service

· Project outcomes

· Sustainability

· FE Report

· Abstract

· PowerPoint slide

1. DELIVERABLES AND TIMELINE

The Final Evaluation draft is scheduled to take place from June to August 2019. The following table indicates deliverables and indicative timeline that will be finalized with the selected consultant.

FINAL EVALUATION DELIVERABLES

TIMELINE

PAYMENT SCHEDULE

Draft of ToR for FE

June

Hiring of Consultant

June/July

PowerPoint, Abstract, Analysis and Draft Report

August

TBD

Dissemination

September

Final Report

End of August

TBD

The following table outlines details to the deliverables and estimated time for consultants to provide specific dates to complete the deliverables as per the above overall timeframe of the final evaluation.

DELIVERABLES

ESTIMATED NUMBER OF DAYS NEEDED TO COMPLETE

TARGET DATES TO COMPLETE

Document review completed

3 days

June

Data analysis plan for both quantitative and qualitative studies submitted and revised

3 days

June

Data analyzed

7 days

June/July

Draft report completed

10 days

Early August

Draft review by CRS

10 days

August

Final report completed with 3-4 page nontechnical summary and submitted to USDA

10 days

End of August

Results dissemination meeting with stakeholders

1 day

September

Datasets, codebooks, syntax or do files and recordings and transcripts/notes submitted

Same day as results dissemination

September

TBD: Detailed timeline of activities shall be worked out with the consulting firm before signing the agreement.

1. ETHICAL CONSIDERATIONS

The evaluator is expected to follow American Evaluation Association’s Guiding Principles for Evaluators (http://www.eval.org/p/cm/ld/fid=51). In addition,

• Dependent upon participants in the evaluation, consultant should specify steps that will be taken to ensure informed consent, confidentiality, and protection of minors.
• Consultant should specify steps taken to safeguard data collected and data management procedures to be used in the evaluation.

• Final reports must not contain any propriety or personally identifiable information (PII). PII is any information that directly or indirectly identifies an individual. This information can be used on its own or with other information to identify, contact or locate a single person, or to identify an individual in a specific situation. This may include, for example, a name, national ID number, address, birthplace, etc. PII includes both direct and indirect identifiers that, when taken together, could allow for identification of an individual (such as a village name, gender, age, name, and/ or facial image).”
  • In addition, final reports should not allow for the identification of individual schools or communities. Any list of schools or communities provided should be included as in the report annex, so that it can be easily removed before submitting to USDA for external sharing.
  • Consultant(s) should obtain permission from CRS before sharing the final evaluation report with any external party, including posting it to their organization’s website.

1. PROPOSAL SUBMISSION

Each proposal must include technical proposal and financial offer to be considered.

Technical proposal should include:

• A note acknowledging understanding of the TOR and any proposed amendments to the TOR;
• Proposed methodology to be used to undertake the final evaluation that describes data analysis;
• Detailed tentative work plan with a description of the roles and responsibilities of each team member of the consulting team;
• Current detailed resumes of consultants that will be involved in this evaluation;
• Three references of current or recent clients including telephone number and email addresses of clients’ point persons;
• Copies of references (if available) demonstrating good execution and /or services in similar evaluations;
• Valid administrative identification for the consultants or office (with complete address) (State registration/ fiscal situation) for official registered companies;

Financial proposal:

• Detailed financial bid in USD, covering all the costs related to the evaluation.

Proposal submission is possible all business days until 5 PM at CRS Burkina office located in Ouagadougou:

Street 9-107 door 452, Zone Industrielle Goughin,

01 BP 469 Ouagadougou 01

Telephone: (+226) 50 34 31 65/50 34 34 70

Alternatively, send documents electronically to the address below:

Amedee.Guire@crs.org with copy to: Abdoulaye.bikienga@crs.org, heather.robinson@crs.org, and Boubacar.kamissokho@crs.org .

Proposal submission deadline

The deadline for the submission is July 3rd, 2019 at 5 PM (GMT).

About the Organization

CRS staff, alongside our partners and the people we serve, work daily to bring our mission and guiding principles to life. We share our stories of hope, compassion and solidarity to inspire others to join us in creating a more just and prosperous world for all.