Final Evaluation Consultant

Apply by 1 June 2018
Posted on 8 May 2018

Job Description

Project Background

With generous funding from USAID, the DHARTE project is a three-year activity that builds on AHM’s previous USAID-funded EJEMPLAR (Empowerment and Education for the Advancement and Respect of Youth in Monterrey) in Monterrey, Mexico by adopting best practices and lessons learned and strengthening the model to include the use of audiovisual tools and arts to promote youth development, with a stronger focus in this continued phase on violence prevention. The DHARTE activity supports a comprehensive, multi-intervention prevention model that includes primary and secondary prevention components. The DHARTE technical approach includes a crime and violence assessment and monitoring system that will detect the most at-risk youth in primary and secondary schools. DHARTE applies arts-based interventions to address alternatives to prevent crime and violence, and includes the development of a local hub to pilot international and national best practices and evaluate their impact in the local context.

The DHARTE model incorporates innovative approaches to amplify at-risk youth’s safeguards by integrating a mixed model of interventions with a focus on art therapy and artistic expression along with healthy interpersonal and family relationships. Additionally, DHARTE provides entrepreneurship training for secondary school students based on the development of new skills related to entrepreneurial capacity, particularly leadership and problem-solving skills.

As part of the program, the ERI (Evaluacion de Riesgo individual, Individual Risk Assessment) was developed using the SAVRY (Structured Assessment of Violence Risk in Youth) tool. The SAVRY is a tool that can be used to create context specific violence risk assessments for youth. Violence risk is defined as the risk of being victims or perpetrators of violence. The ERI is used primarily as a screening tool to assess the violence risk in student participants and assign them to the primary or secondary intervention depending on their risk levels. It was applied for the first time in November 2015, and then in May and August of 2016 and June and September of 2017. Between the two applications in 2016 and 2017, the ERI was not applied to any of the students twice, but rather was applied at the latter point for those students who started the school year and were not captured by the previous application. The ERI will be applied a final time at the beginning of June 2018.

Evaluation Rationale

HAI seeks a Contractor to conduct a final evaluation of DHARTE over the life of the project. The evaluation will measure the project’s effectiveness, efficiency, relevance, and potential for sustainability. HAI and the Contractor will collaboratively design the evaluation methodology to investigate research questions that include but are not limited to:

  1. Was the intervention successful in reducing both exposure to and perpetration of crime and violence among the target beneficiaries? What were the key factors that facilitated or hindered violence reduction? Did outcomes related to violence prevention differ by location or gender?
  1. To what extent is the replication model efficient and effective? How do results from intervention replication activities compare to the full project intervention? Is further replication desirable and/or feasible? Are there emerging project practices or experiences that are worth highlighting as good practices or lessons learned which are not captured in the replication model?
  1. What progress did the project make in implementing the midterm recommendations? What effect did the midterm recommendations have on the effectiveness and impact of the project’s interventions?
  1. What are the key project achievements that will be sustained beyond the life and the context of the project at the community, organizational, and participant level?

Evaluation Design & Methodology

The design and methodology applied to this evaluation should generate the highest quality and most credible evidence to answer the evaluation questions listed above, and should primarily focus on assessing project activities implemented under DHARTE. Prior to the start of data collection, the Contractor will develop and present, for HAI review and approval, a written methodology that details the evaluation activities, evaluation tools, and data analysis plan for this evaluation. The written methodology should be guided by a participatory approach to evaluation, and will be comprised of a mix of quantitative and qualitative tools appropriate to the evaluation’s research questions. HAI expects the methodology to include, but not be limited to, the following activities:

  • A desk review of relevant information on the status of primary and secondary youth crime and violence prevention in Mexico, the social and political context of the target population, and strategies to measure youth crime and violence prevention efforts;
  • A review of all relevant project documentation (in English and Spanish);
  • A review of project data (please note that data from the final ERI installment will not be available until XXX;
  • Site visits to occur between June 4 and June 22 (dates set to coincide with the end of the school year – not necessarily all of these dates need to be used during the field visit, but must take place during this time period. This can be agreed upon in the methodology);
  • Data collection methods and tools, including a pre-selection of key informant interviews and focus groups;
  • Data analysis plan. Evaluation questions should be answered with information that is disaggregated by age, location, and gender, and when applicable, grade level and school.

HAI will provide the evaluation team access to all relevant project documents, including but not limited to:

  • The original grant agreement and subsequent modifications to the grant agreement;
  • Quarterly reports, work plans and management reviews developed as part of routine project monitoring; and
  • All data, including data collection tools, previously collected from routine project monitoring.

Evaluation Deliverables

  • Evaluation Methodology & Work Plan: A written, draft methodology for the evaluation will be submitted to HAI for review and approval no later than the 7th day following the signing of the agreement. The methodology plan should include the evaluation design, data collection plan, instruments, tools, and an operational work plan, including a timeline. Please note that due to timing of the school year end, the field visit will need to occur before all project data can be made available to the evaluator. The work plan should reflect this timeline accordingly. The methodology plan will be discussed with HAI prior to any data collection activities. The Contractor may change the methodology but must submit any changes for approval to HAI prior to implementation.
  • Field Visit Report: A field visit to project sites should be held from between June 4 and June 22 in order to overlap with the end of the school year. A report outlining field visit activities will be submitted by July 6.
  • Evaluation Data and Analysis Files: All raw data generated as part of the evaluation should be submitted no later than two weeks after data collection activities are completed. Any files generated as part of the data analysis should also be provided. The Contractor will provide data and analysis files in an electronic file in an easy to read format; organized and fully documented for use by those not familiar with the project or evaluation.
  • Draft Evaluation Report: The Contractor will submit a draft report of the findings and recommendations from the evaluation no later than August 1, 2018. Prior approval by the Company will be required should the Contractor want to revise this deadline. The format of the evaluation report should follow the guidelines listed in AHM’s final evaluation template. The report must be submitted in English, electronically.
  • Debriefing with AHM, HAI, USAID, and key stakeholders: The Contractor will present major accomplishments and findings from the evaluation to AHM, HAI and key stakeholders through PowerPoint presentation after completion of field-based data collection. The debriefing will include a discussion of achievement and issues as well as any recommendation the team has for possible modifications to project approaches, results, or activities. The Contractor will consider HAI project staff comments and revise the evaluation report accordingly, as appropriate.
  • Final Report: HAI will provide the Contractor with written comments on the Draft Evaluation Report no later than August 8, 2018. The Contractor will submit a Final Evaluation Report that incorporates HAI’s responses to the Draft Evaluation Report no later than August 13, 2018. The format of the evaluation report should follow the guidelines provided to the Contractor by HAI. The final version of the Final Evaluation Report will be submitted in English, electronically no later than August 17, 2018.
  1. Evaluation Team Composition & Desired Qualifications

The Contractor will receive support from HAI’s project team, as well as HAI’s Monitoring & Evaluation team. The Contractor should possess the following qualifications:

  • Fluency in Spanish and English;
  • Outstanding oral and written communication skills;
  • At least five years demonstrated experience in conducting evaluations and writing reports;
  • Demonstrated experience working in the field of crime and violence prevention;
  • Demonstrated experience working in the field of mental health and psychosocial support services a plus;
  • Demonstrated knowledge of the key challenges and actors associated crime and violence prevention in Latin America.

Evaluation Management

HAI will provide overall supervision to the Contractor during contract period. HAI will assist the Contractor with finalizing the evaluation methodology and work plan, and assist in identifying and providing key project documents for the Contractor to review during the contract period. HAI will provide overall logistical support and assistance to the Contractor, as needed and when possible. HAI will provide transportation and accompaniment to project sites, all project documentation and data, and other support as approved in the evaluation’s work plan. HAI’s project-based staff will be made available to the Contractor for consultations during the evaluation process.

The Contractor is responsible for arranging and completing all evaluation-related activities listed in the approved methodology and work plan, in coordination with HAI’s local project and logistics staff. The Contractor will advise HAI prior to each of those activities. The Contractor will be responsible for completing and submitting to the Company the evaluation products listed in Section E of this document. The Contractor will be responsible for procuring his/her own work/office space, computers, Internet access, printing, and photocopying throughout the evaluation. The Contractor will be required to make his/her own payments for such services. The Contractor will be responsible for any expenses beyond the fee provided by the Company as determined in this document.

  1. Payment Schedule
  • 1st installment: 30% upon receipt of the first deliverable, Evaluation Methodology & Work plan – due on the 7th day after signing the contract.
  • 2nd installment: 70% upon review and approval by the Company of all other items listed under the ‘Evaluation Products’ section of this SOW and receipt of invoice from the Contractor. The Contractor will provide the Company with a written invoice within 15 days prior to the end of the consulting period for services rendered.

Submission Process

To be considered for this contract, please submit the following to

  1. A letter of interest, indicating how your skills and experience meet the qualifications listed above;
  2. A résumé or CV (no more than four pages);
  3. A writing sample (no more than three pages);
  4. A draft budget, including consultation fees and all other evaluation-related expenses, including travel, with a maximum amount of $14,000 USD.

Incomplete submission packages and submission packages that do not adhere to the page limits stated above will not be considered. Present employees of HAI or HAI’s implementing partners need not apply.

About the Organization

Heartland Alliance International (HAI) is the youngest and fastest growing part of Heartland Alliance for Human Needs & Human Rights, a family of organizations that has been leading anti-poverty and social justice work in Chicago for more than 125 years. HAI is comprised of nearly a dozen country offices implementing programs on a broad range of human rights issues globally, as well as the Chicago-based Marjorie Kovler Center for the Treatment of Survivors of Torture, which serves individuals from more than 50 countries. HAI has significant expertise in the fields of trauma-informed mental health care and access to justice for survivors of rights abuses. It is also an industry leader in access to high-quality and stigma-free health care. Across all of its programs, HAI promotes progressive, innovative approaches to human rights protections and gender equality.

Similar Jobs