Evaluation Consultant(s): Guatemala

Job Description

Introduction

The National Democratic Institute (NDI or The Institute) seeks a consultant(s) to conduct an independent evaluation of its 17-month program in Guatemala, “Safeguarding the Integrity of the 2015 Electoral Process,” funded by the Swedish International Development Cooperation Agency (Sida), the United States Agency for International Development (USAID) and the Norwegian Embassy. The Institute was responsible for overseeing implementation, providing technical assistance and administering subgrants to members of the Electoral Watch (Mirador Electoral) consortium. This evaluation will examine aspects of the NDI and Mirador Electoral program to confirm what was achieved over the course of the program and identify continued needs and unaddressed challenges to form the foundation of future interventions.

Specifically, the evaluation will consider the: 1) relevance, 2) effectiveness, 3) efficiency, 4) sustainability, and 5) impact of NDI’s intervention and the Mirador Electoral project, including whether the program achieved the expected results. It will also assess and document the original problem as it relates to future programming and provide recommendations to inform future interventions.

Program Background and Objectives:

Two decades have passed since the 1996 United Nations-sponsored peace accords ended 36 years of civil war in Guatemala. While there has been an expansion of civil and political rights in that time, democratic development in the country still faces major challenges. State institutions face difficulty in effectively responding to threats, such as high levels of political violence, opaque campaign finance practices and ineffective electoral oversight regulations. These issues are compounded by corruption, legal impunity from criminal acts, organized crime, and some of the highest homicide and femicide rates in the world. The September 2015 general elections tested the country’s fragile democracy. Despite improvements in legal and electoral institutions, democracy in Guatemala continues to face serious challenges, evident in negative public perceptions of the country’s electoral system. In a 2013 public opinion poll, Latinobarómetro found that Guatemala remains at the bottom of the list for Latin American countries in terms of support for democracy. The electoral process lacks widespread support, with only 49 percent of Guatemalans expressing confidence in the Supreme Electoral Tribunal (Tribunal Supremo Electoral, TSE) and 48 percent in the outcome of the elections.

The first round of the 2015 general elections took place against a backdrop of high-level criminal cases and civic protests against corruption. In April 2015, the United Nations-backed International Commission Against Impunity in Guatemala (Comisión Internacional Contra la Impunidad en Guatemala, CICIG) and the Guatemalan Attorney General’s Office (Ministerio Publico) uncovered a massive corruption scheme that involved high-level government officials. The criminal network, known as La Linea, involved public officials charging importers bribes in exchange for lower import taxes, defrauding the country of an estimated $120 million in tax revenue. The Ministerio Publico arrested 22 people in connection with the scheme, including the vice president. Between April and the general elections, tens of thousands of Guatemalan citizens gathered to protest, calling for an end to corruption and impunity, reforms to the political and electoral system, and the resignation of the president. Just days before the elections, President Otto Perez Molina resigned over corruption allegations and was placed under arrest, a first in Guatemalan history. This project was implemented in the context of this political crisis, which generated uncertainty around the electoral process.

Based on the Institute’s two decades of experience providing technical assistance to citizen observation groups in Guatemala, NDI has observed the following obstacles to the transparency and integrity of the 2015 electoral process: 1) citizens’ lack of trust in the electoral process pointing to a crucial need for independently verified results; 2) lack of access to and knowledge of latest innovations in monitoring approaches by citizen observation groups, including methodologies for organizing election-day quick counts and pre-election monitoring activities,; 3) systemic barriers to political participation and representation experienced by marginalized populations, including women and indigenous peoples; and 4) weak or insufficient electoral regulations to guarantee an effective electoral process, including a lack of compliance by political parties.

With funding from Sida (as well as USAID and the Norwegian Embassy) in Guatemala, NDI launched a program in April 2015 to further strengthen Guatemala’s democracy by bolstering the transparency and integrity of the 2015 electoral process. The program’s specific objective is to: strengthen the technical capacity of citizen election observation groups to organize pre-election and election-day monitoring activities and to provide evidence-based recommendations to improve the transparency and effectiveness of elections.

This specific objective is supported by three intermediate results:

• IR 1.1: The technical capacity of Mirador Electoral member organizations to organize and conduct a comprehensive range of election monitoring activities is strengthened;
• IR 1.2: Internal and external coordination and communication mechanisms among election observation actors are consolidated; and
• IR 1.3: More informed and well-coordinated electoral reform advocacy efforts are developed

During the 17-month program, the Institute provided technical and financial assistance to the nine Mirador Electoral member organizations. These organizations carried out the following activities: 1) conduct focus groups and a national survey to assess citizen perceptions of the primary election-related challenges; 2) monitor the pre-election environment in 52 municipalities; 3) analyze media coverage of candidates to encourage balanced coverage; 4) evaluate and report on the work of the TSE and National Civil Registry (Registro Nacional de las Personas, RENAP); 5) assess women and indigenous people’s political participation as individuals and groups and how their concerns are being addressed by candidates; and 6) evaluate the elections process, identify lessons learned, propose recommendations for electoral and institutional reforms and help support advocacy on a reform agenda. For the four month program extension, NDI has proposed to focus on the following activities, including: 1) monitoring key aspects of the post-election period and the beginning of the incoming municipal administrations’ and Congress’ terms; 2) strengthening newly-formed local observation networks to establish a permanent citizen observation and oversight initiative; and 3) developing a comprehensive, systematic study of Guatemalan political attitudes, policy concerns and expectations that would analyze the current state of democracy and political culture in the wake of developments during the 2015 electoral process.

Objectives and Illustrative Evaluation Questions

The objective of the program assessment is to evaluate the extent to which Mirador Electoral and NDI have successfully completed the program objectives. Using the Development Assistance Committee (DAC) evaluation criteria to guide the evaluation design, the illustrative evaluation questions may include:

Relevance:

• How relevant were NDI and Mirador Electoral program strategies given the context of the electoral process leading up to, during and after 2015 elections?

Effectiveness:

• To what extent has the program been successful in achieving the main objective and three supporting intermediate results?
• To what extent has NDI helped build the capacity of its program partners to achieve their strategic objectives?

Efficiency:

• How well did the Mirador Electoral members work together as a coalition? To what extent have the members coordinated with external election observation actors?

Sustainability:

• To what extent has Mirador Electoral contributed to a culture of domestic election observation in Guatemala in which citizens and electoral authorities understand and recognize the importance of civil society election observation?
• Was the technical capacity of Mirador Electoral member groups to organize and conduct a comprehensive range of election monitoring activities strengthened? How capable are program partners in continuing their observation and monitoring strategies without NDI technical assistance?
• Were more informed and well-coordinated electoral and political reform advocacy efforts developed through this program?
• What additional programming is necessary to ensure the long-term sustainability of the gains seen as a result of the program?

Impact:

• To what extent has the program contributed to developing more transparent electoral processes in Guatemala?

• Was the program successful in increasing the political participation of all citizens,? Specifically, how has the program affected the political participation of marginalized populations in the electoral process - especially youth, women and indigenous peoples? What are the remaining gaps and challenges?

Evaluation Methodology

Applicants must submit a document outlining proposed evaluation approaches; however, the methodology will be finalized collaboratively with NDI and Sida. It is anticipated that the evaluation will include a desk review of program reports and outputs. It also should include a combination of qualitative and quantitative data collection from a purposive sample of key individuals and stakeholders including, but not limited to, partner civil society organizations (CSOs), Guatemalan political analysts, political parties’ representatives, journalists, and electoral authorities. The design also should include participatory evaluation processes with the Mirador Electoral member organizations.

The evaluator should consider a range of possible methods and approaches to collecting and analyzing the information required to assess program results, establish relationships between activities and outcomes, contributions to the overall impact and make programmatic recommendations. Emphasis should be on capturing the arc of the intervention, looking at how the program as a whole contributed to change, as well as the success or effectiveness of specific types of interventions at different points over the course of the program. In determining the effectiveness in achieving the program objective and results, the evaluation consultant should draw heavily from indicators outlined in the proposal, as well as program reports.

Evaluation Team

The proposed evaluation will be awarded to the evaluation team who best fits the below qualifications. NDI will weigh the value for money and the feasibility of the program in its selection.

Essential Qualifications

• Shall have documented experience in conducting evaluations of development projects and demonstrated expertise in evaluation methodologies;
• Extensive knowledge of or experience in managing or implementing democracy and governance programming;
• Masters degree or higher in law, political science, applied research, international relations or related field;
• At least five years managing and conducting evaluations, preferably related to the analysis of election monitoring programs;
• Strong proficiency in collecting and analyzing qualitative and quantitative monitoring and evaluation data;
• Proven excellent research skills, including development and application of analytical frameworks as well as tools and production of analytical reporting;
• Ability to work with diverse stakeholders and achieve quality results with limited supervision;
• Knowledge and/or previous work with citizen engagement and domestic election observation programs.
• Excellent communication skills in English and Spanish;
• Strong facilitation, presentation and communication skills; and
• Previous experience working in the Central America region (preferably in Guatemala).

Timeline and Key Deliverables

The evaluator will be expected to develop an evaluation strategy and delineate tasks necessary for the successful completion of the evaluation.

Tasks:

• Desk review of program documents and partner outputs, including NDI donor reports and tools and reports produced by partners;
• Design of the overall evaluation strategy and methodological approach, including identification of appropriate data collection and analysis methods, development of evaluation instruments and questionnaires, and establishment of protocols for data collection and analysis;
• Data collection and analysis, likely including a review of project reports, partner outputs, data collected through ongoing program monitoring, interviews with key informants and focus groups, and participatory evaluation processes with domestic election observation partners;
• Data analysis against program results;
• Validation session with NDI Guatemala and Mirador Electoral;
• Submission of draft report for comments by NDI and Sida; and
• Submission of final evaluation report.

Deliverables:

• Evaluation inception report that includes: refined terms of reference (TOR) and evaluation strategy, evaluation questions, methodology and work plan;
• Informed consent documentation;
• Data Confidentiality plan;
• Data collection, analysis instruments, and protocols;
• Weekly progress reports to Sida;
• Raw and processed data;
• Draft program assessment summary and oral briefing of findings during validation session with NDI-Guatemala and Sida prior to departure from Guatemala;
• Draft report for comments and questions from NDI and Sida; and
• Final report.

Reporting:

The final report should include:

• Executive summary of findings, conclusions and recommendations;
• Introduction and background to the evaluation;
• Evaluation questions, methodologies and approach, scope, constraints, and limitations;
• Description of the project objectives evaluated;
• Data collection and analysis;
• Key findings and conclusions;
• Recommendations for future programming to be implemented by Sida, NDI and/or Mirador Electoral; and
• Annexes including: TORs, workplan, instruments and protocols, databases, references, documentation, etc.
• The report can be in either English or Spanish with an executive summary in English.

NDI anticipates the consultancy will take place between May and July, finishing no later than July 31, as follows:

• 6 days - Document review and inception report from consultant’s home base.
• Up to 20 days - Data collection, partial analysis and validation session in Guatemala (including travel).
• 6 days - Data analysis and draft report from consultant’s home base.
• 2 days - Finalizing of report from consultant’s home base.

The evaluation can take place during the stated three month period, but the consultant fee is not to exceed 34 days.

Evaluation Proposal Package

Please submit the following online with a copy sent via email to Katherine Conway at kconway@ndi.org by April 18, 2016:

• CVs of the proposed evaluation team;
• Proposed daily rate;
• A cover letter outlining interest and proposed evaluative approaches;
• Example of past assessments or detailed proposal;
• A detailed budget that includes all related costs; and
• At least two references.

Only shortlisted candidates will be contacted. NO PHONE CALLS PLEASE.

PI93357367

About the Organization

Founded in 1983, NDI is a nonprofit, nonpartisan organization that works to strengthen and expand democracy worldwide. Headquartered in Washington, DC and with 59 country offices on five continents, NDI has conducted democratic development programs in more than 100 countries around the world. Calling on a global network of more than 500 volunteer practitioners from approximately 70 countries, NDI provides practical assistance to civic and political leaders advancing democratic values, practices and institutions. The Institute works with democrats around the world to build political and civic organizations, safeguard elections, and promote citizen participation, openness and accountability in government. The Institute receives funding to carry out its mission from a variety of sources, including the U.S. Agency for International Development (USAID), the National Endowment for Democracy (NED), the Department for International Development (DFID), the United Nations Development Programme (UNDP), and other donor agencies and private foundations.