INDICATORS, LOGICALFRAMEWORK AND
MONITORING AND EVALUATION SYSTEM
Indicators
What is an indicator?
An indicator is a means of measuring actual results against planned or expected results in terms of quantity, quality and timeliness. In other words an indicator is an evidence that helps us to measure progress towards achieving results (Source: RBM hand book on developing results chain, CIDA, December 2000). Actually indicators provide a standard against which to measure brought about by program activities. Therefore an indicator is a “marker” which when used overtime, shows what progress has been made. Indicators must be directly related to result they are measuring. Whenever possible it is important to ensure a balance between quantitative and qualitative indicators to ensure the validity of findings. Note that there can be indicator for measuring progress at every level: input, process, output, effect and impact.
Example: (1) Number of learner capable of learning basic materials; (2) % of average attendance by gender, age and income; (3) The application of information and knowledge by adolescent learner; (4) Number of women has business management skills (ability to run a business as an entrepreneur)
Different types and characteristic of a good indicator
There are various types of indicators and their categories depend on their scope and purposes. Base on quantitative and qualitative aspects, the indicators are of two types:
(i) Quantitative Indicators: Quantitative indicators are statistically measure the amount or value of inputs or resources available. Quantitative indicators have numerical value, are measures of quantity such as –number of man and woman in decision making; position or level of income per year by sex as compared to baseline level. Therefore “quantity” reflects a numerical condition such as –the number of learners, teachers, costs, facilities or text books at a specified time. Examples of quantitative indicators can include-Number of household with homestead vegetable garden; Number of latrine constructed; Average animal asset loss during annual floods in CHAR area.
(ii) Qualitative Indicator: Qualitative indicator reflects peoples judgment, opinions, perceptions and attitudes of a given situation or subjects. They measure performance relative to given standards and norms. They are intended to measure the “quality” of the input, process and output of the program. The term “quality” can mean different things depending on the context. The term refers to a perceived improvement in the implementation of the (literacy) program. Example-The sense of well-being; the application of information or knowledge; the degree of openness; the quality of participation; the nature of dialogue; class room in good condition or condition of class room; level of awareness.
Difference between quantitative indicator and qualitative indicator is given below:
Whereas considering level of objectives to be measured as per logframe structure of a project, the indicators are categorized into five levels:
Input Indicators: Input indicator describes resources that go into the project; such as amount of money spent; different logistic items procured and used; number of hours of staff time.
Activity Indicator or Process Indicator: Activity indicator documents or records the number of activity conducted and /or completed. Such as number of training event completed, number of group meeting held.
Output Indicator: Output indicator describes the goods and services produced by project activities such as number of training modules developed; Number of women trained; number of NGO staff trained; Test score of learners; Percentage and position of women involve in new CBOs; number of family planning service provided.
Effect Indicator: Effect indicator describes the changes in system or behavior as a result of achieving project outputs. Such as Farmer practicing technique taught; Extent to which trained staff use their new skills; Changes of attitude about educating girls; couple practicing family planning; number of people (F/M) who have access to and has been drinking potable water and using sanitation facility.
Impact Indicator: Impact indicatormeasures actual change in condition of identified problem, including in changes in livelihood status, health and wealth of target households. Such as reduction of malnutrition rate, availability and access to essential foodstuff; utilization rate of various health services; average income of IGA participating household; average score of consumer satisfaction of local government activities.
Types of indicator -subject of interest
Direct Indicator: Direct indicators refer directly to the subject they have been developed for.
Example: A good example of direct indicator, which might not be so easy to measure , is “the proportion of population below $ 1 per day “.
Indirect Indicator: Indirect indicators (or proxy-indicators) refer in an indirect way to the subject of interest.
Why indirect indicator?
The subject of interest (mainly qualitative subjects) cannot be measured directly. For examples behavioral change, good governance etc
The subject of analysis is too sensitive to measure, such as in the context of HIV/AIDS intervention “safe sex”, sometimes level of income etc.
The use of an indirect indicator can be more cost-effective then the use of direct one. For example MUAC is cost-effective measure of child malnutrition compare to height and weight measurement.
Aggregate and operational indicators
Recently, a concept has been emerged of dividing indicators into two broad classes called aggregate and operational indicators. The distinction is providing to be beneficial, because it helps clearly exactly what we are being measured, and it helps link interventions with indicators.
Aggregate indicators: Aggregate indicators are variables that broadly measure progress towards a goal. They are useful as an intermediate step to defining more specific indicators. In fact, when a design team is trying to decide on appropriate indicators to measure an effect or impact goal, its first thought is often an indicator that describes the expected changes in fairly general terms. The problem is that aggregate indicators are not very specific as to what will be measured, so that if two people were to independently measure the indicator, they might measure different things and each different conclusion.
“Aggregate indicators are variables that broadly measure progress towards a goal. They are useful as an intermediate step to defining more specific indicators”
Examples of aggregate indicators:
Effect goal: 50% of project farmers in the village community of Rajbari district will apply pesticide to potato, bean and rice field, using safe and proper technique, July 2012
Aggregate indicators:
% of farmers safely applying pesticides in their fields
% of farmers properly applying pesticides in their fields
Note that the first indicator reflects the farmers whose behavior follows safe application of pesticides. The second reflects farmers who are using their right pesticides with proper doges. Measuring changes for these indicators would provide excellent information as to whether or not behavioral changes related to pesticide use are occurring. However, the terminology used in these indicators is not directly measurable. Other, more specific indicators may be needed.
Operational indicators: Operational indicators are more specific, measurable indicators that help measure change at the level of aggregate indicators. The first step to identify operational indicators is to look at the operational definitions for the word “safe use” and “proper use”. Intervention related to pesticide use will include a number of project activities related to training farmers in specific methods of pesticide application and use. Eventually, the design team will require variables that measure the adoption rate of the methods the project is promoting. These variables, then, are our operational indicators. Operational indicators reflect a subset of the aggregate indicators; they are more specific about what to measure and are based on criteria developed for the aggregate indicator. For each operational indicator, what will be measured should be obvious.
“Operational indicators reflect a sub-set of aggregate indicators. They are variables that are more specific in terms of what o measure and are based on the criteria developed for aggregate indicators”
Examples of operational indicators:
Effect goal: 50% of project farmers in the village community of Rajbari district will apply pesticide to potato, bean and rice field, using safe and proper technique, July 2012
Aggregate indicators:
a. % of farmers safely applying pesticides in their fields
b. % of farmers properly applying pesticides in their fields
Operational indicators:
For aggregate indicator (a):
% of farmer who properly use hand pump sprayers to apply pesticides
% of farmer who were protective clothing while applying pesticide
For aggregate indicator (b):
% of farmers who apply pesticides in the recommended dosage
% of farmer who apply the appropriate pesticides to control the identified pest or diseases
Note: The operational indicator chosen here are unlike the aggregate indicators presented above in that they are tangible, measurable variables. Measurement here would result in indirect but likely to change at the aggregate level.
Characteristic of a good indicator:
Characteristics of a good indicator include the following:
“CREAM”
Clear: Each indicator must be clearly measurable (unambiguous definition) and hence required a precise definition. For example- nature of Extension Officers and quality of Diplomats is not correctly measurable but we can measure quantity of production such as:
Relevant: Indicators should apply to project objectives at the appropriate level in the hierarchy
In general, indicators of activity towards the left of following diagram are more specific.
Input……………..Activities……………….Output……………………Effect………………..Impact
DMore specific Indicator
Economic : The information obtained should be worth of time and money. Monitoring information need to be timely. Further data collection can be expensive and often required a lot of staff time. A tradeoff between the ideal indicator and ease of collection is therefore important.
Adequate: The indicator must have adequate information. They must respect what they claim to be. Manager and other user need to be involved in selecting indicators to ensure that they includes required information.
Monitorable: The indicator need to relate to action that management can take otherwise the data remain unused and therefore not worth collecting.
Why we need indicator?
There are number of reasons why it is important to use indicators:
to determine progress towards achieving planned results;
to inform decision making for more effective annual work plans of projects;
to improve a project’s performance and development impact;
to support sound resource allocation and decisions; and to mitigate the risk to the achievement of results
Logical framework or Logframe
What is logical framework?
Logical framework (as sometimes called logframe) is a project matrix that makes a brief presentation of impact, effect, output and activities along with verifiable indicators, means of verification and assumptions. It provides an at-a-glance view of the project plan for managers and a basis for M&E needs and purposes. The logical framework was tested by the USAID in 1970 for evaluation of technical assistance project. The logical framework approach (LFA) allows a step-by-step conceptualization of important project element. When we design a project using logical framework, we make a series of predictions which we usually called hypothesis. This can be view graphically as follows:
Example of Logframe:
Project completion date: 31/12/2012 ; Date of this summary: 25/07/2010
Project title: Reduction of malnutrition at Nageswary Upazila of Kurigram district in Bangladesh
Project hierarchy with an example
A project has five basic levels such as- input, activity, output, effect and impact. Following example of child nutrition project will show project hierarchy at five basic levels:
Monitoring and Evaluation System
What is M&E system?
The monitoring and evaluation system is a planning and management tool of projects; it is actually the information system used to assess project’s progress, performance and impact. Monitoring refers to regular collection, analysis and use of information within the project about its progress. Evaluation refers to comparison of objectives with accomplishments and how the objectives were achieved.
The M&E system is very important in its ability to assist project staff, target population, and other stakeholders to develop the project throughout its lifespan. As the logframe the structure of M&E system is also characterized by several levels. Each level relates closely to the hierarchy of objectives in the logframe.
Essential components of M&E systems are as follows;
Selection of indicators
Collection of data based on indicators
Analysis of data
Presenting the information, and
Using the information to improve the work
Overview of structure of M&E information system
The following table shows how each level of objectives links with specific monitoring and evaluation assessments.
Table: Overview of structure of M&E information system.
Some clear decisions during design an M&E system
Defining the aim of the M&E system: The system is not just the generation of information. It should be specifically geared to the needs of users. M&E system should be designed to meet specific needs and those will vary according to the nature and aim of the project, concern stakeholder like donor, direct beneficiaries. In order to assess the performance of M&E system to be designed it is important to set overall objectives of the system before it is designed. Describing who need a monitoring system and what for. Who will use the information and for what purpose?
Identification of what is to be monitored: This is done in conjunction with users to determine the key information needs of each user. Project is needed to ensure that all information specified is essential. The selection of indicator is a vital, which reflects progress towards meeting objectives.
The collection and analysis of data: Information may be available from the existing sources and primary data collection would be undertaken for the system. The total technique for data collection would be based on information and its resources. As the system involves some primary data collection activities so arrangement must be made for processing and analysis.
Presenting and using the result: The findings should be presented in a way that can assist the user to take appropriate decisions. The result can be used for planning process, institutional learning and basis for further development. User required different information at varying frequencies. In general the more the senior user, the more aggregated and infrequent the information needs to be.
Organization: Funds, staff and other physical resources available for M&E are identified. This help to assess what form of organization will be appropriate for M&E and in particular, whether it will be appropriate to establish a specific unit the project management structure.
Review the system with users: The ultimate user may have had little involvement in the design of the system. It is therefore important to ensure their agreement to and support the system before its installation.
Step by step designing an M&E system
Step by step designing of M&E system is given below:
Step 1: Review Logframe
Step 2: Identify Indicator
Step 3: Select relevant data
We should keep in mind
Step 4: Design tools
Step 5: Select Methods and Techniques
Step 6: Conduct monitoring and collect data
v Decide technique of data collection
v Go to field with tools or give the tools to the field staff explaining how to use it
v Collect data applying appropriate techniques and put it in the tools
v Review the raw data
v Organize data and ready in the reporting form, and
v Send the report in the next level of flow
Step 7: Analyze data and generate report
Step 8: Circulate the report
Identify to whom we will circulate the report
Determine the time schedule for report circulation
Find and follow the route of report circulation
Send the report accordingly
Ensure the report has been reached to the proper person timely