Measuring impact: What makes it harder for nutrition?

    A woman weighs her child in a clinic supported by the Kuwait Patient Helping Fund Society in North Darfur’s Abu Shouk camp for internally displaced persons. Photo by: Albert González Farran / UNAMID / CC BY-NC-ND

    Last week, I was asked to speak at the Institute of Development Studies Nutrition Summer School.

    It was a wonderfully sunny day and a chance for me to reflect on the challenges for all of us to ensure that scaling up programming and money will count so child nutrition can improve.

    A year ago at the Nutrition for Growth Summit, the U.K. Department for International Development committed to triple spending on programs that directly tackle child malnutrition. We also agreed to ensure an extra 8 percent of our programs that have an indirect and long-term impact on child stunting — such as agriculture or education — will measure their impact on nutrition indicators. Since 2010, we have been rapidly increasing our use of independent evaluations to measure impact, with over 300 evaluations commissioned, 36 of those of our nutrition programs.

    I think there are some unique challenges for measuring the impact of nutrition programs:

    1. What makes a child well or poorly nourished is the result of many influences and inputs, not just one sector or intervention. Whether they have a toilet, their mother went to school, or they get sick often, will all have an impact. Therefore it is hard to pin down the specific causes of improvements, unlike more direct programs.

    2. The technical experts that need to champion nutrition come from very different communities. Agricultural scientists will be happier talking about yields, while gender specialists will want to try and measure empowerment. However, these groups need to have common understandings and monitoring and impact frameworks, a tall order when they think about the world in such different ways.

    3. On top of that, The Lancet recently highlighted that 165 million children globally are stunted or will not reach their full potential, and that 80 percent of stunting will be tackled through “indirect” or nutrition-sensitive programming, like empowering mothers or making agricultural crops more nutritious. These are sectors, though, that have traditionally not considered nutrition important, and we know little about what works in these sectors for nutrition. These gaps in knowledge mean we need to take risks and test what could work, and measure it, even if the gaps in knowledge will make risk of apparent failure higher. For example, we need to be brave to test how to make education for girls’ better deliver for them and their children’s nutrition.

    4. We have a huge shortfall in data for nutrition on all key indicators, such as underweight children, stunting and wasting. In addition, different programs have used different indicators just to further complicate issues. Lawrence Haddad and his team at the International Food Policy Research Institute are currently pulling together the new Global Nutrition Report to bring together the fragmented and sparse data on nutrition. It will be launched in November.

    5.  That, however will take time to measure improvements in child nutrition and children reaching their ideal height. It takes even longer to see how a girl’s education or her use of family planning can impact on the nutrition of her children or her grandchildren.

    6. There is a lack of capacity globally in nutrition and public health. This is mirrored in evaluation, so two gaps in global expertise collide make it more acute for assessing the impact of nutrition investments. In addition, measuring a child’s nutrition status is tough, as children wriggle when being weighed, they cry and won’t stand still, making it hard to measure their height.

    7. There is a history in the humanitarian field and in food aid of a lack of impact measurement, and sometimes an active resistance to this. This is due to the myth that the rapid response needed in relief efforts is not easily evaluated.

    8. The nutrition community has been made up of many people who tend to promote single intervention solutions or “techno-fixes” such as vitamin A, micronutrients or de-worming.

    While its is extremely important to know that nutrition will only improve by “fixing” many things, evidence of impact needs to be across the range of single interventions.

    Lastly, not to be too depressing about the unique challenges, forearmed and forewarned as we are, once we have great results from our nutrition program, we need to make sure that someone is listening to them and adjusting their policies accordingly. But in the absence of a nutrition minister (have you ever heard of one?), we need to find our champions and carve out advocacy messages that are appealing.

    It is, however, all possible. There are some great examples the impact of the recent Lancet Nutrition Series, highlighting burden and calculating impact of successful interventions. Look at the successful advocacy campaign to promote of the orange sweet-fleshed potato.  Through exciting promotion techniques — including a one-man flash mob who wears a potato suit (I kid you not, I have seen him in our offices) and serious evidence that appeals to a range of technical experts (economists, gender specialists, nutritionists) — this bio-fortification program has gone from strength to strength, and it is now grown in many more countries.

    So although we are up against some unique challenges, showing impact in people’s nutrition is critical if we are tackle the huge burden of disease it causes, with 45 percent of child deaths globally due to undernutrition, we cannot afford to go wrong. We need to fund in new risky areas where we don’t know what works yet, measure impact well and present that evidence in a way that appeals to a much wider community that the already converted.

    Want to learn more? Check out Feeding Development's campaign site and tweet us using #FeedingDev.

    Feeding Development is an online conversation hosted by Devex in partnership with ACDI/VOCA, Chemonics, Fintrac, GAIN, Nestlé and Tetra Tech to reimagine solutions for a food-secure future from seed and soil to a healthy meal.

    About the author

    • Anne philpott1

      Anne Philpott

      Anne Philpott is head of evaluation support group at DFID, working for 5 years on DFID India’s response to under nutrition in India — home to 40% of the world’s underweight children. She worked in the ‘Nutrition for Growth’ summit team — an event which ended with 94 governments and organisations making commitments to tackle nutrition.