Evaluating field projects isn't easy. Sometimes, vague terms of reference may be to blame, Francesca Mosca said. EuropeAid's director of quality of operations talked with Devex about ways to ensure effective aid delivery.
Mosca led the European Commission's delegation to Zimbabwe for four years before moving to Nicaragua to lead its Central America operations, from 2005 to 2008. Before that, she worked for the commission in areas such as food security and assistance, human rights, democracy, governance and cooperation with East and Central Africa.
Devex spoke with Mosca about the problems her team faces when evaluating field projects - and complaints from partner organizations about the commission's very complex project application procedures.
What obstacles, if any, does EuropeAid encounter when evaluating projects in the field?
Projects are usually evaluated directly by our delegations. Here at headquarters, we mostly deal with thematic evaluations or with those in common with the Relex [External Relations] family. But when it comes to evaluating a project, it's our delegations who do that.
Having said that, we found that our difficulties are maybe related to terms of reference, which are too generic or not very well expressed. So, in the end, evaluation is difficult and it doesn't produce the desired results.
Sometimes, also, experts [evaluators] are not as qualified as we would want them to be. Sometimes there is no follow-up after our recommendations.
What we do is use the recommendations we get after evaluating a project to modify our field operations. Although, this is not always the case. Sometimes [our projects] don't continue in the same sector, and our recommendations may not translate. This is why sometimes they are not observed. … But I would say difficulties are mostly due to excessively generic terms of reference and the quality of evaluators.
Other difficulties we encounter are due to the fact that many evaluations are carried out ex-post, so sometimes technical personnel and partners who worked on a project are no longer there; therefore, it is difficult to access the entire documentation as we do when the project is still in place.
But apart from evaluations, we also have another system named ROM [results-oriented monitoring]. This is a much-lighter procedure than an evaluation. It consists in very short missions of maximum three or four days, which we carry out while the project is still in place in order to evaluate whether it is well-organized and so that - should problems arise - they can be fixed by the commission. So, it is a much more agile tool, and it should be a management instrument.
Do you think this easier procedure will be employed more and more often by EuropeAid?
It has existed for some years already. I personally think it is a useful management tool, and because it is a useful management tool, its methodology should be employed by delegations, especially when they go verify and visit projects. If we could get to that point, I think we would obtain excellent results.
I remember that when I was heading a delegation some results-oriented monitoring experts came, and we asked them to share their methodology with us because it would have been useful when we visited projects in the field. But there we have a human resources problem with the [small] number of people in some delegations, which have many vacant positions and we don't have enough personnel.
We spoke about the difficulties encountered by EuropeAid in the evaluation of field projects. Are there any obstacles partners in the field complain about?
Answering this question is really easy. Partners in the field always tell us the commission is not flexible and fast enough, and our procedures are too complex. For organizations operating in the field, which are often local partners, it is difficult to respond and start projects according to our calls, which are published online [as well as on the Devex Web site, the editor]. Complying with all of our requirements is really complicated.
Is EuropeAid considering easing its procedures in response to these complaints?
I think easing the procedures should always be among our aims. Since I've been working here and even when I worked in the delegations, I always heard about work groups trying to ease the procedures. My experience, though, showed me we haven't been able to do much. But it is not only us. The [European] Council takes part in the setting of the financial regulation. It is not adopted by the commission. It is only presented by the commission to the member states. So, all we can do is ease the procedures within that context and ensure the observation of the financial regulation. We cannot do more than that.
How can current and potential partners stay up-to-date on collaboration opportunities offered by EuropeAid?
If we talk about non-governmental partners, they normally send projects when we launch a call for applications. They [calls] are always advertised on the commission's or EuropeAid's Web site. So, they are always informed about them.
Are there any areas in which EuropeAid is particularly focusing when it comes to partnering opportunities?
There are a number of thematic areas in which we operate. "Human rights" is one of them, as well as "investing in people" or "asylum and migration." So, there are several thematic lines, including the "food facility," and they [partner organizations] can send projects within all of them. Some calls [for projects] are based here in Brussels, others are decentralized where delegations are.
