How to include a randomized controlled trial in your project proposal

Randomized controlled trials, or RCTs, can be an effective way of determining the true impact of interventions. It is a scientific approach to information gathering that includes a group receiving an intervention and a control group that does not. Randomization within the trial means there is minimal bias in the selection of participants, to allow true insights to be generated. For development projects, they are not always a viable option. But when they are, they need to be carefully planned and prepared to ensure they can provide high quality results that effectively communicate why an intervention does or doesn't work — improving aid and development programs for the future. The value of RCTs Dr Tessa Hillgrove, a senior research adviser with The Fred Hollows Foundation, explained to Devex that in knowing “what works” in development practice, there are a range of sources of evidence that can be used. As part of this, RCTs are considered the highest form of evidence — the “gold standard” of understanding what works in development practice. “By using random assignment methods for individuals or clusters of organizations or people by geographical location — more commonly in development research — RCTs allow us to say whether a particular program really makes a difference,” she explained. “For example, does [mobile] health technology encourage more patients to seek follow-up care after surgery, or would they have sought care anyway? Does a special curriculum improve learning outcomes for children above and beyond standard practice?” But Hillgrove explained that their use is not always practical. “RCTs take considerable time, money, and expertise to do well, and that level of investment isn’t warranted for every project,” she said. “Additionally, RCTs require standardized implementation of project activities in different or randomly selected areas, with pre-defined timelines, which can limit their use for real-world settings.” How they are used should then be considered on a case-by-case basis where the circumstances allow. What to think about when considering RCTs Hillgrove’s advice is for organizations to first consider the purpose for wanting to undertake an RCT, including the evidence that is required from this evaluation and its use. The investment of effort in an RCT must be warranted and high quality evidence should be the desired output. Financial and human resources support must also be available. “It takes considerable resources, and expertise, to undertake high-quality research,” Hillgrove explained. “RCTs of development programs are usually conducted through a partnership between a development agency and Australian or local research institute, however it’s important that there’s internal staff with the technical capacity and time to manage the research relationship.” Seeking advice at an early stage to make sure the right support is available is an important step in considering the use of RCTs. Realistic timelines also need to be set to allow for effective RCTs and high-quality results for evaluation. “Good research takes time in setting up the partnerships, designing the protocol and tools, seeking ethical approval, conducting the research, analysis and dissemination,” Hillgrove said. “It’s likely to be a multi-year commitment.” Alternative options should be considered before going down the path of considering RCTs. “The cost [of RCTs] can be high — usually higher than the standard 5 to 10 percent of budget allocated to project evaluation — so it is a trade-off between what’s needed from the evaluation and what time and resources are available,” Hillgrove said. “Seek expertise if you’re not sure on the evaluation approach to take.” Challenges for RCTs in the development sector RCTs are not right for every program or intervention — not just in the development setting, but any research project requiring evaluation. Untargeted projects, such as mass-media awareness raising campaigns or policy changes are certainly projects that do not need to invest in RCTs. Operational contexts should also be avoided for RCTs. “In an operational research context, programs often change during implementation,” Hillgrove said. “This is normal and entirely appropriate in response to lessons from formative evaluation activities, but has implications for the quality of the RCT.” The controlled environment setting required for quality RCT results is not something commonly implemented in development projects. Additionally, the change sought through development projects often takes more time than can be achieved within the RCT timeframes. Observational studies, as opposed to experimental design, are more appropriate in development research according to Hillgrove. Lessons from on-the-ground implementations of RCTs Dr Mike Nunn, research program manager of animal health with the Australian Centre for International Agricultural Research, explained to Devex the complexity of the use of RCTs in a development setting. Nunn oversees research programs in a range of countries that support smallholder farmers to refine livestock management practices, creating healthier stock for improved production and income-generation. Nutrition is linked to this research, which seeks to understand how improved animal handling and health can lead to improved childhood health. One of the programs overseen by Nunn utilized RCTs to investigate changing animal handling practices on the quality of milk. A control village, in which the intervention did not occur, was nearby. Initial results suggested the health of children in the village with the new practice were improving. It was a positive sign for the research. But in establishing the RCT, the researchers underestimated the ability for word to spread between communities. Soon, the new practice was also seen in the control village. Parents wanted what was best for their children and trial or no trial, they were going to implement a practice that would improve health and opportunities. Embedding RCTs into project proposals The key in understanding whether RCTs are relevant to a project is to seek expert advice early in the project proposal stage. “RCTs are very important, but they are not the be all and end all,” Hillgrove advises. “There’s a lot that can be learnt from other study designs.” If RCTs are deemed critical to the intelligence and reporting requirements of the development project, realistic budgets and timelines need to be included in the proposal. Internal capacity within the organization needs to also be planned as part of the project proposal, to support both the research partnership and implementation of the study on the ground. And throughout the project, inception and ongoing meetings should ensure all direct and indirect stakeholders in the project are on the same page. “Research institutes may not [be] aware of the amount of effort project and country office staff have to put into facilitating research activities,” Hillgrove said. The result will be a planned and effective RCT that delivers important insights and intelligence on the effectiveness of development interventions. Devex delivers cutting-edge insights and analysis to the leaders shaping and innovating the business of development. Make sure you don't miss out. Become a Devex Executive Member today.