WASHINGTON — An independent evaluation of a project in Liberia that outsourced the management of nearly 100 schools to mostly international private operators has revealed that student learning improved by 60 percent during its first year.
However, the evaluation also raised serious concerns about the financial sustainability and cost effectiveness of the program.
The report, titled “Can outsourcing improve Liberia’s schools?”, was published Thursday by think tank the Center for Global Development and research NGO Innovations for Poverty Action. It offers the first independent findings from the Partnership Schools for Liberia (PSL) program launched by the country’s Education Minister George Werner in 2016.
The project was developed in a bid to transform the country’s lagging education system, which has been decimated by years of civil war and the Ebola epidemic. According to the United Nations, nearly two-thirds of children in Liberia miss out on primary school education, the highest rate in the world.
Read more Devex coverage
Under the PSL, management of 93 public primary schools was outsourced to eight contractors — a mixture of for-profit and nonprofit education providers and NGOs, only two of which are Liberian. Since its launch last year, the pilot has attracted vocal opposition from teachers’ unions in Liberia and abroad, as well as from a range of academics and development professionals who argue against the privatization of the education system.
Under the scheme, most operators get the same $50 per student from the government that all state schools receive, and an additional $50 per pupil from a pool of philanthropic funds managed by Ark, a U.K. education charity hired by the Liberian government to work on the program. Operators are then free to supplement their budgets with any additional finances they raise.
With Liberia heading to the polls in October, the PSL has become a political hot potato. The government is currently working to double the number of participating schools to nearly 200, using the same eight operators — but it remains to be seen whether the first-year results are positive enough to ensure the program’s survival post-election.
The randomized control trial evaluation suggests that learning outcomes, teacher attendance, and pupil and parent satisfaction all improved during the first year of PSL. The evaluators found that, on average, partnership schools improved learning by an equivalent of about 0.6 years of school, and that PSL teachers were more likely to be in school and teaching classes during random spot checks.
Students spent roughly twice as long in the classroom per week, and were more likely to say they enjoyed school. Parents were similarly more likely to say they were satisfied with their children’s education.
However, the researchers suggested that such gains may not be sustainable or cost effective, with the warning that the program may have negatively affected non-participating schools and their students.
“There is solid evidence of positive effects for Liberian children during the first year of PSL,” the evaluation report states, adding that “the program has yet to demonstrate it can work in average Liberian schools, with sustainable budgets and staffing levels, and without negative side-effects on other schools.”
Of the eight private operators contracted through PSL, only one showed negative results in terms of student learning outcomes. According to the RCT results, Omega Schools — a low-cost school provider based in Ghana, but founded in the United Kingdom — saw learning levels decline slightly during year one of the program.
Liberia’s education ministry welcomed the report as positive news. In a statement sent to Devex, Minister Werner said: “We are pleased to see strong, independent evidence the PSL program is achieving the objectives we set out: improving education quality and increasing learning gains.”
The statement also described the first year costings per pupil as “illustrating [that] financial sustainability is within reach as economies of scale are achieved.”
However, the report suggests that spending per child during the first year of the program ranged between the operators from $40 at Omega Schools to $663 at Bridge International Academies (BIA), which critics say is too high. A spokesperson for BIA noted that the organization does not receive funding from Ark under its agreement with the Ministry of Education, and that it comes from philanthropic sources.
While both the Ministry of Education and BIA have disputed some of the evaluation’s findings, some education advocates have dismissed the positive results as “modest and uneven,” especially considering the amount of additional resources being put into PSL schools.
“Improvements are much less than lower-cost alternative reforms,” said David Archer, head of education at ActionAid. “It does seem the outcomes fall far short of what would have been expected and there are serious questions raised about the sustainability owing to the high costs.”
Bridge International Academies
Much of the opposition to the PSL has centered on one operator: Bridge International Academies, a for-profit chain of schools that resembles the American charter school model.
Critics have found fault with BIA at a variety of levels. For example, they argue that the use of tablet-based technology to provide teachers with detailed lesson plans reduces teacher freedom and interaction with children. Critics allege that students have been turned away in order to keep class sizes smaller, and that BIA has requested a high proportion of teachers be reassigned to non-PSL state schools.
Bridge International Academies set up 63 low-cost private schools in Uganda. In 2016, the Ministry of Education decided to shut them down, arguing they cut corners and didn't meet standards. Bridge refuted the allegations and some see the crackdown as part of a broader campaign from public sector unions to limit private enterprise. Devex investigated.
They also claim that because BIA has deep-pocketed funders — including Mark Zuckerberg, Bill Gates, the U.K. government and the World Bank — it has an unfair advantage over other operators. In August, a coalition of more than 170 civil society organizations published a statement calling on donors to stop funding BIA, citing a range of issues, including concerns about its track record in Kenya and Uganda, where it was threatened with closure for noncompliance with national education standards. BIA published an open letter in response.
Addressing its operations in Liberia, a BIA spokesperson told Devex that: “We are incredibly proud that Bridge PSL pupils have been learning at double the speed of their peers in traditional public schools … These extraordinary results are due to the bold vision of the Liberian [ministry of education],” adding that “the ministry’s PSL policies have been validated.”
High costs and questions about sustainability
Evaluators concluded that the costs incurred by the operators “greatly exceeded” the targets set, calling into question the sustainability of the program over the long term — the importance of which the Liberian education ministry has stressed.
“Ultimately, the PSL program must be financially sustainable. It cannot rely on the goodwill of international donors forever,” Minister Werner said during a session organized by the American Enterprise Institute in Washington, D.C., in July.
The RCT report lists self-reported expenditure provided by five of the eight contractors for year one. These figures show that the majority exceeded the program’s long-term target of $50 per student, ranging from $40 for Omega Schools to $663 for BIA. However, the evaluation notes that this includes one-off start-up costs, recurring fixed costs that may reduce per unit as the program expands, as well as variable costs per student. As a result, long-term costs may be “considerably reduced.” When start-up costs are taken out, recurring costs per child are estimated to range from $39 for Omega Schools to $321 for BIA.
Some pupils excluded and teachers pushed out
As had long been claimed by critics of PSL, the evaluators found that the program led to some children being turned away from their school of choice. However, six of the PSL schools saw an increase in enrollment numbers.
“Overall, enrollment levels did not change. But there are signs that some children were turned away from their school when PSL arrived,” the report states. It adds that most were absorbed by other, non-PSL schools.
In response to these concerns, a BIA spokesperson stressed the importance of smaller class sizes in improving learning outcomes: “A small classroom with more than 100 children, no space for the teacher to circulate and give feedback, is not an environment conducive to learning,” they told Devex. “What the RCT demonstrates is that this new enrollment policy is beneficial to students’ learning.”
The evaluation also noted that PSL operators were able to request the transfer of low-performing teachers to other, non-PSL schools. It found that BIA had dismissed half of the teachers working at its assigned schools in this way, while two other operators, Liberian Youth Network and More Than Me, had lost a much smaller number of teachers. According to the evaluation, this raises concerns about the fairness of performance scores, the sustainability of the program as it expands, and negative consequences for non-PSL schools as they absorbed additional pupils and under-performing staff.
It also appears to contradict the government’s stated aim of allowing PSL operators “limited ability” to request teacher transfers. According to the evaluation, the ability to “push excess pupils and under-performing teachers onto other government schools” is the result of “contracting flaws” within the PSL.
The BIA spokesperson told Devex: “Bridge does not make education policy decisions in Liberia — the Liberian Ministry of Education does, and we simply implement their decisions. Removing illiterate teachers and ghost teachers was part of the pilot program design by the Ministry, as well as a national policy being implemented in public schools across the country. Teachers transitioned from some classes could not read at a first grade level. It’s never a good idea to have illiterate teachers leading learning on reading and writing.”
The Ministry of Education told Devex that all teachers who were moved had been given training opportunities and reassigned to other schools, adding that “in year two, we have implemented new provisions in providers’ contracts which mean that 75 percent of teachers will stay in the schools to which they are assigned.” The spokesperson also said that children previously enrolled in schools would be given priority for registration in year two.
The Liberian education minister has said he plans to double the size of the PSL pilot and add approximately 100 new schools to the program this year. That decision was taken against the advice of multiple education activists, academics and the RCT evaluators, who sent an open letter to the ministry earlier this year advising against scaling up before the RCT results came in.
BIA, which runs 25 of the schools and is the single largest operator under the contract, is set to take on an additional 43 schools. Rising Academies, which runs low-cost private schools in Sierra Leone, will receive 24 new schools, bringing its total up to 29. This is due to strong performance during year one from both providers. By contrast, Omega, which currently operates 19 of the PSL schools, will only be given two more. Devex was unable to reach Omega for comment.
According to the ministry, the second year of the PSL will see the introduction of a number of reforms designed to address some of the concerns raised by the evaluation. In addition to new provisions regarding teacher transfers, a spokesperson said that contracts will include “new financial reporting structures in Year 2, so we can more closely monitor providers’ progress towards sustainability.”
It is also working toward building a new “Education Delivery Unit” designed to ensure “effective management of the PSL and the integration of PSL learnings into the broader education system.”
Updates, September 7 and 8: This article was amended to clarify that Mark Zuckerberg is a funder of Bridge International Academies, and to include additional comments from Bridge provided after publication
Read more international development news online, and subscribe to The Development Newswire to receive the latest from the world’s leading donors and decision-makers — emailed to you free every business day.