The fight to save the SDG indicator for foundational learners

The fight is on to preserve a key education indicator to measure reading and math levels among early-grade school children, which may be dropped from the Sustainable Development Goal monitoring framework due to low reporting levels. Tensions are running high within the global education community as different players weigh in on how to preserve SDG 4.1.1a — which focuses on the proportion of children in grades two and three “achieving at least a minimum proficiency level” in reading and mathematics, by sex — which is set to be deleted from the SDG monitoring framework next year unless more countries start providing data. The decision to downgrade SDG 4.1.1a was announced during an October meeting of the United Nations Inter-agency and Expert Group on SDG Indicators, sending shockwaves through the global education community amid fears that important progress in improving foundational learning will be lost. The news inspired numerous blog posts, including from major education donors, think tanks, and advocacy groups calling on the U.N. to be pragmatic in its approach to data collection and use learning assessment data, already being collected through assessments such as the Early Grade Reading Assessment, or EGRA, the foundational learning module of the Multiple Indicators Cluster Survey, or MICS, and the citizen-led assessments of the PAL Network. Doing so would significantly increase country coverage and save the indicator, which given dismal learning levels, especially across Africa, is more important than having technically perfect data, they say. “When learning outcomes are so low it doesn’t make sense to have such a high bar for reporting. Being able to differentiate whether 70% or 68% of kids aren’t meeting a benchmark, doesn’t matter that much; it’s still a disaster,” one researcher told Devex on the condition of anonymity due to the professional sensitivities. “So being able to have good enough data is what’s important. You can get into nuances later along,” they added. However, the U.N. technical agency in charge of the indicator, the UNESCO Institute for Statistics, or UIS, disagrees, arguing that including the assessments as they stand would compromise the quality and comparability of the indicator. They want all data to meet global reporting standards and be vetted by UIS. They also complain that donors have failed to support countries in developing better measurement systems that could produce high-quality data for the indicator. Now, the education statistics community has less than a year to come up with a plan for raising reporting levels or the indicator will be dropped. “There is a need to move beyond the blame game, and for UIS and GEMR to show leadership and take action to ensure data are available in a timely manner,” Pauline Rose, professor of international education at the University of Cambridge and a former head of GEMR, told Devex. Rose thinks a compromise is possible but it needs to be made fast. “It needs a low-stakes, low-cost approach that is straightforward to collect and analyse data in a timely way for national and comparative purposes. The data need to allow for comparisons of different population groups and the assessments need to be age-appropriate, avoiding being a burden on children. “We knew all this 10 years ago, and we can’t wait another 10 years for data to be available,” she said. A hot topic Experts agree that the SDG 4.1.1a indicator is crucial to improving global learning outcomes because it tracks children’s education progress, or lack of it, when there is still time to intervene and correct. Indeed, foundational literacy is critical for SDG progress across the board since children who fail to master the foundational skills by grade four will find it much harder to catch up later on, and are more likely to drop out of school and struggle to get good jobs. They are also more inclined to have illiterate children, thus creating “intergenerational transmission of poverty and vulnerability,” according to the World Bank. With an estimated 70% of children in low- and middle-income countries in “learning poverty” — unable to read and understand a simple sentence by age 10 — it is vital to keep a pulse on foundational learning levels, advocates argue. However, collecting sufficient data on reading and math in these early years has proved difficult. The prospect of losing SDG 4.1.1a has ignited intense debate in the global education blogosphere over how best to save the indicator. Posts from the Center for Global Development, the Global Coalition for Foundational Learning, the Global Education Evidence Advisory Panel, or GEEAP, and the civil society-led assessment group, PAL Network, argue that reporting levels will increase if UIS accepts data from several existing assessments that measure foundational learning. This would bring coverage over the 50% threshold needed to keep the indicator and would make good use of existing resources when data and funding are both in short supply. Dropping the indicator is not an option, they argue. “Losing SDG 4.1.1a would signify that focusing on foundational learning skills in the early grades is no longer a priority in global education monitoring,” the coalition donors said in a December blog. Writing to Devex, Ben Piper, head of global education at the Bill & Melinda Gates Foundation, added: “It’s important to have a global indicator for foundational learning because what gets measured gets managed. Without comparable data over time on the basics, we won’t know if we are making progress and we won’t be able to learn from what has worked across countries,” he wrote. Furthermore, members of GEEAP, which includes researchers and academics, claim that the data from the existing surveys, while not perfect, is robust enough to be included. “Our professional opinion is that several of the current surveys which measure learning across multiple low- and middle-income countries are of sufficient quality (and have been used to produce sufficiently credible data in recent years) to contribute to a valid stock of learning data, and hence, to be counted against the requirements of the Technical Cooperation Group (TCG) chaired by UIS,” they wrote in January. They, and others, want UIS to be “pragmatic” in its approach and allow this data to count toward SDG 4.1.1a to “leverage the existing measures to continue to monitor foundational learning,” the panelists wrote. Many low-income countries are also keen to see the indicator preserved. “If we have that indicator, then as countries we are able to identify the challenges early enough so that we can intervene then rather than waiting until the end of primary or during secondary,” Shadreck Nkoya, a director at the Examinations Council of Zambia, told Devex. The counterargument UIS disagrees and hit back last week with its own blog entitled “setting the record straight,” in which director Silvia Montoya described the alternative assessments as being “unable to withstand the scrutiny of global reporting,” for a variety of reasons, including that they do not assess reading to the globally agreed minimum proficiency level and that the data produced is not internationally comparable. “Accepting the results of assessments that we positively know are not measuring the minimum proficiency level and are loose in their documentation is not likely to lead to progress,” Montoya wrote. Montoya’s blog was quickly followed by another from the Global Education Monitoring Report, or GEMR, which accused donors of pushing assessments that fit their own needs instead of taking a country-driven approach to building local assessment capacity and meeting country education needs. Epha Ngota, the coordinator of the National Assessment Centre, Kenya National Examinations Council, or KNEC, wants SDG 4.1.1a to remain but not if it means sacrificing quality. “If we reduce the rigor of reporting on the indicator, it only means those who are lagging behind will continue to lag behind as nobody will be checking comparably or early enough,” she told Devex by phone. A way forward? Instead, UIS wants countries to adapt existing assessments or create new ones so that they meet new global reporting standards, which UIS is currently compiling with an expert technical group. However, according to Lee Crawfurd, an education research fellow at the Center for Global Development, some of the criteria and tools being proposed set an unrealistic gold standard quality of data for countries, including that all assessments must include at least 40 questions — 20 for reading and 20 for mathematics. “It’s a fairly high bar and so there’s a good chance the foundational skills indicator will be dropped altogether because it won’t be possible to reach coverage of enough countries with the new survey by the 2025 deadline,” he told Devex. But even if the MICS, EGRA, and PAL Network assessments can be brought into line with the technical standards, UIS will still have to “vet” the assessment reports to make sure they meet the global reporting criteria, according to Montoya’s latest blog. No timeline for this has been given, leading some education experts to question whether their data could be included in time to meet the deadline. Another solution would be to include MICS, PAL, and EGRA tools as interim reporting, which will leave time to make adjustments to the assessments. However, though a logical solution, this does not seem to be on the table, education experts told Devex. Setting such a high standard for assessment data may also have negative consequences for the children taking the tests, according to Crawfurd. “In any low-income countries that do rush to field the new tests, the questions will likely be too hard for most children, who will have to sit through the awkward and unpleasant experience of not being able to answer any questions,” he explained. UIS also proposes an alternative solution, whereby countries can ask donors to pay for a new type of foundational learning assessment, pioneered by UIS in partnership with the Australian Council for Educational Research, or ACER. Known as Assessments for Minimum Proficiency Levels a, or AMPLa, the test is part of a wider group that adheres to global technical standards and methods. To date, AMPLa has been piloted in Zambia, Gambia, Kenya, Lesotho, and India and this data can be included in the next round of 4.1.1a monitoring. The World Bank has said it will support the rollout of AMPLa in more countries, a spokesperson told Devex. However, UIS’ documents on AMPLa admit that the test may be too hard for many students and recommend the assessment be complemented alongside others, such as EGRA. “40 questions is definitely too much for 9 year olds. The assessment needs to be age-appropriate, easy to implement and analyse, low-cost, and avoiding high-stakes and non-contextually appropriate approaches,” Pauline Rose told Devex. Introducing AMPLa will be technically complex, expensive, and take time. There is also no guarantee it will create enough coverage given the significant gaps that need to be filled in a very short time. USAID is the biggest global education donor and is especially focused on foundational skills. A spokesperson told Devex it is important not to lose sight of the practical purpose behind SDG indicators such as 4.1.1a. “We must keep our eye on the goal of measuring skills at early grades so that policymakers and the community understand children’s learning levels and take early and appropriate action to ensure all children gain the foundational learning skills they need for future academic achievement.” “The criteria for reporting on this indicator should consider the statistical, time, and financial costs associated with the requirements and seek to make reporting feasible for all countries,” the spokesperson said. But in the end However, despite the flurry of activity around saving SDG 4.1.1a, some experts, including UIS, seem resigned to SDG 4.1.1a being dropped. “There is a high chance that low country coverage of the indicator on minimum proficiency in early grades, 4.1.1a, will lead to it being dropped from the list of global SDG indicators in 2025,” Montoya wrote in February. Pauline Rose agreed: “I fear that perfect will be the enemy of the good, unless there is a shift in the current position,” she wrote in an email to Devex. But while many are convinced that dropping the indicator will be a disaster and mean foundational learning falls off the global education agenda, Montoya is more sanguine. “Dropping an indicator from the global list for pragmatic reasons (i.e., because countries are not reporting) does not mean that the indicator loses its relevance: the out-of-school rate is not a global SDG indicator, yet it continues to receive global attention,” she wrote. Update, April 4, 2024: This article has been updated to clarify the nature of the Global Education Monitoring Report.