Exclusive: Chemonics battles wave of challenges with $9.5B health supply chain project

This story is part of Devex’s ongoing investigation into the largest USAID contract in history. This Thursday, we invite our Executive Members to an exclusive digital event with Senior Reporter Michael Igoe, who broke the story, in conversation with Managing Editor Paul Harris. RSVP here.

WASHINGTON — The contractor managing a multibillion dollar, United States government-funded global health supply chain project has limited time to prove it can overcome a long list of problems that have plagued this lifesaving effort.

Chemonics International is under mounting pressure to show that the largest project ever funded by the U.S. Agency for International Development has turned a corner. Facing a December performance deadline and scrutiny from a growing list of U.S. lawmakers and watchdog groups, project leaders are pointing to a range of actions they have taken in recent months to improve delivery times and cut down on chronic delays.

Internal documents obtained by Devex and interviews with USAID officials and current and former employees describe a massive logistical operation thrown into disarray by problems with planning, communication, information systems, and management — and a government contractor fighting to regain control.

The Global Health Supply Chain — Procurement Supply Management project is an “indefinite delivery/indefinite quantity” — or IDIQ — contract worth up to $9.5 billion that coordinates the purchase and delivery of lifesaving health products in dozens of developing countries. Devex first revealed that the supply chain project was encountering problems in August, when reports showed that performance metrics hit all-time lows in early 2017. Only 7 percent of shipments delivered during the first three months of the year arrived “on time and in full.”

In April, Chemonics and USAID jointly agreed to a corrective action plan that ends in December. Chemonics representatives say they are already meeting the improvement benchmarks they committed to, though it is not clear whether doing so will satisfy the growing number of U.S. lawmakers, oversight officials, and USAID leaders who have expressed concerns about the project’s performance — and raised questions about USAID’s decision to award it to the contractor in the first place.

Beyond the ramifications for government contracting and internal management decisions at USAID, this massive global health supply chain system forms the backbone of multiple U.S. government efforts to combat the world’s deadliest diseases, including PEPFAR, the flagship global HIV/AIDS initiative. For patients in need of malaria drugs or antiretroviral HIV treatments, the difference between a functioning and a faltering medical supply chain in their country can be the difference between life and death.

‘Performance challenges’

On April 13, 2017, Sherif Mowafy, the USAID contracting officer’s representative — or COR — overseeing the contract, sent a six-page memo to Chemonics and GHSC-PSM leadership outlining a litany of concerns about their implementation of the project.

The memo, which Devex obtained, raises complaints that range from mishandling urgent orders, to bad data and IT systems, to nonexistent oversight and inaccurate financial reporting. It refers to “systematic challenges in PSM’s procurement and delivery systems,” “chronic delays in shipping orders from suppliers,” and a degree of poor planning that had the project’s government clients “questioning the level and quality of service being provided.”

“Such challenges are overshadowing many of the positive achievements of PSM so far,” the memo reads, before requesting a corrective action plan be delivered to USAID within eight days. That action plan, which USAID and Chemonics agreed to, is completed in December.

“December is one of those months when we really need to look at — have we seen improvements in the on time delivery rates, ... have they completed all of the actions that they committed to do in their action plan, and has that resulted in performance improvement,” a USAID official who spoke on background told Devex.

The official did not elaborate on what would happen if the implementer failed to show satisfactory improvement by the end of the year.

Chemonics executives point to a range of actions they have taken to restructure the project, including a leadership overhaul, streamlining the ordering process, a more realistic order promise tool, aggressive supplier management, and better early warning systems for potential product stockouts and delays.

While recent reports have shown the GHSC–PSM project’s key metrics trending upwards — Chemonics leadership says their “on time delivery” reached 60 percent in October — USAID officials still described the last quarterly report as “unacceptable.” Representatives from both Chemonics and USAID point out that the way the indicators are designed means that improvements made to the supply chain take months to translate into better numbers, and they both say they expect that will happen.

In the latest quarterly report — for the three months ending Sept. 30 — Chemonics reported that only 23 percent of adult ARV shipments arrived on time and in full, up from 3 percent in the prior quarter. Chemonics has begun reporting an additional statistic — “on time delivery,” or OTD — which the project team says is a more accurate representation of current performance. Adult ARVs reached 48 percent “on time delivery” in October, Chemonics informed Devex.

The USAID official who spoke to Devex on background said the agency has taken additional steps to hold Chemonics accountable. The agency issued a moratorium on salary increases and on promotions until Chemonics improves their performance. USAID has also submitted a negative performance review into the government system that reports on past performance any time an organization bids on a government contract, the official said.

Lost in transition?

While Chemonics executives have publicly acknowledged that implementation has not met expectations, the contractor also argues that issues during the transition from USAID’s previous supply chain projects — implemented by a different group of contractors — contributed to the accumulation of an order backlog and to processing delays.

Chemonics representatives allege that USAID’s former supply chain partner — the Partnership for Supply Chain Management, led by John Snow, Inc. — was reluctant to collaborate during the transition from one project to the next.

“Our predecessors’ reluctance to collaborate left us at times with short deadlines to accomplish large tasks,” Jane Gotiangco, Chemonics’ director of strategic communications, told Devex in a written statement.

“For instance, in one case we received four days’ notice of an official handover of a warehouse containing $16.9 million of commodities. This was contrary to our practices and experience all of which pointed to an expectation that information would be openly shared between the predecessor projects and GHSC-PSM,” Gotiangco wrote.

Representatives from the Partnership for Supply Chain Management flatly reject that charge.

“PFSCM and JSI were fully collaborative and cooperative in the transition to the new contractor, and any suggestion to the contrary is simply a deflection,” Rich Owens, PFSCM’s director, told Devex in a written statement.

Owens sent Devex a copy of USAID’s July 2017 “Contractor Performance Assessment Report,” which evaluated PFSCM’s implementation of the former supply chain project.

“To the last minute of the life of the contract, management was committed to successful completion and leaving a legacy of success, continued flow of commodities and services,” USAID’s evaluation concluded.

While recognizing that Chemonics is not meeting its own self-imposed target of 90 percent of shipments delivered on time and in full — or OTIF — the project’s leaders also argue that the way OTIF rates have been calculated for GHSC-PSM is more stringent than that of previous supply chain projects.

“Since both Chemonics and our predecessors use the same term, OTIF … , it would be natural to assume that we have all been using the same criteria to measure OTIF delivery rates, but that is not the case,” James Butcher, the acting GHSC-PSM director, told Devex in a written statement.

The two previous projects applied four and eight-week windows for on time delivery, respectively, while GHSC-PSM has a three-week window, Butcher explained. This difference “makes comparisons between the projects inaccurate,” Butcher said.

The project director also told Devex that GHSC-PSM abstained from using “reason codes” in its OTIF calculations. These are allowances applied to deliveries delayed by circumstances outside the supply chain manager’s control — for example, a hurricane, a global product recall, or a coup d'état.

“While we do not have an official complete data set from [the previous project], we understand they applied reason codes throughout their project. Until July 2017, GHSC-PSM did not,” Butcher told Devex.

“These two material differences in measurement are not meant to be excuses — GHSC-PSM’s numbers must meet USAID’s expectations. However, they do make comparing the previous projects and GHSC-PSM problematic,” he added.

People in key places

Chemonics’ performance challenges are not only evident in comparison to past projects. People working on the GHSC-PSM project shared numerous complaints about the supply chain’s structure and operations.

Former employees who spoke to Devex on the condition of anonymity attribute many of the problems to a failure to put the right people in the right positions — and with the information and authority to make good decisions about product sourcing, pricing, and delivery. Operating a health commodities supply chain requires a unique combination of expertise in logistics and in health product markets — or an organizational structure that effectively combines those skills into effective teams.

“The people who are ordering have no idea what the drugs are,” said one former employee.

That observation echoes concerns raised in USAID’s April memo.

“Some product buyer groups and client relations teams do not possess intimate, or at least adequate, knowledge of the products they are working with,” the memo stated. “This has resulted in incorrect processing of orders, improper handling of orders (e.g., bundling products that appear similar, but in fact need to be prioritized differently), and confusing and/or incorrect information being provided to Missions/clients.”

Another former employee offered a glimpse of what that lack of product knowledge looked like during the first year of implementation. Nine months into the project, according to the former employee, a business analyst responsible for malaria nets asked whether or not they were for daily use — a question the former employee felt demonstrated a basic lack of knowledge about the items they were ordering and shipping.

“These are big, bulky, heavy products that are very difficult to get to the end of a supply chain, and we need these to last at least four years. These people can’t just walk to Walmart and buy it and throw it away at the end of the day,” the former employee said.

“There’s a lot of people in key places on that contract who don’t know anything about international development, or these supplies, or third-world supply chains,” said the former employee, who left the project several months ago after concluding that a previous attempt by Chemonics to reorganize the project was not going to be successful.

USAID reached a similar conclusion about the project’s initial restructuring efforts.

“The GSC team has been reorganized several times, which resulted in some disruption without significant improvements in performance,” said the April memo from Mowafy.

“The revised structure has resulted in siloed teams that do not communicate effectively with each other. Due to such frequent changes in the management structure, USAID staff, Missions, and vendors struggle to figure out the project’s operating model,” the memo read.

After identifying the performance issues, Chemonics made extensive personnel and leadership changes to the project, including firing and replacing its top managers, which the USAID official who spoke to Devex on background said was the contractor’s own decision. Chemonics also brought in “surge staff” to help fill key positions and improve the project’s performance.

These actions did not incur additional costs for the U.S. government, according to Eric Howell, chief financial and compliance officer for Chemonics’ Global Health and Supply Chain Office.

“We can confirm the GHSC-PSM did not exceed its approved labor ceilings or cause additional cost to the U.S. government for implementation of its action plan to improve performance, and more specifically to improve the project’s on time delivery,” Howell said in a written statement to Devex.

‘A giant red flag’

The Chemonics team has repeatedly stated that none of the delays caused by the GHSC-PSM project have forced patients off their treatment.

“Targeted analysis of our late HIV/AIDS orders shows that most reported stockouts were limited to the central warehouse, however there has still been availability in other levels of the health system,” Butcher, the acting IDIQ director, told Devex.

“To date, we have not found evidence that any patients have been forced off treatment due to GHSC-PSM’s late deliveries,” Butcher said.

The USAID official who spoke to Devex on background agreed with that statement, but added that delays have caused at least one “negative outcome.”

“I have not seen any indication that patients have not had access to treatment that were on treatment,” the official said. “We do know of one country where there were delays in expanding the treatment to new people who were not on treatment previously. That is a negative outcome, obviously.”

Despite these assurances, one of the documents Devex obtained suggests USAID officials have openly worried about the possibility that patients could go without drugs.

According to a readout of a July 26 meeting with the team working on HIV commodities, Mowafy, the contracting officer’s representative, asked of stockouts occurring in Côte d’Ivoire: “Will there be patients who miss their drugs?” He then referred to this possibility as “a giant red flag.”

In several cases, stockouts at the central medical stores have been mitigated by the availability of drugs further down the supply chain — at the regional level, or in health clinics, for example. In other cases, the availability of a substitute drug has filled the gap between agreed and actual delivery dates, according to the internal documents. The USAID supply chain is also not the only one operating in many of these countries. The Global Fund to Fight AIDS, Tuberculosis and Malaria, in particular, delivers large amounts of similar commodities, and countries are able to draw on one supply chain when problems arise in the other.

In response to its performance challenges, Chemonics put in place an early warning system to detect potential delays. Butcher, the acting IDIQ director, said that the project team’s visibility into the supply chain allows them to track stock levels, shipment progress, and shift priorities based on urgent need.

According to USAID’s April memo and to former employees who spoke to Devex, that kind of visibility into stock status and shipment progress in countries was previously severely lacking.

A former employee who spoke anonymously told Devex that the project team tended to learn about stockouts only after USAID Mission staff complained to their own counterparts in Washington, who would then circle back to the project team and inform them that there was a problem in their supply chain.

USAID’s April memo raised the same complaint.

“Overall oversight has fallen to USAID’s staff rather than PSM’s. In many cases, if not most, urgent issues with delays and accuracy of orders were identified by HQ USAID staff, or in direct response to complaints from USAID Missions to USAID HQ,” the memo stated.

All systems go?

Data visibility was one of the key factors that distinguished Chemonics in its winning bid for the $9.5 billion contract.

USAID’s contracting officer for the award described the information management system that each firm proposed as, “integral to all aspects of the award, from ensuring quality-assured commodities are procured and delivered in a timely fashion, to ensuring no shortages or stockouts occur,” according to a report by the U.S. Government Accountability Office.

In its evaluation of Chemonics’ and PFSCM’s proposals, USAID rated Chemonics “outstanding” on data visibility and PFSCM as “poor.” In addition, Chemonics’ proposed cost was roughly $50 million less than PFSCM’s, and USAID rated Chemonics better on “logistics.”

These findings led PFSCM to protest USAID’s award decision in 2015 — a protest that first the GAO, and then a federal claims court denied. That protest process took months to resolve, delaying the start of the GHSC-PSM project.

According to the GAO document, Chemonics’ proposed information system — called ARTMIS — represented a significant advantage in USAID’s view over the system that the incumbent group of implementers proposed to use. Months into the project, however, the GHSC-PSM team was still not using the new system that they had proposed, and were relying instead on “manual trackers,” which are Microsoft Excel spreadsheets in which they manually edited out details from one order and inserted details for the next, according to both former employees and USAID documents.

“Data quality within ARTMIS is poor,” USAID’s April memo reported, pointing to incorrectly coded orders and duplicated entries that have resulted in “inaccurate financial reporting to USAID HQ and Mission staff.” The same memo complained that training on the new system has been “less than satisfactory.”

In response to USAID’s concerns, Chemonics agreed to fully and exclusively use ARTMIS for placing orders by June 30, 2017, the USAID official told Devex on background. The same official said that after ensuring an error rate of less than 5 percent, Chemonics was able to retire manual trackers and fully utilize ARTMIS at the end of August 2017, roughly 18 months after the project began.

“We are using ARTMIS and are continuing to refine and improve our data to provide better visibility and reporting on our progress which we will be sharing with our client and all of you periodically as the year progresses,” Butcher wrote in an email to staff on Sept. 12.

A world of delays

The issues the project has encountered span countries, as well as functional areas. A Sept. 7, 2017 report on stockout risks, which Devex obtained, lists six countries where a variety of problems led to current or projected stockouts of various health commodities.

In Côte d’Ivoire, a delay in placing a purchasing order for Co-trimoxazole, an antibiotic, caused that order’s estimated delivery to fall five months after the agreed delivery date, resulting in a stockout of the drug at the central medical store.

In Haiti, another shipment of antibiotics was expected to arrive five months late due to “internal process delays” related to sourcing and subcontracting with a provider.

For Nigeria, the document describes “critically low” stock levels of the pediatric antiretroviral drug Nevirapine, which it blamed on supplier delays. Those stockouts necessitated the delivery of an “emergency supply” of drugs from South Africa and additional shipments to be relocated from Ghana.

In total, the Sept. 7 document listed 23 current or projected stockouts for products related to the portion of the contract that covers HIV/AIDS commodities, which is one of four components — or task orders — included in the contract.

In at least a few countries, the delays and inability to meet agreed delivery dates has created tension between implementing partners and their U.S. government clients.

The April memo from USAID reported, “A number of Missions are complaining about PSM performance on the ground, i.e. Liberia, Madagascar, Lesotho, Vietnam and Nigeria. Issues that have been raised include lack of in-country autonomy, not empowering competent staff, and routine requirements needing PSM HQ approval.”

A few weeks before USAID sent the memo, on March 28, 2017, three Chemonics and project executives traveled to Haiti to present an order management action plan for improving the health supply chain’s performance on the island nation. A biweekly update one week later, which Devex obtained, reported that the “meetings with USAID Mission and CDC were cordial, but both clients made it clear current delivery performance on orders is unacceptable.”

The update continues, “both encouraged GHSC-PSM to fix problems, but stated ramifications of continued poor performance threaten the relationship.”

Read more Devex coverage of U.S. aid.