Covid Findings — with some Controversy

When the World Health Organization (WHO) published an estimate of excess deaths worldwide during the COVID pandemic, Jon Wakefield expected some media interest. He did not anticipate a deluge of news stories and interview requests about the findings.

Wakefield, UW professor of statistics and biostatistics, led the team that calculated the global death toll. They estimated that by the end of 2021, there had been 14.8 million excess deaths due to the pandemic — more than double the official estimate of 5.4 million COVID deaths reported by countries individually.

So yes, the media took notice.

“I was inundated with requests,” recalls Wakefield. “Multiple BBC interviews, PBS News Hour, radio stations in New Zealand and Ireland….and a live interview on Australian TV news. I’ve published a lot of papers in my career, and some of them have had influence on statistical practice, but this was totally new.”

As a statistician, Wakefield has always focused on health issues, from the impact of pollution on chronic disease to child mortality in low- and middle-income countries. So when the WHO sought statisticians for an advisory group to study aspects of the pandemic, he volunteered.

Leading a small international team of researchers — William Msemburi (WHO), Ariel Karlinsky (Hebrew University), and biostatistics doctoral students Victoria Knutson and Serge Aleshin-Guendel (University of Washington) — Wakefield looked at excess mortality during the COVID pandemic. In February 2021, the group began meeting regularly. “There were many hideous 5 am calls,” Wakefield recalls with a laugh. “It was brutal, but also cool because people from all over the world were involved.”

Studying excess mortality meant looking at all deaths — not just COVID deaths — to estimate the difference between how many people died during the COVID pandemic and how many likely would have died in the absence of the pandemic, country by country and month by month. Many excess deaths were COVID deaths, but reduced access to medical care, avoidance of seeking health care, and other pandemic challenges also figured in. (Some countries had fewer deaths during COVID since, for example, staying home meant fewer car accidents.)

To calculate excess mortality, the team first had to develop a model for baseline mortality in non-pandemic circumstances. Then they gathered mortality data during the COVID pandemic. The difference between these figures is the excess mortality estimate.

Sound straightforward? Imagine working with data from 194 countries, each using different methods and procedures for data collection. Imagine creating a mortality baseline for countries where data about deaths is meager, unreliable, or non-existent. Imagine trying to turn that patchwork of data into a credible statistical model, to be judged by the world.

Many countries, particularly in Europe and the Americas, had relatively reliable data. But elsewhere there were massive gaps, which Wakefield’s team addressed using statistical prediction methods based on time series and regression models. Country-specific variables were used within the model to predict the pandemic mortality in countries with sparse or no data.

Despite the data challenges, Wakefield’s team created a model they feel provides the most accurate estimates yet for excess mortality during the COVID pandemic. With those estimates, countries can better understand the effectiveness of their vaccine campaigns and COVID policies, see the impact of various variants, and explore other health questions.

“In my professional life, I’ve tried to develop models that can be useful,” Wakefield says. “This project has been an opportunity to do something really important.”

Important, but also controversial. For some countries, the findings confirmed the accuracy of previous estimates. For others, the new information suggested their reported COVID death estimates were way off. That perspective was not always welcome.

The WHO shared a preliminary draft of the methods and estimates with country representatives for feedback in a process known as country consultation. Many had few or no changes. Even Russia, with excess morbidity numbers much higher than previously reported, did not protest the findings. But India vehemently rejected the WHO’s estimate of excess deaths in India. (The Indian government had estimated about 481,000 COVID deaths; the WHO team estimated the excess mortality as 4.7 million.) Wakefield’s team responded to India’s complaints, but the Indian government continued to reject the estimates, effectively delaying the report’s official rollout for more than four months.

Wakefield is reticent about discussing the disagreement publicly (though the The New York Times covered the controversy), but he stands behind his work. In fact, he was prepared to release the estimates himself when the WHO repeatedly delayed the release of his group’s report due to politics. “My feeling was that it’s all public data,” Wakefield says. “Politics shouldn’t stand in the way of science.”

In the end, there was no need to go rogue. The WHO released the report on May 5, 2022. That was an important accomplishment, and the resulting interviews with international media were exciting. But for Wakefield, the project is far from complete. His team intends to continue refining the model as new data arrive, incorporating constructive feedback on the initial estimates from researchers all over the world.

“The model and the data are evolving over time, and we will come up with a new set of estimates later in the year,” he says. “It troubles some people that we may get different answers, but we are producing estimates and there is massive uncertainty in the excess for many places, particularly in Africa where mortality data are sparse. You just have to be aware of that when interpreting the estimates.”

And if there’s more controversy, Wakefield is ready to take that on.

“All modesty aside, I think I’m better at doing this than most people in the world, because I’ve got a lot of experience,” he says. “I’ve spent 40 years doing this. This is what I trained to do.”

Read the WHO report on global excess deaths associate with COVID.