‘Flawed’ findings on closures fueled province’s decision to end warrants

The New Brunswick Department of Health’s March 2022 decision to end mask mandates and other COVID-19 measures was made after officials circulated a widely criticized academic paper as “fundamentally flawed”.

The paper, written by three researchers linked to libertarian think tanks, concluded that lockdowns did not significantly reduce the death rate during the first wave of COVID-19 in early 2020.

But the study, a so-called “meta-analysis” that synthesizes dozens of individual studies, is fraught with problems, other researchers say.

“The study has methodological issues that make the conclusion simply unreliable, so there is a high risk of potential bias,” says Adrian Lison, PhD student and infectious disease researcher at ETH Zurich.

“And since the study has not yet been peer-reviewed, we believe that it does not really add new reliable information to the body of knowledge, so we recommend against using the strong enough conclusions of this consideration in politics.”

The document was circulating among New Brunswick Department of Health officials in early February. It has not been published in an academic journal.

Since the study has not yet been peer-reviewed, infectious disease researcher Adrian Lison recommends that it not be used to revise the policy. (Submitted by Adrian Lison)

Lockdowns “have had little to no public health effects”, but huge economic costs, he says. They are “ill-founded and must be rejected as an instrument of pandemic policy”.

Health Department spokesman Adam Bowie said officials look at data “from many different studies” as well as data from epidemiologists across the province when making decisions.

“A Public Health recommendation to introduce or remove a health mandate would never be based on any particular study or report,” he said.

The meta-analysis “would only have been one source of information for the study,” he said, although he did not identify any other studies used.

Kathleen Gadd of the citizens’ group Protect our Province, which obtained the study through an access to information request, says it is the only so-called academic research the province has delivery.

“The fact that they didn’t provide any other papers says a lot,” she said.

“If other studies and peer-reviewed documents and evidence had been used to inform the lifting of the mandatory order, now would have been the time to provide that information to the public.”

Using the general term ‘lockdown’ is ‘dangerous’, says researcher

Among the flaws identified by critics is the newspaper’s use of the blanket term “lockdown” for a range of policies from stay-at-home orders to mask mandates.

This makes it difficult to measure the impact of individual restrictions, says Lison, co-author of a rebuttal posted on the Social Science Research Network website.

He says it’s “dangerous in a systematic review to use such a broad term, because if you draw conclusions at the end, there’s a high risk that people will misinterpret your results.”

The paper also measures death rates without looking at other impacts, such as reduced transmission and a reduced impact of hospitalizations on the healthcare system.

Lison says transmission is a better measure of the impacts of the lockdown because it is most directly affected by the restrictions, while hospitalization and death come after a delay.

A health official called the paper’s findings “interesting”

The province lifted its mandatory COVID-19 order on March 14, ending mask mandates, vaccination proof requirements and limits on gatherings.

The 60-page paper, “A literature review and meta-analysis of the effects of lockdowns on Covid-19 mortality,” was the only research study included in the 200 pages given to the PoP group when it requested documents relating to the decision.

In an email dated February 4, 2022, Nina van der Pluijm, director of welfare, legislation and standards for the department, sent the study to several colleagues, remarking: “Interesting information on the effectiveness of the confinement”.

She noted her finding that the medium lockdown only reduced mortality by 0.2%.

Gadd, a former health sciences librarian, says “a number” of people with expertise in universities and health authorities could have provided officials with better studies.

“There is no evidence that information professionals, such as health sciences librarians, were involved in these fairly significant decisions that were made regarding the health of New Brunswickers,” she said.

“There is a vast amount of high-quality, peer-reviewed evidence that has been completely ignored by those in power regarding how to handle the pandemic.”

Article criticized as “unbalanced” for excluding other studies

The meta-analysis, published in January, synthesizes the results of 34 separate studies. The authors started with hundreds of studies, but eliminated most of them.

“Unfortunately, this review only focused on a very small subset of the analyses… and excluded a lot of the body evidence, the whole epidemiological research, so it’s also rather unbalanced. “Lison said.

Dr Seth Flaxman, professor of computer science at Oxford University, called the meta-analysis “fundamentally flawed” for excluding studies rooted in epidemiology.

Researchers at Imperial College London said in June 2020 that lockdowns in the first wave of the pandemic likely averted 3.1 million deaths in 11 European countries.

New Brunswick lifted its mandatory COVID-19 order on March 14, ending mask mandates, vaccination proof requirements and limits on gatherings. (Evan Mitsui/CBC)

Another study estimated that lockdowns in six countries, including the United States and China, prevented or delayed 530 million cases of COVID-19.

“It’s just problematic to exclude the majority of available research, and it risks making your review very unbalanced and unrepresentative,” Lison said.

“There is of course still some uncertainty and a lot of open questions, but there is already substantial evidence for the overall effectiveness of interventions.”

Authors linked to libertarian think tanks

The authors are neither epidemiologists nor public health experts, and PoP highlighted their ties to libertarian think tanks opposed to most government intervention in society.

“The authors do not represent areas of study that apply to epidemiology, public health, disease transmission, those types of areas that we should seek for information about our response to the pandemic” , said Gadd.

Co-author Jonas Herby, an economist at the Center for Policy Studies in Copenhagen, Denmark, is a contributor to the American Institute for Economic Research, which bills itself as promoting “pure freedom and private governance,” with a government “highly confined”.

Another co-author, Steve Hanke, is a fellow at the right-wing Cato Institute who has called lockdowns and mandates “cracking the fascist whip”.

Jonas Herby, co-author of the paper, says most of the “flattening of the curve” in COVID deaths early in the pandemic was due to voluntary distancing, not lockdown requirements. (Radio Canada)

In an interview, Herby acknowledged the authors’ conservative leanings and said it shaped the question they wanted to ask in the article. But their research method was sound, he added.

He said the meta-analysis focused on deaths because there were not many studies available on hospitalizations.

On criticism that the authors used the general term “lockdown” for a range of different restrictions, Herby said that’s why the 0.2% mortality reduction is based on a “medium” lockdown.

“I think it’s bad science to say your science is bad just because you have a different political view than mine.”

Revised version shows greater reduction in deaths

The three authors of the article published a revised version in May.

It recalculates the reduction in deaths caused by lockdowns at 3.2% – still lower than other studies, but 16 times higher than the 0.2% estimate in the first version.

Mask mandates had the biggest effect, reducing COVID-19 mortality by 18.7%, he says.

The authors say what matters are not particular numbers, but the fact that the number of lives saved by the closures “is much smaller and far removed” from what epidemiologists, politicians and the media have promised. .

Most of the “flattening of the curve” in COVID deaths early in the pandemic was due to voluntary distancing, not lockdown requirements, Herby says.

People changed their behavior voluntarily in response to a threat, he says.

“The question is, can lockdowns improve this behavior change? I think they can, but the effect is very limited.

The new version includes a statement that it “does not imply that locks don’t work.

“It simply indicates that the more lenient lockdowns had virtually the same effect on mortality as the stricter lockdowns. Since no country did anything, we cannot reject the thesis that some NPIs are necessary, for example, to stimulate voluntary behavioral changes.

Lison said he found the authors’ defense of the meta-analysis “largely unconvincing” and said the revised version did not change his view that the paper should not be used by governments as political advice.

“Most of the problems from the first version remain,” he said.

Joan J. Holland