Canadian health researchers unhappy with new grant reviewing system

[Health Minister Jane Philpott responded in July to an open letter from health researchers expressing their frustration with the Canadian Institutes of Health Research’s new peer review system. THE CANADIAN PRESS/Adrian Wyld]

With a deadline for funding applications just weeks away, many in the country’s research community say it’s essential that the Canadian Institutes of Health Research (CIHR) quickly fix a peer-review system that is harming the national scientific community.

And while the federal health-research funding agency’s decision to move away from a problematic automated review process will mitigate some of its consequences, damage has already been done, a Toronto-based research director tells Yahoo Canada News.

“I think the changes will restore some level of confidence in the applicants, which is essential,” says Dr. Jim Woodgett, director of research for the Lunenfeld-Tanenbaum Research Institute in Toronto. “However, it’s still going to be pretty bad news. The number of grants that are going to be funded will be at an all-time low for this year, and perhaps going forward.”

The primary source of complaints from scientists about CIHR’s changes to its funding review model was the end of face-to-face meetings of research peers to discuss grant proposals together. That was replaced by an online process that relied on keywords within the proposals to assign them to reviewers.

That change has resulted in a record high number of grants being rejected, under-representation of female and young researchers among grant approvals, grants being assigned to inappropriate reviewers and a bias against basic science proposals, health research publication STAT reported on Aug. 1.

For example, 87 per cent of applicants in the most recent funding cycle for CIHR were rejected, STAT reported, and a backlog of proposals built up this spring when two other funding cycles were cancelled while the new system was implemented.

Because the automated system required five reviewers per grants instead of the usual two or three, CIHR had difficulty finding enough high-quality reviewers — a problem that was exacerbated when the changes shortened the time available for funding review, Woodgett says. And there were cases when the reviewers were poorly matched with grant proposals that were outside their area of expertise, or didn’t review the funding proposals adequately, he says.

“The problem was that it took away the accountability of the reviewers because they weren’t having to defend their reviews, and some of them were quite simply abysmal,” Woodgett says of the move to virtual reviews versus the “gold standard” of face-to-face meetings to discuss proposals.

Frustrated researchers

The outcry against CIHR’s changes to peer review for grant applications for the billion dollars in funding the agency distributes each year has come from researchers across the country. An open letter written by Woodgett to Health Minister Jane Philpott and posted on Medium on June 27 was signed by more than 1,300 Canadian researchers.

“The design of the new peer review system shows deep flaws and erroneous assumptions,” the letter read. “These problems were entirely foreseen by the broader research community, which repeatedly presented these concerns to CIHR leadership over the past 3 years.”

Philpott responded to the letter last month in a statement that read in part “I have noted with growing concern the views that have been expressed within the health research community about changes being implemented at the Canadian Institutes of Health Research, including the new online peer review process.”

Some in Canada’s health research community also went online to express their frustration using the hashtag #Pscream, a play on Project Scheme, the name of CIHR’s funding program.

"Many of us saw a real and explicitly stated bias against basic research in last #pscream round. Mechanisms needed to prevent this,” wrote Paul Gribble, a researcher with Western University in London, Ont.

“Only 1 Cognitive Neuroscience grant funded in Canada. Can a scientific field apply for protection as an "endangered species”? #pscream,” wrote Toronto neuroscientist Brad Buchsbaum.

“A lot of work being done on fixing #CIHR Foundation and #pscream issues, but all seems like 1st order change. Need 2nd order change!,” wrote University of Toronto healthcare professor Aviv Schachak.

Undoing changes

In mid-July CIHR agreed to undo reforms to its peer review system after a daylong emergency meeting with the funding agency’s president and concerned parties from across the country. At that meeting CIHR’s governing council agreed to a hybrid system where reviewers will first vet and eliminate weaker funding applicants before conducting face-to-face reviews for the remainder.

The unusual meeting was held at the request of Philpott, who didn’t attend but issued a statement saying that while she was pleased with the results, this was “only a first step towards restoring a productive and respectful relationship between CIHR and Canada’s health researchers.”

The meeting was positive, Woodgett says, and the health ministry seems serious about fixing the issues with the changes to the system. But the problems will have a lasting effect that will reduce the number of successful grants in the short term and create even more competition for the already limited funds available for health research in Canada.

For the fiscal year ending March 31, just over half a billion dollars in funding was split between two funding competitions, Woodgett says. But for the next fiscal year that same amount of money will be split three ways.

A good success rate for grant approvals for a funding program is 20 to 25 per cent, Woodgett says. For CIHR’s most recent program the success rate was only 13 per cent, the lowest in the agency’s history. Because of increased competition, the upcoming funding programs could have success rates as low as the single digits, he says.

“At that point the system collapses,” Woodgett says.

While he is glad to see work now being done to reverse the CIHR changes, Woodgett says it’s hard to tell at this point what the long-term effects for Canadian science will be. Global circumstances like Brexit, fiscal issues for EU states and uncertainty about the upcoming U.S. election make it undesirable for established researchers to relocate elsewhere, he says.

The more likely consequence will be the loss of experienced lab technicians, who are often among the first to be cut when a lab loses funding, Woodgett says. And fewer Canadians overall may choose to do graduate and post-graduate studies in health research because of funding uncertainty, he says, which will impact the next generation of health researchers in the country.

“I do blame CIHR. This federal agency has been incompetent and really did not plan this in a realistic manner. They brought in changes that no other funding agency around the world has done,” Woodgett says. “I find it remarkable that a federal agency has managed to do so much damage in such a short period of time.”