Second update from Dr. Paul Kubes, Chair of the CIHR Peer Review Working Group

August 16, 2016

First, I would like to thank the CIHR community for their patience and suggestions. I appreciate the variety of perspectives that have been presented, and I assure you that your feedback has been a key element of the Working Group’s deliberations. I would also like to thank the members of the Working Group, including those from CIHR, for changing their summer plans to devote time to this important work. We all want to get this right, but we do realize that time is of the essence, especially since we can now confirm the dates for Fall 2016 Project Grant competition and the Foundation Grant competition.

The Peer Review Working Group held a productive face-to-face meeting in Ottawa on August 5. As I noted in my previous update, the members of our group do not represent any specific research area or institution. After the July 13 meeting, they volunteered to represent the CIHR community as a whole. This is a very difficult time for the research community, as low success rates are being felt across all pillars. While the Working Group recognizes that our work won’t change funding levels or success rates—they will remain no matter how we slice the current pie—we all want a system where each application is matched with the right expertise, and reviews provided are well-informed, constructive and fair.

Although our discussions have been very fruitful, we still have a number of details to pin down before I can share the full picture of how the different stages of review will work for the Project Grant fall competition. That said, we have collected our recommendations to date in a separate table for easy reference. I should note that the committee arrived at consensus for optimal peer review for almost all recommendations. I invite you to review the table, with the caveat that it is still a work in progress.

The table provides additional information, but there are a few points that I would like to highlight:

  • The Common CV (CCV) for the Project Grant application will include publications from the past seven years AND a separate PDF for applicants who have taken leave (parental, medical, etc.) during this time. In my last update, I noted the Working Group’s recommendation to increase the number of years for the list of publications for the CCV. We received feedback from the community that, although the increase from five years to seven years was a welcome change, this increase didn’t go far enough to ensure that researchers who have taken leaves of absence are not penalized. As the CCV cannot accommodate the addition of information about leaves in its current form, applicants will be able to upload a PDF to supplement the CCV information. Whatever time an applicant has taken off in the past seven years is the amount of time that they may append. So, for example, if an applicant took one year of parental leave, then that applicant would be allowed to upload a PDF that outlines one additional year of publication history.
  • The Working Group endorsed the proposed criteria to select Chairs, Scientific Officers, and to screen reviewers. Chairs must have significant experience on CIHR committees, as well as Tri-Council funding within the last five years. As well, a list of Chair candidates will be validated by the College Chairs. The Working Group felt that it was appropriate for the Chairs to have a role in choosing Scientific Officers, as well. For this round, reviewers must have held at least one federally-funded, peer reviewed grant in the role of Principal Investigator. The working group recognized this may exclude some early career investigators as reviewers, however, they recommended the concept of mentoring through observer status at the face-to-face peer review meetings. Lists of potential reviewers will be vetted through the appropriate Chairs and Scientific Officers.
  • We have agreed that 40% of the applications will move from Stage 1 to Stage 2, but we are still working out the peer review process details. It is important to note that this will be done based on the review results of experienced reviewers with the right expertise. We acknowledge that the idea of not all applications being discussed face-to-face makes the community uncomfortable, but in discussing the sheer volume of anticipated applications, we realized that bringing 100% of the applications to face-to-face committees would not be possible. Even if 100% of those applications did go to such committees, 50-60% would get triaged, anyway, with no discussion and no Scientific Officer notes. As such, we agreed with the decision from the July 13 Working Meeting that 40% of the grants will be discussed face-to-face in Ottawa, including highly ranked applications and those with large scoring discrepancies. It is important that all applications deemed as most promising have the opportunity for fine-detailed discussion to ensure the best grants are funded. The review of the remaining 60% will be carefully examined by the Chairs and Scientific Officers. The specific details regarding how to determine which applications are included in the top 40% are still under discussion.

We will continue discussions over the coming weeks. Personally, I feel that the Working Group is well on its way to putting forward a peer review system that makes significant improvements compared to the first Project Grant competition, while continuing to embrace positive change for peer review.

One example of a change recommended by the working group is to incorporate a mechanism between Stage 1 and Stage 2 to let applicants respond to reviewer comments. For those applications moving ahead to Stage 2, the Working Group proposed giving applicants the opportunity to submit a 1-page response to reviewer comments that could then be used as part of the Stage 2 deliberations. Although it is not feasible to build this mechanism into the fall competition, it will be explored for the spring 2017 competition.

If you have major concerns or constructive suggestions that you would like to share regarding the recommendations outlined in the table, please feel free to e-mail me. I will try to answer every one. I brought every e-mail that I received from the community to the meeting on August 5 and summarized the contents that were relevant to the discussion for the group. Again, I thank you for your input, as we have all appreciated receiving it.

Dr. Paul Kubes
Chair of the CIHR Peer Review Working Group

Date modified: