CIHR implements recommendations from the research community for the CIHR Project Grant competition

Applicants may submit a maximum of two applications in the role of Nominated Principal Applicant (NPA) per Project Grant competition.

Implemented.

The revised eligibility is detailed in the Project Grant funding opportunity.

The existing page limits for applications will be expanded to 10 pages (including figures and tables) and applicants will be able to attach additional unlimited supporting material, such as references and letters of support.

• Project Grant applications will be 10 pages long (including figures and tables).
• Those 10 pages will be “free flow” (unstructured), allowing applicants to decide how to address the review criteria.
• Applicants can attach unlimited references and letters of support.

Implemented.

The new page limit is detailed in the Project Grant application instructions.

Applications will be assessed based on “significance and impact of the research” (25% of final score), “approaches and methods” (50% of the final score), and “expertise, experience and resources” (25% of final score).

Implemented.

The revised adjudication criteria are outlined in the Project Grant application instructions.

The Common CV (CCV) for the Project Grant application will include publications from the past seven years and applicants will be able to upload a PDF to supplement the CCV information if they have taken leaves of absence in the past seven years.

Implemented.

The details on additional CV information allowed are part of the Project Grant application instructions.

A one-page rebuttal will also be included in the revised structure to give the applicant the opportunity to explain how the application was improved since the previous submission.

Implemented.

The one-page response to reviewers is detailed in the Project Grant application instructions.

Chairs will now be paired with Scientific Officers to collaboratively manage a cluster of applications and assist CIHR with ensuring that high quality reviewers are assigned to all applications.

Implemented.

The revised role for competition Chairs and Scientific Officers is detailed in the Project Grant Peer Review Manual.

Reviewer assignments will be approved by the Chairs and Scientific Officers. In addition, Chairs and Scientific Officers will have the ability to remove or add reviewers after the reviewers have completed the new Conflict of Interest/ Ability to Review (CAR) assessment for a group of applications.

Implemented.

The revised role for competition Chairs and Scientific Officers is detailed in the Project Grant Peer Review Manual.

Applicants can make recommendations regarding what types of expertise are required to review their application.

Implemented.

The fields have been added in the application and are detailed in the Project Grant application instructions.

Each application will be assigned to four (4) reviewers at Stage 1

Implemented.

The revised reviewer assignment details are part of the Project Grant Peer Review Manual.

Applicants can now be reviewers at Stage 1 of the competition. However, they cannot participate in the cluster of applications containing their own application.

Implemented.

Applicants to the competition are not excluded, as per the peer reviewer criteria for the Project Grant competition.

Peer review members participate in accordance with the Conflict of Interest and Confidentiality Policy of the Federal Research Funding Organizations.

Asynchronous online discussion will be eliminated from the Stage 1 process.

Implemented.

Asynchronous online discussion has been removed from the competition process.

CIHR will revert to a numeric scoring system (rather than the current alpha scoring system) to aid in ranking of applications for the Project Grant competition.

Implemented.

The new numeric scoring is detailed in the Project Grant Peer Review Manual.

Approximately 40% of applications reviewed at Stage 1 will move on to Stage 2 for a face-to-face review in Ottawa.

Implemented.

The approximate percentage of applications moving to Stage 2 is included in the Funding Opportunity and the Project Grant Peer Review Manual.

The working group recommended that Stage 1 reviewers’ comments for applications that don’t move on to Stage 2 be reviewed by chairs and SOs to ensure appropriate review.

Implemented.

The role of competition Chairs and Scientific Officers is detailed in the Project Grant Peer Review Manual.

Stage 2 will include highly ranked applications and those with large scoring discrepancies.

• The Working Group’s recommendation is to have a process where the top-ranking ~30 per cent of applications across clusters plus those in the top ~30 per cent within their cluster will go forward to Stage 2. This approach is recommended until data collected during the next Project Grant can inform this ranking strategy and address issues related to applicants who are also reviewers. The remaining portion will include applications such as those that have large scoring discrepancies, were specifically flagged by Competition Chairs, or were highly ranked within their clusters. The final breakdown of the 40 per cent will depend on the actual applications submitted during the competition.

Implemented.

The overview of how applications are selected to move to Stage 2 is included in the Project Grant Peer Review Manual.

As per the recommendation, CIHR will monitor the review process to help inform refinements to ranking strategies moving forward.

Chairs will work with CIHR to regroup and build dynamic panels, based on the content of applications advancing to Stage 2.

Implemented

As per the Project Grant Peer Review Manual, Competition Chairs and Scientific Officers will be assigned to a cluster of applications in their general area of expertise. Competition clusters will be determined based on the number and types of applications submitted to the competition and may therefore vary from competition to competition.

The clusters are expected to remain largely the same from Stage 1 to Stage 2.

Applications moving to Stage 2 will be reviewed by two of the original four reviewers from Stage 1 during face-to-face meetings within cluster-based panels in Ottawa. They will be expected to present their own and the reviews of other two Stage 1 reviewers at the meeting.

Implemented

Applications moving to Stage 2 will be reviewed by two of the original four reviewers from Stage 1. This is detailed in the Project Grant Peer Review Manual.

In order to increase accountability, reviewer names will accompany their reviews to the final assessment stage.

Implemented.

Detailed in the Project Grant Peer Review Manual.

The Working Group has recommended ranking within each cluster for Stage 2, as opposed to a ranking across face-to-face committees.

Implemented

The ranking of applications (and funding decisions) will be made within each cluster, as per the Project Grant Peer Review Manual.

CIHR grantees will be strongly encouraged to review if invited. The Working Group feels that it is important for grant recipients to give back to the process, if invited. The College Chairs will continue this discussion.

Implemented.

The recruitment of reviewers who meet the Project Grant reviewer criteria has begun. CIHR grantees are strongly encouraged to review if invited.

Host a face-to-face meeting with all of the Chairs and Scientific Officers in the fall. The Working Group agreed with the proposed selection criteria to choose Chairs, and also felt that it was appropriate for the Chairs to help choose the Scientific Officers, but recommended that CIHR bring everyone together. CIHR is evaluating the feasibility of hosting this meeting for the 2016 competition.

Implemented.

CIHR will host a face-to-face meeting with all the Chairs of the peer review process in November 2016.  

Competition Chairs will help recruit the Scientific Officers based on expertise required as per the Project Grant Peer Review Manual.

Make peer review training mandatory for all reviewers. Completing the training – including a module on unconscious bias – should be a requirement for all reviewers, including seasoned senior investigators, Chairs and Scientific Officers.

All peer reviewers will be invited to complete training and participate in information sessions on the Project Grant review process, including a specific module on unconscious bias.

Invite early career investigators (ECIs) to observe the Stage 2 face-to-face peer review meetings. The Chairs in the College of Reviewers have made mentoring ECIs a priority. They will work with CIHR to find opportunities for mentorship—including observing peer review—and will discuss using the Project Grant competition as one of those opportunities, whether it takes place as part of the fall competition or future ones.

New

Implemented.

CIHR launched a pilot observer program for early career investigators (ECIs) to participate in the peer review process of the Project Grant: Fall 2016 competition.

A list of reviewer criteria was defined for the Project Grant competition. The criteria apply to all career stages, and recruitment will include early career investigators to ensure they have the opportunity to acquire the required experience.

CIHR, in consultation with the College of Reviewers Chairs, is developing additional mentorship strategies. Details about these strategies will be shared at a later date.

Consider a variety of mechanisms related to ensuring equity across different career stages and sex of applicants. The Peer Review Working Group strongly recommended that CIHR continue this conversation with its Science Council, as well. There was agreement that this important topic requires more in-depth consultation and analysis across all CIHR funding programs.

Discussions still under way.

There was unanimous support from the Working Group for equalizing success rates for early career investigators (ECIs) in the Project Grant program. Equalizing success rates means ensuring that the success rate for ECIs in a competition matches the overall competition success rate. The Working Group recommended that the additional 30 million dollars received in the last budget be used for this purpose, similar to how these additional funds were used in the first Project Grant competition.

Discussions still under way.

As noted in April 2016, however, the additional 30 million dollars will be entirely dedicated to the ongoing and future Project Grant competitions with a focus on early career investigators.

Share the appropriate level of data from the Fall 2016 Project Grant competition after the results are released. The Working Group advised CIHR to share this competition data publicly in the spirit of transparency, but also to ensure that the data can be used by CIHR to shape important decisions in the future.

CIHR will share data from the Project Grant competitions, in accordance with privacy laws. (Note: The data from each competition will be shared after the results are released.)

CIHR will evaluate the changes implemented and make analyses available as they are conducted (e.g., data about the reviewer profile matching algorithm and its validation). It will be instrumental for future refinement, improvement, and transparency of Peer Review for CIHR to make this data publicly available, at the finest resolution possible, while still protecting privacy and confidentiality.

CIHR and its Science Council will carefully monitor and evaluate the competition and work with the necessary advisory bodies to make the necessary changes. CIHR will share those progress updates.

Adjust the CCV requirements for non-academic co-applicants as well as academic co-applicants who are not appointed at a Canadian institution. The Working Group proposed that such applicants may upload other documents, rather than requiring a CCV.

A plan will be put together to address this issue for future competitions.

Incorporate a mechanism between Stage 1 and Stage 2 to let applicants respond to reviewer comments. For those applications moving ahead to Stage 2, the Working Group proposed giving applicants the opportunity to submit a 1-page response to reviewer comments that could then be used as part of the Stage 2 deliberations.

It will be further explored for implementation for future competitions.