Types of Review at CIHR

1. Program objectives
2. Anticipated number of applications
3. Length of applications
4. Nature of the assessment required (i.e., weighted criteria, reaching consensus, ranking)

1. Structured Review

For this type of review, reviewers assign scores to a set of predetermined criteria. The final score is generated automatically according to the weighting assigned to each criterion; thus, generating a rank order which is used to allocate funding. The weighting for each criterion can be adjusted based on the program's objectives (e.g., track record, research achievements, etc.).

Peer Review Meeting Format

This type of review is commonly referred to as virtual review since the process is done entirely on ResearchNet and therefore the reviewers do not meet in person to discuss the applications. Examples of funding opportunities that currently use structured review include: Fellowships, Doctoral Research Awards, Master's Awards and Meetings, Planning and Dissemination Grants (MPD).

2. Unstructured Review

Unstructured review means that the peer reviewers "weigh" the evaluation criteria as they see fit to determine a final score. The philosophy of this type of review is that funding applications can be extensively diverse in nature and therefore, merit open discussion and debate. Reviewers assign an initial rating and a discussion by the committee follows. After discussing the strengths and weaknesses of the application, the assigned reviewers reach a consensus score. The committee members then score within +/-0.5 of the consensus score. The mean of the committee members' score is then used to generate a rank order list, which is used to allocate funding.

Peer Review Meeting Format

The peer reviewers meet in person, or via teleconference, to discuss and rank the applications. Examples of funding opportunities that currently use unstructured review include: Open Operating Grant Program, Catalyst Grants and Team Grants.

3. Merit Review

Merit Review is a type of unstructured review that employs separate scores for Potential Impact and Scientific Merit.

In addition to the above mentioned description, Merit Review is employed for certain funding opportunities which require the assessment of both the scientific merit and potential impact of the projects. An individual score is assigned for each of the two components where committee members score within +/-0.5 of the consensus score. The mean of the committee members' scores is then used to generate a rank order list, which is used to make funding decision.

Merit review is used for research programs that engage knowledge-users throughout the research process to inform the research plan, carry out the project, and apply the findings. In general, the potential impact score reflects the importance of the project to the knowledge-users and the likelihood that it will have a substantive and sustainable impact on health outcomes, practice, programs and/or policy in the study context. Therefore, while unstructured review is used to assess the application, there are two separate scoring components.

Peer Review Meeting Format

Merit Review is carried out using an unstructured review meeting format. Examples of funding opportunities that currently use merit review include: Partnerships for Health System Improvement (PHSI), Knowledge Synthesis Grant and Operating Grant: HIV/AIDS (Community-Based Research).

4. Binning

The binning process encourages members to differentiate between highly promising applicants and to ensure that the full range of the rating scale is used. Examples of funding opportunities that currently use the binning process include: Banting Postdoctoral Fellowships, Vanier Graduate ScholarshipsS & New Investigator Salary Award.

Stage 1 – Structured review

• Reviewers review the applications assigned to them and score each one according to predetermined criteria;
• Reviewers assign these applications within three bins:
  • recommended for funding;
  • could be recommended for funding; and,
  • not recommended for funding.

Stage 2 – Unstructured review

• Face-to-face peer review meeting to discuss a selected group of applications;
• An average of the Stage 1 scores for each application is calculated and the applications are ranked
  • The ranked list is used to identify the top-ranked applications that may automatically be approved for funding;
  • The applications ranked just below the top-ranked are discussed at the peer review meeting;
  • The applications ranked the lowest are not discussed except at a committee member's request or if there is a significant discrepancy between the reviewers pre-meeting scores that would enable the application to be funded.

5. Other specialized review process - Relevance Review

The Relevance Review Process is used by strategic leads and/or partners to assess the alignment of an application with a specific research theme described in the funding opportunity (FO). As the name implies, the process is used when it is important for applications to be relevant to (or in alignment with) targeted research components of the FO. This review approach is generally reserved for strategic FOs and Priority Announcements (PAs).

The purpose of this review is to:

• assess the alignment of an application with the FO;
• ensure that only relevant applications are peer reviewed; and,
• ensure that CIHR and its partners only fund applications that are relevant to the FO.

The Relevance Review Process typically takes place after eligibility screening is complete and prior to peer review.

Date modified:

2011-12-02