Project Grant Competition
Part 3: Adjudication Criteria and Rating Scale
Title
Welcome to this learning module in the Project Grant competition series: Part 3: Adjudication Criteria and Rating Scale. In this module, reviewers will learn how to interpret and apply the adjudication criteria to ensure they feel prepared to effectively participate in the peer review process.
Playbar Buttons
This course is designed to be self paced.
Use the playbar below to resume playback, navigate between slides, mute and unmute audio, and toggle closed captions. You can also browse the full table of contents, and collapse or move the playbar.
Adjudication Criteria
In this section, reviewers will learn how to interpret the adjudication criteria for the Project Grant competition.
Adjudication Criteria 2
In the Project Grant competition, reviewers are to consider the context of the applicant, such as leave history, career stage, areas of research, experiential knowledge, diverse career paths, family responsibilities, and pandemic impact. Reviewers should also consider how this context may have affected their progress and productivity when assessing each criterion. With consideration of the applicant's context, the evidence should be notable compared to peers in similar fields and career stages.
Additional areas of consideration that should be explored in applications include:
- The review of applications submitted to the Indigenous Health Research (IHR) committee.
- Evaluation of all applications containing a Randomized Controlled Trial (RCT) as a major component will need to consider the specific RCT evaluation criteria in their assessment, irrespective of the assigned peer review committee.
- Evaluation of applications reviewed in the Commercialization committee will need to include the assessment of both the Research and Technical plan, and the Commercialization plan, according to specific review criteria.
- The review of applications in the Tri-Agency Interdisciplinary Peer Review (TIR) committee.
Adjudication Criteria 3
The adjudication criteria focus on assessing a project’s concept and feasibility, which is founded on a sound and important idea supported by a feasible plan of execution.
To interpret the adjudication criteria, it is important to keep in mind that the research proposal may exert only a basic and/or mechanistic impact, which is as important as the translational impact. The impact does not only mean near-future clinical relevance. Reviewers should evaluate whether the work proposed will significantly advance the proposed area of research.
The Concept will be assessed using the sub-criterion: Significance and Impact of the Research, which looks at what is being proposed and why the project is worth doing. The Feasibility will be assessed using two sub-criteria: Approaches and Methods, which looks at HOW and WHEN the project is expected to be completed. And Expertise, Experience, and Resources, which looks at WHO is involved and WHERE.
Significance and Impact of the Research
The sub-criterion of Significance and Impact of the Research is intended to assess the quality of what is being proposed.
Consider the following:
- Is the project idea creative?
- Is the rationale of the project idea sound?
- Are the overall goals and objectives of the project well-defined?
- Are the anticipated project contributions likely to advance basic and applied health-related knowledge, or health care, or health systems or health outcomes?
This sub-criterion is not intended to assess feasibility of the project, expertise of the team or the potential of success. These will be assessed under Criterion 2: Feasibility.
The proposed research should focus on addressing an issue in any area across the spectrum of health.
Approaches and Methods
The sub-criterion of Approaches and Methods is intended to assess the quality of the project's design and plan, including how, and when, the project will be completed.
Consider the following:
- Are the approaches and methods appropriate to deliver the proposed outputs, and achieve the proposed contributions, to advancing health-related knowledge, health care, health systems, and/or health outcomes?
- Are the timelines and related deliverables of the project realistic?
- Does the proposal identify potential challenges, and appropriate mitigation strategies?
Applicants should apply appropriate considerations regarding Sex and Gender, Indigenous Health Research, and knowledge translation approaches and focuses.
Experience, Expertise, and Resources
The Expertise, Experience and Resources sub-criterion is intended to assess the appropriateness of the complement of expertise, experience and resources among the applicants, and their institutions/organizations, as it relates to the ability to collectively deliver on the objectives of the project.
Consider the following:
- Does the applicant bring the appropriate expertise and experience, to lead and deliver the proposed outputs, and to achieve the proposed contributions?
- Is there an appropriate level of engagement and/or commitment from the applicants?
- Is the environment (academic institution and/or other organization) appropriate to enable the conduct and success of the project?
- Does the applicant adequately demonstrate productivity, progress, and impact of their research program?
Reviewers are to assess productivity and progress broadly, by taking into consideration a range of contributions and impacts, as well as the context of the applicant, and how it may have affected their progress.
Metrics such as number of publications and citations, and size/number of research grants should not be used in isolation to assess productivity and progress. Reviewers should not use journal-based metrics (such as h-index) as surrogate measures of quality and impact of individual research publications. As stated in DORA, the "scientific content of a paper is much more important than publication metrics or the identity of the journal in which it was published".
Review Quality
Reviewers provide written reviews for each of their assigned applications, and they are read by applicants, their fellow reviewers, Chairs, Scientific Officers, and other review users. Therefore, when applying these adjudication criteria, reviewers must also ensure their reviews are of high quality.
Reviews of high quality must meet each of these criteria:
- Appropriateness: The review comments are fair, understandable, original, confidential and respectful.
- Robustness: The review is thorough, complete and credible. This means it contains a detailed justification of the rating, including meaningful and clearly expressed descriptions of both the application strengths and weaknesses. The review must also address all applicable adjudication criteria.
- And utility: the review comments are constructive and may help applicants to improve their future submissions and/or advance their research.
For more information, consult the Review Quality checklist.
Rating Scale
In this section, reviewers will learn about the rating scale for the Project Grant competition.
Rating Scale 2
To ensure consistency, reviewers must adhere to the following rating scale. The following ranges indicate an application which may be funded.
Applications with a range of 4.5 to 4.9 are defined as innovative, filling an important critical gap in knowledge. They have very few flaws, and the investigators have a comprehensive complement of expertise, experience and resources to perform the research.
Applications with a range of 4.0 to 4.4 are defined as very interesting and making important advances. The investigators have an appropriate complement of expertise, experience and resources to perform the research, but there are some minor limitations that need to be addressed or a clear description of impact is missing.
Applications with a range of 3.5 to 3.9 are defined as compelling but have limited scope or impact. They may also have raised some concerns about the feasibility and/or team; or in other words, the grant has strengths, but needs work.
Rating Scale 3
The following ranges indicate an application which will not be funded.
Applications with a range of 3.0 to 3.4 are defined as having merit but with many limitations.
Applications with a range of 0 to 2.9 are defined as having significant flaws and are not ready to be funded.
Budget Assessment
In this section, reviewers will learn about the budget assessment in the Project Grant competition.
Budget Assessment 2
One additional component of the review process is for reviewers to check the requested budget and justification, although detailed scrutiny of each item is not expected.
Reviewers are required to determine if the requested budget is appropriate to support the proposed project, and if it is realistic and well justified in relation to the applicant's current funding. It is incumbent on the applicant to illustrate clearly to reviewers why the requested funds are needed, and how they are distinct from funds currently held. Reviewers may recommend that the budget remain as requested or recommend a reduction.
Reviewers are required to provide comments in the "Budget" section of their reviews to support their recommendation, regardless of whether they recommend that the budget remains as requested or recommend a reduction.
The budget assessment must not be factored into the scientific assessment and must not influence the rating of applications. While budget recommendations are not factored into the overall rating of an application, CIHR values the experience and perspectives of reviewers in estimating overall budget requirements of the proposed project.
Additional Resources
The resources listed on screen will provide you with additional details to prepare you for reviewing applications in the Project Grant competition. Before concluding this module, please complete the survey to assist CIHR in tracking the uptake and improving the quality of the learning.
You may choose to exit the module and return to the learning page or continue to the next part of the Project Grant competition series, Part 4: Committee Member Roles.
- Date modified: