Reviewing for the Fellowship Awards Program
Title
Welcome to this learning module for peer reviewers of the Fellowship Awards Program. The goal of this module is to ensure that reviewers understand the Fellowship Awards Program and feel prepared to effectively participate in the review process.
Navigation
This course is designed to be self paced.
Use the playbar below to resume playback, navigate between slides, mute and unmute audio, and toggle closed captions. You can also browse the full table of contents, and collapse or move the playbar.
Objectives
By the end of this module, you will be able to: identify key features of the Fellowship Awards Program, understand the adjudication criteria and rating scale used in the evaluation of applications, and summarize the steps in the peer review process for the Fellowship Awards Program.
Objective 1: Fellowship Awards Programs
In this section, you will learn about the Fellowship Awards Program and the eligibility criteria for applicants.
The Fellowship Awards Programs
The objective of the Fellowship Awards Program is to provide support for highly qualified applicants in all areas of health research at the post-PhD or post-health professional degree stages, and to add to their experience by engaging in health research either in Canada or abroad.
The value and term of the awards varies depending on the recipient's credentials and location of tenure.
Eligibility of Applicants
Please note that the eligibility requirement that all trainees be within 3 years of having completed their PhD has been removed.
Reviewers are asked to think critically about whether the research training position or program for which an individual is applying for funding will have the desired career benefits and impact compared to other applicants.
CIHR staff have reviewed all applications to confirm the eligibility of the applicants.
Reviewers may flag eligibility cases in the review process if they believe an application that should have been screened out was allowed into the competition.
To help to conduct effective and fair peer review, reviewers must also complete the Bias learning module.
Formatting Requirements
Although the applicants can present their research proposals in a free form format, the reviewers assess the application based on the specific adjudication criteria.
To ensure that all applicants have the same amount of space to write their research proposals, applicants must adhere to the formatting requirements outlined by CIHR such as font sizes, spacing, and page limits.
Should a reviewer be assigned an application that may not have followed instructions, this should be brought to the attention of CIHR staff as soon as possible.
For the full list of formatting requirements, consult the Acceptable Application Formats and PDF Attachments webpage.
Principles of Peer Review
The integrity of the peer review process relies on well-established principles and policies that ensure fair and effective evaluation, and support CIHR's mandate and objectives.
CIHR's principles of review relate to: confidentiality, conflict of interest, fairness, and transparency.
Review Quality Expectations
CIHR has established a set of guiding principles to help support high quality in peer review.
For each application, reviewers must provide a concisely written assessment highlighting the strengths and weaknesses for each adjudication criterion.
When entering written feedback reviewers are asked to consider the following review quality criteria:
- Appropriateness - Review comments are fair, understandable, confidential and respectful
- Robustness - Review is thorough, complete and credible
- Utility - Review provides feedback that addresses the needs of reviewers, applicants and funders
CIHR has developed a checklist as a practical tool to assist both applicants and reviewers to apply the review quality criteria to their received or submitted reviews. You can also explore the conduct quality reviews learning module to learn more.
Objective 2: Adjudication Criteria and Rating Scale
In this section, you will learn about the adjudication criteria and rating scale used for the Fellowship Awards Program.
Assignment of Applications
You will primarily be assessing the applicant and the research training environment. As such, it is not essential for your research expertise to align directly with the research area of the application.
Reviewers are asked to apply their research expertise generally when assessing the diverse array of application assignments.
Adjudication Criteria
The evaluation of the application should be based on the following adjudication criteria: Achievements and Activities of the Applicant, Characteristics and Abilities of the Applicant and the Research Training Environment.
Please take into account the career stage of the applicants when reviewing applications to better assess and calibrate your set of applications.
Adjudication Criteria
In the following slides you will learn more about the adjudication criteria listed below.
Click the button in the top right for additional details on the Adjudication Criteria found in Appendix A of the Reviewers' Guide for Fellowship Awards.
Note: The program literature has been updated to incorporate the San Francisco Declaration on Research Assessment (or DORA) guidelines. CIHR is a signatory of DORA, which "recognizes the need to improve the ways in which the outputs of research are evaluated".
Adjudication Criteria
Achievements and Activities of the Applicant is divided into 4 sub-criteria: Training Expectations; Proposed Research Project; Honours, Awards and Academic Distinction; and Research-Related Contributions and Activities.
For this section review the CCV of the Applicant including the Additional CCV Information, Training Expectations and Research Project Summary.
Adjudication Criteria
The second criterion is the Characteristics and Abilities of the Applicant. For this criterion focus your review on the three sponsor assessment forms in the applicant's application package.
They should offer a perspective on the applicant from individuals who are familiar with the applicant's characteristics and abilities.
Recognize that positive comments are common while negative ones are not. Keep in mind that applicants have no opportunity within the application to provide a justification for their choices of sponsors.
Adjudication Criteria
The Research Training Environment considers the career stage and discipline of the supervisor(s). Expectations of mentoring by a recently established investigator should differ from your expectations of mentoring by a long-established researcher. For this section review the CCV of the primary supervisor and the research training environment sections of the application.
Note: applicants should not be penalized when the choice of research environment is duly justified (Examples of valid justifications are scientific reasons, family reasons, health reasons, applicants conducting research with Indigenous communities and/or reasons related to community or cultural responsibilities. Indigenous communities are broadly defined as individuals, groups, organizations, and populations who self-identify as Indigenous (First Nations, Inuk (Inuit), and/or Métis) living and working anywhere in Canada.).
Sex and Gender Integration – Resources
Learn more about Sex, Gender, and Health Research by exploring the resources on this slide. All links will open in new windows, so you can return to this module when you are ready.
The Rating Scale
The ranking of Fellowship Award applications is completed using a structured review process. In a structured review process, reviewers are asked to provide an assessment for each adjudication criterion.
The score that you submit for each criterion on the 0.0 to 4.9 scale will be weighted automatically in the calculation of an overall score for each application you review in ResearchNet.
Reviewers should be assessing each application based on the information provided and should not be seeking additional information or outside opinion to supplement what is contained in the application.
Remember that all applications with a final rating of 3.5 or higher may be considered for funding.
As such, if you feel that an application should not be considered for funding, it is important that you assign a score below 3.5 to the application. Please use the full rating scale when evaluating your assigned applications.
Test your knowledge!
Test your knowledge!
You may refer to Appendix A of the Reviewers' Guide for Fellowship Awards to assist you.
In the application you are reviewing, the applicant has justified why they chose their proposed training location and what they expect to learn.
Which sub-criterion of "Achievements and Activities of the Applicant" does this apply to?
- Proposed Research Project,
- Training Expectations,
- Honours, Awards and Academic Distinctions, or
- Research-Related Contributions and Activities
Test your knowledge!
Which of the following does not apply when considering "Research Training Environment" in your evaluation?
- Training record of the supervisor,
- The appropriateness of research environment including space, personnel and facilities,
- Sponsor's assessment letter about applicant, or
- Supervisor's research experience.
True or False?
True or False. To ensure that all applications are treated equally, reviewers should NOT perform additional research in addition to their evaluation on the content of the application.
Objective 3: The Review Process
In this section, you will learn about the review process for the Fellowship Awards Program.
Peer Review Process for Fellowship Awards
We will discuss the following steps of the peer review process.
- assigning applications to committees;
- identifying conflicts of interest;
- conducting reviews of assigned applications
- submitting reviews and ratings, and;
- participating in re-review, if required.
Distribution of Applications by Committee
Applications are assigned to a specific committee based on the following: the theme of the proposed research activities; the applicant's educational background at the time of taking up the award; and the applicant's licensure status, if applicable, at the time of the competition's application deadline date.
A diagram summarizing the peer review committees for Fellowship awards is displayed on your screen. Please visit the web link provided if you would like more information.
Identify Conflicts
In ResearchNet, you will first need to agree to the Conflict of Interest and Confidentiality Agreement, which includes the expanded version of the Equity, Diversity and Inclusion Self-identification Questionnaire. Completing this questionnaire is a requirement for peer review committee members, enhancing our ability to understand the degree of diversity in peer review committees.
The questionnaire is accompanied by a Privacy Notice Statement which outlines CIHR's intended purpose and uses of the self-identification data.
Next, to start conducting your reviews, you will click on the "Manage Conflicts/Ability to Review" task.
This will bring you to the set of applications that have been assigned to you.
You'll select each application by clicking on the hyperlinked application numbers.
You will have access to the relevant information to determine if you are in conflict. This is also where you will indicate whether you are in conflict or not in conflict with the application.
Conduct Reviews
Please ensure you are familiar with the adjudication criteria for the Fellowship Awards Program. You can now start working on your reviews by clicking on the "Conduct Reviews" task.
To assess the applications, click on the hyperlinked application numbers.
From this screen, insert your numeric rating. Then, you must provide written comments that justify your ratings, and highlight the strengths and weaknesses of the application for each adjudication criteria. We recommend using the adjudication criteria as titles to section your comments. The written reviews will provide constructive advice to applicants to assist them in improving the quality and efficiency of the proposed training.
Submit Reviews
When you are ready to submit your reviews, you will first need to select the reviews and then click on "Submit Selected Final Review".
When all reviews are submitted, a calculation is applied to determine which applications have received discrepant scores.
In this instance, CIHR will ask these reviewers to discuss the application in order to reconcile their scores.
If you are requested to do a discrepancy review on an application, you will be notified by e-mail with the contact information of the other reviewers and your access to that specific application on ResearchNet will be re-opened.
The goal of this process is to have all three reviewers listen to and consider each other's opinions. In the end, you are not obliged to change your score but we ask that you re-submit.
Once all re-reviews are submitted, the final score for each application will be determined by taking the average of the three scores.
Finally, CIHR will generate ranking lists for each committee.
Applications are recommended for funding by committee from the top down in order of ranking as far as the budget will allow.
Summary
Congratulations! You have now completed the learning module for peer reviewers of the Fellowship Awards Program.
You should now be able to: identify key features of the Fellowship Awards Program, understand the adjudication criteria and rating scale used in the evaluation of applications, and summarize the steps in the peer review process for the Fellowship Awards Program.
Survey
Before continuing on to explore the Additional Resources, please complete the survey to assist CIHR in tracking progress and improving the quality of our learning. The survey will open in a new window.
Additional Resources
Click on the additional resources below to view important links for peer reviewers of the Fellowship Awards Program.
- Date modified: