Case Study # 1: CIHR’s Community Reviewers Program

In the fall of 2004, CIHR started the Community Reviewers Program as a pilot project to involve the public in the CIHR peer review system. This project reflected CIHR's commitment to enhancing public and stakeholder engagement in health research in Canada, and also increased the transparency and accountability of our peer review processes. It began as a pilot project in order to establish the feasibility of including the public in peer review committees. Throughout its evolution from pilot project to full program, the Community Reviewers Program has enjoyed continuous endorsement and support. The pilot project officially became a full program in the summer of 2007. Initially, four peer review committees were involved in the program; but, as of the spring of 2009, the program has grown to include 25 committees. Essentially, one community reviewer is assigned to one of CIHR's Open Operating Grants peer review committees. While community reviewers are non-voting members of these committees, they do have a number of responsibilities. Part of their role is to provide written feedback to applicants on all lay abstracts submitted to the committee and pick projects of public interest for CIHR to highlight through communication activities. They may also work with CIHR to enhance external communication and public engagement. In addition, the community reviewer provides written feedback on the proceedings of the committee (such as the quality, quantity, and variety of science reviewed) and the structure and objectivity of the discussions.

Since its initial implementation, refinements have been made to the program's tools for the volunteer selection process, orientation and training, roles and responsibilities, and feedback loops. In the beginning, there was a lack of clarity in defining the program's objectives and the recruitment/orientation processes were conducted in a relatively informal manner. It was difficult to determine the best way to ensure meaningful and appropriate involvement of non-scientists in a peer review environment. Initially, there was also some apprehension from stakeholders who were concerned about the level of involvement from these individuals and how they would impact the peer review discussions. Today, through orientation and training, these apprehensions have been reduced significantly. This orientation and training for the Community Reviewers in Peer Review Program is very comprehensive and includes training for the community reviewers themselves, but also for CIHR staff members and committee chairs, as well. Various sessions have been designed for each target audience; they include PowerPoint presentations and written documentation.

Recruitment of the community reviewers involves specific selection criteria. A community reviewer must be impartial and must value and support the mission of CIHR. This reviewer cannot be currently involved in academia or health research but must have experience working on a committee. The goal is to get broad representation on the committees, but the reality is that those who volunteer to be a community reviewer usually have a vested interest in health or scientific research.

The program uses a self-nomination process through ResearchNet. The opportunity to become a reviewer is "advertised" through existing committee members and community reviewers, CIHR staff, the Canadian Association of University Research Administrators, CIHR's university delegates, CIHR e-news alerts, CIHR's Institute Advisory Boards, and CIHR's website. The selection process for new volunteers begins with an initial screening of the current online applications. Once an applicant is deemed to meet the selection criteria, the process proceeds to a screening interview via telephone and a reference check. During the phone interview, the Community Reviewer Program Officer explains clearly (and realistically) what becoming a community reviewer entails. This allows the volunteers to develop an informed opinion about whether or not the position is something of interest to them — and something that they can commit to in their schedule. The selection process is finalized by the CIHR Deputy Director and the chair of each peer review committee. Once the process is complete, the new community reviewer is sent a confirmation letter and asked to serve a three-year term.

Feedback mechanisms are crucial to the program's success. A Community Reviewers Feedback Form was created to solicit input from the community reviewers on the value and quality of their orientation, the ease of use of ResearchNet, CIHR staff's level of service, their experience at the committee meeting, recruitment, and general feedback. Similarly, an Evaluation Questionnaire is given to the committee Chairs and Deputy Directors to gather their comments about the role, involvement, and performance of the community reviewer on their committee. This system allows CIHR to stay in touch with both sides of the program and constantly evaluate the alignment of the program with its objectives.

Overall, the program provides valuable lessons in effective volunteer management: it involves in-depth orientation and training, selection, placement, and evaluation. Above all, CIHR staff members must remember that the community reviewers are volunteers (i.e., they are giving their time because they are interested and want to give back). While peer reviewers are also volunteers, they are volunteers of a different sort; researchers who are funded by CIHR are expected to volunteer their time to contribute to the peer review system and they also benefit from the career development that such experience brings. Community reviewers, in contrast, are members of the public who approach CIHR on their own to contribute to the health research enterprise on their own time. The organization needs to be realistic in what it asks these volunteers to do and needs to appreciate whatever contributions they can make to CIHR.

This program has tremendous value for CIHR. Not only does it demystify CIHR and the peer review process for members of the public, but it also helps us to develop spokespeople for CIHR and for health research in general. Community reviewers can also help health researchers to understand the importance of engaging the public and to write for a lay audience. Researchers often say that writing the lay abstract is the most difficult part of an application, so feedback from a lay reviewer can be extremely beneficial. The success of the program has led to buy-in from key stakeholders and the program expansion (noted above). CIHR has received positive feedback from the Tri-Council, universities, and the public. Through the program, CIHR has also been able to identify activities to help communicate the merits of investment in health research.

Date modified:

2014-01-03