Strategy for Patient-Oriented Research – Evaluation Summary

About the Strategy for Patient-Oriented Research

The Strategy for Patient-Oriented Research (SPOR) was launched in 2011 as a response to a recognized need for greater uptake of researchbased evidence to improve the health of Canadians while improving the cost-effectiveness of the health care system. SPOR's vision is for Canada to have demonstrably improved health outcomes and enhanced the health care experience for patients through the integration of evidence at all levels of the health care system by 2025.

Results: What We Found

• There is a continued need to prioritize and foster patient-oriented evidence-informed health care in Canada, with substantive evidence of the relevance and benefits of patient engagement on the research process. SPOR is aligned with the roles and responsibilities of the federal government, and CIHR is well positioned to continue to play a leadership role in SPOR, particularly as a research funder and as a coordinating body or convener.
• In general, the design features of SPOR support the achievement of intended outcomes; however, communication within and across the core elements was identified as inadequate, resulting in duplicative efforts rather than a cohesive approach.
• SPOR's governance structure is not meeting its current objectives and lacks adequate patient representation. The National Steering Committee has not met in recent years and generally provided advice rather than steering the SPOR program.
• While collaboration between CIHR and partners was generally reported to be satisfactory, SPOR's current approach to patient engagement does not adequately support recruitment of diverse patient partners, with some patient partner groups disproportionately underrepresented in SPOR research (e.g., Indigenous communities). Additional challenges include a lack of harmonized patient compensation standards, the need for a safe and supportive sharing environment for patient partners, and opportunities for increased awareness of ongoing SPOR activities.
• SPOR's core elements are contributing to the achievement of immediate outcomes, including the generation of new knowledge, infrastructure, capacity development and engagement of patients and stakeholders. Additionally, SPOR's core elements are contributing to the achievement of intermediate outcomes including the establishment of infrastructure and support services that respond to stakeholder needs and the application of research evidence in decision-making.
• SPOR's core elements are also contributing to the achievement of a cultural shift towards patient-oriented research – a key expected ultimate outcome that should be maintained. At this point in time the evaluation found little evidence to demonstrate that SPOR has contributed to the expected ultimate outcomes to improve patient health care experiences, health outcomes or health system outcomes.

Recommendations and Management Response

1. CIHR should use SPOR to inform an organization-wide approach to patient engagement in research to continue its leadership role, further investment and sustain progress on the outcome of a cultural shift toward POR.

   Response: CIHR will develop and implement a POR Action Plan to embed patient-oriented research approaches across its funding programs.

2. CIHR needs to do the following to improve the program design and delivery of SPOR:

   • Increase awareness of the benefits of patient-oriented research among the Canadian public, research community, decision-makers.
   • Enhance communications among and across SPOR core elements and CIHR institutes to avoid duplicative efforts, promote cohesion, and enhance partnerships.
   • Improve overall program monitoring to ensure that research is delivering on intended objectives, such as the engagement of communities and patients in research and provide feedback.
   • Establish consistent priorities, mandates and readiness across SPOR core elements to support linkages, alignment and coordination of initiatives.

   Response: a) CIHR will embed a focus on patient-oriented research as part of the agency's brand, core communications messages and planning. CIHR will develop an enhanced outreach and communications strategy for SPOR, based on updated strategic objectives as determined through the SPOR Refresh. b) CIHR will work with SPOR funded entities and other partners to identify and implement an action plan to enhance collaboration and partnerships and reduce duplication. c) Based on the SPOR Refresh, CIHR will develop a new SPOR logic model, performance measurement framework, indicators, and reporting tools to enhance data collection – including data related to patient health care experiences, health outcomes and health system performance, in consultation with partners. d) Based on the results of the SPOR Refresh, CIHR will revise POR funding opportunity objectives and evaluation criteria to enhance consistency, linkages, and alignment across SPOR investments.

3. CIHR should re-establish an external and internal governance structure for patient-oriented research with defined roles and responsibilities, including better representation from patients, partners, and funders, to improve SPOR's decision-making on patient-oriented research.

   Response: CIHR will renew internal and external governance structures for SPOR, following the SPOR Refresh. It is anticipated this structure will take a collective impact approach, influencing both CIHR and partner decision-making on patient-oriented research initiatives.

4. CIHR needs to improve patient and community engagement both in SPOR and in research in the following manner:

   • Embed equity, diversity and inclusion considerations into the recruitment of patient partners to address the underrepresentation of important patient partner groups in research.
   • Harmonize patient compensation standards across SPOR.
   • Enhance accountability for meaningful patient engagement.
   • Ensure consistency in engagement of Indigenous community members across SPOR core elements.

   Response: a) CIHR will develop and disseminate resources outlining leading practices for building Official Languages; equity, diversity and inclusion into the engagement of PWLLE in research. b) CIHR will work in collaboration with others within CIHR and Tri Agency partners to publish compensation guidelines and lead by example in changing the culture across research organizations and systems. c) CIHR will enhance support for peer reviewers to better assess the quality of patient engagement activities within grant proposals by developing guidance materials and clear evaluation criteria. CIHR renewed performance measurement framework will include indicators to measure impact and influence of patients on research. d) CIHR will engage Indigenous Health research communities as part of SPOR Refresh activities to co-develop approaches for SPOR to support meaningful and respectful involvement of Indigenous health researchers and communities across all future SPOR investments.

5. CIHR should improve the management and reporting of SPOR performance measurement data to better inform decisionmaking by establishing a clear set of measures to track progress on expected outcomes related to patient health care experiences, health, and health system.

   Response: Based on the SPOR Refresh CIHR to develop a new SPOR logic model, performance measurement framework, indicators and reporting tools to enhance data collection – including data related to patient health care experiences, health outcomes and health system performance, in consultation with partners. CIHR will consider past performance data in all proposals for funding extensions and renewals.

6. CIHR needs to further improve the following aspects of its financial monitoring and coding for SPOR:

   • Grants and awards expenditures, especially coding of core elements and tracking of partner contributions.
   • Operating and maintenance expenditures, specifically direct salary costs.

   Response: Program lead (SPOR) to identify the specific program elements that are required to be tracked. Once identified, the Finance Unit will support the Program sector by creating specific financial coding elements within the Grants Management System. Moving forward, SPOR is to identify which positions are meant to support the program and at what percentage, which will allow Finance to track the proper positions for future reporting. Finance to continue to track the non-salary expenditures within the SPOR Fund center (i.e., travel and other related expenses for SPOR meetings, SPOR Summit, as well as staff expenditures such as equipment, training and conference fees). For expenditures in other Fund centres, SPOR to identify which activities directly relate to the SPOR function. Once identified, Finance to support in tracking these costs within the Financial System. For partner contributions by Applicant Partners, options regarding collecting applicant partner contribution information will be explored and a recommendation developed for CIHR with representatives from Partnerships, PDD, LHS and Finance, IMIS, Operations Support.

About the Evaluation

CIHR's Evaluation Unit conducted the evaluation to meet requirements of the Policy on Results and provide senior management with valid, insightful and actionable findings regarding: needs addressed by the program; effectiveness of program design; achievement of expected results.

Associated Links

• Full Evaluation Report
• Management Action Plan (MAP)

Date modified:

2023-11-10