Research Impact in Health Services and Policy Research

What is research impact?

Research impact refers to the value, benefit and change made in society as a result of research. There is increasing global recognition that although necessary, tracking traditional academic outputs (e.g., publications, grants) is no longer sufficient to determine and communicate the full reach and significance of research contributions, as acknowledged by the San Francisco Declaration on Research Assessment (DORA).Footnote 1Footnote 2 For example, research can have an effect through mentorship, advances in equity, community engagement, and infrastructure development. Other types of research impact (beyond academia) include: enhancing the capacity of health system organizations to conduct and use research; informing health policy and/or practice decisions; enhancing co-design and stakeholder engagement in research; improving health equity, patient care and outcomes, and health system performance; and more.

What is research impact assessment?

Box 1 - Theory of Change in Health Services & Policy Research (HSPR)

How research achieves impact: Co-identifying pressing problems with end users that warrant research attention, coupled with targeted investments in HSPR and capacity building, produces the timely and relevant evidence and conditions for the mobilization of evidence to inform decision-making, which can contribute to improved system performance and health outcomes. Context and external influences play a mediating role along the full pathways to impact.

Research impact assessment is the systematic approach to determining the links between research and change in the real world. The CIHR Institute of Health Services and Policy Research (IHSPR) uses two complementary, conceptual frameworks – the CHSPRA Informing Decision-making Impact FrameworkFootnote 3and the CAHS Making an Impact frameworkFootnote 4 – to guide, monitor, assess and improve its contributions to research impacts in health services and policy research (HSPR)Footnote 5. Box 1 (right) outlines the theory of change and non-linear pathways to HSPR impact, whereas Figure 1 (below) outlines a sample of impact categories and outcomes.

Figure 1 –CAHS Impact Categories and Outcomes

Figure adapted from: Graham KER et al. 2018. Assessing Health Research and Innovation Impact: Evolution of a Framework and Tools in Alberta, Canada.

Long Description

The CAHS impact categories and outcomes can be summarized into five broad groupings: Capacity building, advancing knowledge, informing decision making, health impacts, and social and economic impacts. Each broad category comprises a menu of indicators that reflect how a research activity may directly or indirectly influence, change or inform an anticipated outcome or impact. For instance, a research activity or investment may directly or indirectly result in building capacity in: people (skills, trainees, embedded scholars), diverse teams, organizations, networks (partnerships, collaborations), infrastructure, data, and research methods. Research typically also advances knowledge through: the co-identification of problems and priorities with end-users, bibliometric outputs (publications, presentations), and/or alt-metric outputs (media mentions, social media). Research could also build the evidence base to inform or influence decisions (priority- and agenda-setting, e.g., via co-design approach), policy or practice. Eventual non-linear pathways to impacts could include improving health outcomes (e.g., improving social determinants of health; or equitable access and care delivery) and social and economic impacts (e.g., enhancing the Quadruple Aim, equity for all, health system performance or commercialization of a product/tool/service).

Box 2 - How does IHSPR assess impact in HSPR?

IHSPR aims to maximize the value of HSPR investment and optimize the potential for research impact. Using a “relevance by design” approach, IHSPR tries to ensure that health system priorities are reflected within funding programs and institute activities. IHSPR uses continuous learning and improvement methods to assess, evaluate and share the impacts of its research investments to the community and to ensure its approaches and activities are based on evidence.Footnote 4

Why assess and share it?

The reasons for measuring and sharing research impacts can be summarized by the “6 As” purposes: Analysis, Advocacy, Allocation, Accountability, Acclaim and AdaptationFootnote 6. Researchers, funding agencies and governments have a vested interest and responsibility to assess and articulate the impacts of research investments in an accessible way to end-users (policy-makers, providers, patients, public).

How and why can you contribute?

Research impact assessment and communication can be difficult because of common challenges such as: time lags between the research and actualized impacts; difficulty in establishing a causal link from research to real-world change; intensive resources needed for impact assessment; incorporating and communicating the collective impacts that are valued by all stakeholders. As a result, impacts are often not fully understood or captured and can be under-valued relative to traditional academic outputs. Many organizations and people around the world are working to improve the ways research and its impacts are assessed, valued, and communicated.Footnote 4

Impact assessment is a collective effort. As a researcher or research partner, you can contribute by:

  • Developing an Impact PlanFootnote 7 and knowledge mobilization strategies at the start of your research.
  • Monitoring, assessing and sharing your research impacts (Tip: the CHSPRA Impact frameworkFootnote 2 includes a prioritized list of impact indicators and a template for writing an Impact Narrative)
  • Reaching out and sharing your impact story with the IHSPR team or CIHR’s health research in action
  • Completing your end-of-grant reporting

Questions? Reach out to the CIHR-IHSPR team (info.ihspr@ices.on.ca)

What is in it for you?

By contributing to these activities, you can personally reap the benefits by increasing the dissemination/mobilization of your findings, elevating your research profile, and helping to focus and improve the impacts of your own research programs. In turn, research funders use this information to understand and communicate the impacts and value-add of research investments. Collectively, we help advocate for and build an understanding of how research investment allocation can be optimized, how to best design impact-generating research funding programs, what can be improved for your research community, and how we can work together to bring about health system improvements to improve the lives of patients, providers, communities and populations.

Date modified: