eHealth Impact Assessment: Summary report

Infographic: Evidence from CIHR's eHealth Innovations Initiative Impact Assessment

In 2012, two CIHR Institutes, the Institute of Health Services and Policy Research (IHSPR) and the Institute of Aging (IA), co-developed the eHealth Innovations Initiative (herein referred to as eHealth Initiative) to promote research and innovation in digital health. The overall aims of the eHealth Initiative were to develop, integrate and evaluate, in collaboration with stakeholders, eHealth innovations that will improve the quality of outcomes and the cost-effectiveness of patient and population-centered care, and enhance Canada's position in digital health.

The eHealth Initiative launched a sequential series of funding opportunities between 2012 and 2019: Catalyst Grant: eHealth Innovation in 2012, eHealth Innovation Catalyst Grant in 2013, eHealth Innovations Partnership Program (eHIPP) in 2014 (the initiative's flagship program), and the Spread and Scale of Existing Community-based Primary Health Care and eHealth Innovations in 2019. The catalyst grants were designed to prime the research community to work on eHealth solutions and to encourage active engagement with patients and other end-users, whereas the eHIPP grants were aimed at taking this initial engagement one step further and providing opportunities to co-develop, implement, and evaluate interventions through strong collaboration between healthcare innovation communities (i.e., researchers, health care providers, patients, policy/decision makers) and industry.

One common strategic objective for both IA and IHSPR was to support research that would lead to digital health solutions for older adults with complex care needs and, as such, the eHealth initiative benefitted from an international partnership with the European Commission on the Active and Assisted Living (AAL) Programme in 2015.

Across the entire eHealth Initiative, a total of 59 grants were awarded over the 2012-2019 period, representing a total CIHR investment of $17.8M and $29M from (competition and applicant) partnersFootnote 1, with $11M going toward digital health solutions for older adults in Canada.

To evaluate outcomes and impacts of the eHealth Initiative, CIHR commissioned Small Globe Inc. to conduct an impact assessment with the following objectives:

  1. Provide evidence about the initiative's successes and challenges in achieving its core objectives;
  2. Identify the initiative's key outcomes and impacts;
  3. Examine the appropriateness and effectiveness of the eHealth Initiative funding design;
  4. Identify the main remaining needs, gaps, and opportunities for future research investment.

Methodology

To conduct the impact assessment, Small Globe Inc. carried out a mixed-methods approachFootnote 2 involving a comprehensive review of documents, a focus group discussion with the eHealth Initiative program architects and current Scientific Directors of IHSPR and IA, online surveys of Nominated Principal Investigators (NPIs, n=59), semi-structured interviews with NPIs and project partners (n=12), and thematic analysis and triangulation to produce the final report.

Outcomes and impacts

Health research advancements: There was a strong consensus among NPIs (92%) that the eHealth Initiative had an important role in advancing digital health research in Canada. There was a broad range of research outcomes and a strong focus on developing and testing eHealth interventions in the areas of digital health solutions for older adults with complex care needs and youth mental health, the two priority areas for eHIPP. The most common types of interventions were virtual care/telemedicine (38%), home health monitoring applications (38%) and patient portals (30%) — most of which were evidence-informed, proven to be effective through systematic testing and ready for broad scale and spread.

Health and health system impacts: According to the surveyed NPIs, the eHealth Initiative interventions reportedly contributed to improved health outcomes (51%), improved population health (47%), enhanced health equity (44%) and changes in determinants of health (42%). Additionally, the NPIs confirmed their projects contributed to several health system impacts including, better patient experience (67%), improved cost effectiveness (50%), improved equitable access to health care (47%), better provider experience (45%), changes in health system practices (45%), improved health system performance (28%) and changes in health policy (19%).

Enablers and barriers: The market demand for these interventions and their co-creation with patient representatives/end-usersFootnote 3 were cited as enabling factors encouraging impacts. The COVID-19 pandemic was also an enabler to innovation, giving rise to more positive attitudes with respect to digital health interventions and a greater need for, and shift towards, alternatives to face-to-face provision of health services. The main barriers to impact were health care providers' lack of time to support the use of the interventions and frequent turnover in the health care labour force. While sometimes considered as an enabler, the pandemic was also seen as a barrier, as it impeded research progress due to challenges with recruitment of research participants and testing of interventions.

Partnerships

The eHealth Initiative had a considerable impact on creating partnerships across stakeholders and the partners heavily contributed to developing and implementing the interventions. The knowledge exchange between partners was essential for adjusting and adapting the interventions and aligning them to the needs and preferences of patients and other end-users.

Types of partnerships: Researchers reported collaborations with health care providers as being key to the success of their research projects (92%). Collaboration with patient representatives (78%) and with industry (72%) were also valued highly. Collaboration with policy/decision makers was less frequently considered a key to success (36%), and a subset noted this type of collaboration was not applicable at all (25%)Footnote 4. While international partnerships were commonly reported across all eHealth Initiative competitions (42% respondents), they were not as critical to the success of the eHIPP or the catalyst grant projects.

Timing and sustainability: The eHealth Initiative helped to sustain existing partnerships (53%) and form new partnerships (47%), particularly with industry and patient representatives. Most survey respondents projected that their partnerships would be sustained beyond five years, particularly the partnerships between the researchers and health care providers (84%) and patient representatives (83%).

Challenges: The main challenges reported with partnerships included sales and acquisitions of industry partners during the lifetime of the grants; and the turnover, or lack of involvement by decision-makers. It was also believed that the eHIPP priority areas and proposed solutions could have been more closely aligned to the needs and priorities of each jurisdiction across Canada.

Program Design Elements and Future Programming

Most researchers felt that the eHealth Initiative was designed effectively to obtain the intended objectives, with strong consensus on the important role of partnerships.

There was a strong agreement among the eHealth Initiative-funded researchers that CIHR should continue its investment in digital health. Recommendations included:

Date modified: