Internal Audit of the Research In Priority Areas Program
Audit Report

March 2019

Contents

• 1. Executive Summary
  • 1.1. Background
  • 1.2. Why this is important
  • 1.3. Objective and Scope
  • 1.4. Overall Audit Opinion
  • 1.5. Key Audit Observations
  • 1.6. Statement of Conformance
  • 1.7. Disclosure of Potential Impairment to Independence and Objectivity
• 2. Detailed Report
  • 2.1. Background
  • 2.2. About the Audit
  • 2.3. Significance of the audit
  • 2.4. Audit Objective and Scope
  • 2.5. Audit Criteria and Methodology
  • 2.6. Observations and Recommendations
  • 2.7. Conclusion
  • 2.8. Management Response to Recommendations
• Appendix A: Color coding of recommendations:
• Appendix B: Relevant Audit Criteria
• Appendix C: Glossary
• Appendix D: RPA Program multi-year budget commitments (As of April, 2018)
• Appendix E: RPA Initiatives (as of April, 2018)

1. Executive Summary

1.1. Background

CIHR invests approximately $1 billion each year to support health research. This funding is divided into three Programs: Investigator-Initiated Research, Training and Career Support, and Research in Priority Areas.

The Research in Priority Areas Program funds priority-driven research (also known as "strategic" or "targeted"). As stated in its program description, the RPA Program "…provides funding for targeted grants and awards aimed at addressing priority areas". The research funded by this Program is intended to address pressing health issues that are of strategic importance to Canadians, identified through government priorities, Institute-driven activities, and national and international consultations with health research and health care stakeholders. At a higher, mandate level, the RPA Program also enables the Agency to address §4(f) of the CIHR Act.

The budget for RPA in 2018/19 was $292M, divided into the following five funding envelopes:

• Institute Strategic Initiatives ($58M);
• Roadmap Accelerator Fund ($46M);
• Ring-Fenced Initiatives ($108M);
• Enabling Platforms and Other Programs ($30M); and
• Flow-Through funding ($50M).

1.2. Why this is important

The RPA Program funds approximately one-third of CIHR's granting budget to deliver against section 4f of the CIHR Act, "addressing emerging health opportunities, threats and challenges and accelerating the discovery of cures and treatments and improvements to health care, prevention and wellness strategies". As such, the RPA Program is an important part of CIHR's business and instrumental in helping the agency meet its objectives under the CIHR Act.

The RPA Program has not been the subject of any previous audits.

1.3. Objective and Scope

The objective of the audit was to provide assurance that:

1. Governance and risk management practices were established and operating effectively for the management of the RPA Program;
2. Operational resourcing and planning were adequate and effective to support program delivery.

Given that the RPA Program has never been audited, the scope of the engagement remained at a high level and focused on the broad design and delivery of the Program. The audit focused primarily on the activities of the 2016/17 fiscal year, though documents and activities outside of that period were also reviewed as appropriate.

1.4. Overall Audit Opinion

The audit has concluded that while structures exist to manage the RPA Program, there are limitations impacting CIHR's ability to respond to emerging priorities which require improvement.

1.5. Key Audit Observations

The following findings require management's attention:

1. RPA's Program-level governance was unclear.

   The audit found that the RPA Program did not have clearly defined program-level governance. A review of corporate documentation, including committee Terms of Reference, job descriptions and delegations of authority demonstrated that key roles and responsibilities for the RPA Program were not formally defined, such as those related to oversight; priority setting; budget setting and funding allocation; and risk management. Instead, individual funding opportunities and activities within the RPA Program relied on their own governance, working groups, or ad hoc mechanisms. A lack of clear program-level governance to ensure priorities are set, sufficient funding is in place, and program-level risks are managed, could result in the Program not achieving its objectives.

2. The RPA Program did not operate within an integrated framework.

   The audit found that while CIHR reported the RPA Program as a single Program in its DRF, the Program was not operating as a single, integrated program. Rather, the RPA Program was comprised of a collection of several dissimilar programs, including Institute Strategic Initiatives, the Roadmap Accelerator Fund, Ring-Fenced Initiatives, Enabling Platforms and Other Programs, and Flow-Through funding. Furthermore, each of these had their own governance, set of priorities, funding mechanisms and methods for measuring and managing risk and performance. The absence of clear priorities, coupled with a complex structure, has limited CIHR's ability to manage the RPA Program as a coherent and integrated program. As such, CIHR has not been able to fully optimize the manner in which the Program is leveraged to address priority health issues for Canadians.

3. The RPA Program had limited flexibility to respond to emerging health research priorities.

   The audit found that the RPA Program had limited flexibility to support new and emerging health research priorities. In 2018/19, 90% of the RPA Program budget was allocated to non-discretionary ring-fenced/flow through initiatives and CIHR's Strategic Initiatives, funds that were largely committed and could not be used for other funding purposes. The remaining budget (EPOP), representing 10% of the Program's overall funding, was discretionary but was also largely committed to funding ongoing research. This suggests there was little remaining to support new or emerging health research priorities should a need arise during the year. Without sufficient funding from a permanent and flexible budget, CIHR could be constrained in responding effectively to the emerging health needs of Canadians. This is particularly problematic for needs that arise quickly, cannot be planned for, and/or require timely action.

4. CIHR had a limited ability to plan or prioritize the workload associated with the RPA Program.

   The audit found the RPA Program operated in a manner that limited effective operational planning and the prioritization of staff's work. For example:

   • The current program model allowed funding opportunity requests to come from multiple stakeholders both internally (e.g. the Institutes) and externally (e.g. Government of Canada or partners) at any time of the year. These requests did not operate within a predictable cycle, and varied considerably in their design and complexity. Furthermore, there were few restrictions placed on the number of requests that could be made at any given time or the complexity of the competition design (funding opportunity).
   • The Program had not established priorities to assist in the triaging of the competition requests during periods of high demand. Stakeholder consultations indicated that the absence of clear priorities impaired CIHR's ability to decline requests for funding opportunity launches.
   • Management had not yet completed an analysis of the level of human resourcing needed to ensure there was enough staff with the appropriate skills to manage the workload volume.

   Not having the flexibility to prioritize the program delivery workload during periods of high demand could result in excessive overtime, absenteeism, and sick leave, and limit CIHR's ability to efficiently deliver on commitments within a reasonable timeframe.

Management's response to the audit observations and recommendations can be found in Appendix A.

1.6. Statement of Conformance

The Audit of RPA conforms with the Policy on Internal Audit as supported by the results of the quality assurance and improvement program.

Internal Audit thanks management and staff for their assistance and cooperation throughout the audit.

1.7. Disclosure of Potential Impairment to Independence and Objectivity

The audit included a review of the processes for corporate planning, reporting, and performance measurement, led and coordinated by the Results and Impact Unit (RIU) within the Performance and Accountability Branch (PAB), which fell under the responsibility of the Chief Audit and Evaluation Executive in his role as the Director General, PAB during the scope period. The audit criteria examined how results are used by the RPA Program, in accordance with the Policy on Results. Although the RIU may develop results frameworks and gather performance data, Program Officials within the Research Programs Portfolio retain responsibility for the review and application of results information in the strategic investment process. As a result, the potential for impairment, as reviewed by Audit Committee, was considered minimal.

Since the completion of audit work, the RIU has been moved to a different Portfolio within CIHR and is no longer the responsibility of the Chief Audit and Evaluation Executive.

Ian Raskin
Chief Audit and Evaluation Executive & Director General, Office of Audit and Evaluation
Canadian Institutes of Health Research

Michael J. Strong, MD, FRCPC, FAAN, FCAHS
President
Canadian Institutes of Health Research

2. Detailed Report

2.1. Background

The Canadian Institutes of Health Research (CIHR) is Canada's federal funding agency for health research. The objective of CIHR, as established in the CIHR Act, is "…to excel, according to internationally accepted standards of scientific excellence, in the creation of new knowledge and its translation into improved health for Canadians, more effective health services and products and a strengthened Canadian health care system."

Composed of an Ottawa-based headquarters and 13 "virtual" Institutes, CIHR collaborates with partners and researchers to support the discoveries and innovations that improve the health of Canadians and strengthen the health care system.

CIHR invests approximately $1 billion each year to support health research. This funding is divided into three Programs: Investigator-Initiated Research^{Footnote 1}, Training and Career Support^{Footnote 2}, and Research in Priority Areas (RPA). Figure 1 (below) illustrates the arrangement of the three programs in the Departmental Results Framework (DRF), the structure used by CIHR to report to Parliament on their activities and use of funds.

Figure 1: CIHR's Departmental Results Framework showing its three funding Programs^{Footnote 3}

Priority-driven research

The Research in Priority Areas Program funds priority-driven research (also known as "strategic" or "targeted").  As stated in its program description, the RPA Program "…provides funding for targeted grants and awards aimed at addressing priority areas". The research funded by this Program is intended to address pressing health issues that are of strategic importance to Canadians, identified through government priorities, Institute-driven activities, and national and international consultations with health research and health care stakeholders.

The budget for RPA in 2018/19 was $292M, divided into the following funding envelopes^{Footnote 4}:

• Institute Strategic Initiatives ($58M);
• Roadmap Accelerator Fund ($46M);
• Ring-Fenced Initiatives ($108M);
• Enabling Platforms and Other Programs ($30M); and
• Flow-Through funding ($50M).

See Appendix E for the list of RPA initiatives.

2.2. About the Audit

This report presents the results of the internal audit of Research in Priority Areas Program. The audit was undertaken by the Internal Audit function of CIHR between November 2017 and October 2018 as part of the 2015-2018 Risk-Based Audit Plan as approved by CIHR's Governing Council. The purpose of the audit was to review how the RPA Program was governed and how its activities were planned and delivered.

2.3. Significance of the audit

The RPA Program funds approximately one-third of CIHR's granting budget to deliver against section 4f of the CIHR Act, "addressing emerging health opportunities, threats and challenges and accelerating the discovery of cures and treatments and improvements to health care, prevention and wellness strategies". As such, the RPA Program is an important part of CIHR's business and instrumental in helping the agency meet its objectives under the CIHR Act.

The RPA Program has not been the subject of any previous audits.

2.4. Audit Objective and Scope

The objective of the audit was to provide assurance that:

1. Governance and risk management practices were established and operating effectively for the management of the RPA Program;
2. Operational resourcing and planning were adequate and effective to support program delivery.

Given that the RPA Program has never been audited, the scope of the engagement remained at a high level and focused on the broad design and delivery of the Program. The audit focused primarily on the activities of the 2016/17 fiscal year, though documents and activities outside of that period were also reviewed as appropriate.

2.5. Audit Criteria and Methodology

The criteria used in the audit were derived from the following documents: the CIHR Act; Office of the Comptroller General Core Management Controls: A Guide for Auditors; the Treasury Board Policy on Results and Policy on Financial Management; and the Financial Administration Act [ PDF (1.07 MB) - external link ]. Please refer to Appendix B for the detailed audit criteria. The criteria guided the audit fieldwork and formed the basis for the overall audit conclusions.

The audit used two principle methods to collect evidence and address the audit criteria:

• The examination of relevant documents including: Government of Canada policies, guidelines, and directives; CIHR corporate documents including the Departmental Results Framework and Program Inventory; CIHR internal processes and procedures; internal and external communications; committee materials (Terms of Reference, meeting minutes, materials, and records of decision); employee role profiles; and financial information (Delegations of Authority, budget dashboards, reports, and tracking spreadsheets).
• Interviews with management and staff of CIHR, including Presidents; Institute Assistant Directors; Directors General; Managers, and working-level staff.

2.6. Observations and Recommendations

The activities and initiatives that comprise the RPA Program (e.g. Institute Strategic Initiatives, Ring-fenced, Roadmap Accelerator Fund (RAF), Flow-Through Funding, EPOP) were grouped together for corporate reporting purposes as part of the transition from the Program Alignment Architecture to the Departmental Results Framework to meet the requirements of the Policy on Results (2016). While these activities and initiatives share a general purpose of supporting priority-driven research, the findings below demonstrate that critical aspects of effective management were missing to ensure the activities and initiatives were coordinated in a manner that would support a unified program.

Governance and Risk Management

Governance is the combination of processes and structures implemented by the Board or management in order to inform, direct, manage, and monitor the activities toward the achievement of its objectives^{Footnote 5}. In order for governance to be effective, senior management must articulate objectives, establish processes and structures to support the objectives, and then clarify key accountabilities, roles, and responsibilities to ensure expected results are met.

Risk Management is an approach to identify, assess, mitigate, communicate and monitor key risks that may impact the achievement of objectives^{Footnote 6}. Risk management is effective when the approach is formal, documented and systematic. Effective risk management is achievable when governance is established and operating effectively.

Planning and Resourcing

Operational planning is the establishment of activities that support operational, and ultimately strategic goals and includes the coordination of financial and human resources.^{Footnote 8}

Similar to CIHR's other Programs, the planning and delivery of the RPA Program is largely supported by two branches within the Research Programs Portfolio:

• Program Design and Delivery manages funding competitions, including application intake, processing, peer review, and post-award adjudication.
• Initiative Management and Institute Support supports priority-driven initiatives throughout their lifecycle including planning, competition design, monitoring, and termination.

Operational planning is effective when objectives are clear, activities have been prioritized, and the required resources have been analyzed, deemed sufficient and appropriately allocated.

2.7. Conclusion

The RPA Program is an important part of delivering on CIHR's mandate. Its significance stems from the research generated on health issues that are of strategic importance to Canadians. While structures exist to manage the RPA Program, the audit found limitations around its governance and planning which impact the extent to which the Program can meet its intended purpose. The areas of improvement identified in this report are structural in nature and will require a sustained effort on the part of management over the longer term. At the end of this audit, CIHR was embarking on a new strategic plan and a renewal of its overall governance structure. In responding to the recommendations outlined in this report, management should ensure its actions take important organizational changes into account.

2.8. Management Response to Recommendations

Management agrees with the four recommendations.

Context: CIHR is currently undergoing a strategic planning process to launch a new strategic plan by June 2020. This new plan will identify priority areas going forward that will shape our organization, including consideration of business and delivery processes that will contribute to the effective and efficient implementation of programs. Concurrently, CIHR is undertaking a governance renewal exercise.

Given that RPA is a key Program for delivering on strategic priorities, management will implement the recommendations once CIHR's new strategic priorities are identified.

Actions will include:

• A review of the financial framework in parallel to the Program's governance to determine continued suitability and appropriate alignment with CIHR's governance processes and structures, as determined through the CIHR governance renewal exercise;
• A review of the overarching Grants and Awards investment model, as well as the Program's design and structure to ensure the Program is optimally aligned with the strategic priorities and resources;
• Determine the appropriate long-term budget envelopes and level of funding necessary to address the Program's priorities and allow for a sustainable investment strategy;
• Develop principles, parameters, tools, and an approach to better measure the cost of delivering our programs, and manage the volume of program delivery requests in order to align with strategic priorities and be effectively managed; and
• Effective program implementation and oversight through reporting to appropriate governance structures.

Once CIHR's strategic priorities are articulated, as early as spring 2020, management will discuss appropriate due dates with the Chief Audit Executive to ensure the implementation of the audit recommendations.

Management understands that there will be follow-up on the recommendations and status updates will be provided to CIHR's Audit Committee as needed.

Appendix A: Color coding of recommendations

It is good practice that all recommendations be cleared within an 18 month window of the approval of an audit report. To that end, the Internal Audit unit uses a color-coded system to assist management with the prioritization of remedial actions.

The color-coding, outlined below, takes into consideration the urgency with which recommendations should be addressed, the complexity of the recommendation and/or the underlying issues or causes for concern, and the level of risk to which the Agency is exposed as a result of the issue identified.

Appendix B: Relevant Audit Criteria

The criteria used for assessing the audit objectives were adapted from the following documents:

• Office of the Comptroller General Core Management Controls: A Guide for Auditors
• Treasury Board Policy on Results
• CIHR Act
• Financial Administration Act [ PDF (1.07 MB) - external link ]
• Treasury Board Policy on Financial Management

The following are the key criteria used for the audit report. They have been simplified for reporting purposes.

Appendix C: Glossary

Canada 150 Research Chairs (C150): An open, investigator-initiated one-time grant provided to institutions to attract internationally recognized scholars to Canada. See the Canada 150 Research Chairs website.

Canada Excellence Research Chairs (CERC): An open, investigator-initiated grant program that supports world-renowned researchers and universities to build Canada's reputation as a global leader in research innovation. See the Canada Excellence Research Chairs website.

Canada First Research Excellence Fund (CFREF): An open grant program that supports efforts by postsecondary institutions to become global research leaders. See the Canada First Research Excellence Fund website.

Canada Graduate Scholarship (CGS): Open training awards that provide a stipend to Masters and Doctoral students while they are completing their studies.

Canada Research Chairs (CRC): An open, investigator-initiated grant program that supports accomplished and promising researchers with a goal of making Canada a top research and development country. See the Canada Research Chairs website.

Enabling Platforms and Other Programs (EPOP): A budget envelope that supports priority research initiatives that are not contained within any other budget envelope. The Canadian Longitudinal Study on Aging is an example of a Platform within the EPOP envelope.

Flow-Through Funding: Program activities are managed under their own set of Terms and Conditions, sometimes by other Government of Canada departments or agencies. The Network Centers of Excellence (discontinued in 2019) was an example of flow-through funding.

Foundation Grants Program: An open, investigator-initiated grant funding program that supported high-cost, long-duration research programs for established researchers.

Institute-Strategic Initiatives (ISI): Funding initiatives sponsored, funded, and led by one or more of CIHR's 13 Institutes out of their $8.5M annual granting budget, addressing priorities within the mandate area of the specific Institute(s).

Major Initiatives: Significant granting investments managed by CIHR, each with a specific purpose and long life cycle. The Strategy on Patient-Oriented Research is an example of a Major Initiative.

Program: The Policy on Results defines a program as "Individual or groups of services, activities or combinations thereof that are managed together within the department and focus on a specific set of outputs, outcomes or service levels". Within CIHR the term "Program" has no uniform definition, and can be used refer to entities as diverse as single funding opportunities, through to the highest-level of reporting within CIHR's DRF. For this report, the term Program refers to the DRF-level entity.

Project Grants Program: An open, investigator-initiated grant funding program that supported research projects for new and established researchers.

Ring-Fenced Initiatives: Federally targeted, program/initiative-specific funds which generally do not have their own stand-alone Terms and Conditions and therefore fall within CIHR's umbrella Terms and Conditions for Research Grants; a commonly-understood term used across government without a formal definition. The Anti-Microbial Resistance Initiatives are an example of ring-fenced funding.

Roadmap II: CIHR's strategic plan from 2014/15-2018/19.

Roadmap Accelerator Fund (RAF): A process and budgetary envelope used from 2015/16 to 2018/19 to support multi-institute high-impact initiatives based on Roadmap II, drawing upon Institute granting budgets. The Pathways to Health Equity for Aboriginal Peoples is an example of a RAF-funded initiative.

Training and Career Support: A general set of open programs that provide stipends to research trainees (e.g. Masters, Doctoral, and Post-Doctoral Fellows) and salary funding for full-time researchers.

Vanier Canada Graduate Scholarships (Vanier/VCGS): An open research program that provides stipends to promising Doctoral students designed to increase the supply of highly-qualified research personnel in Canada. See the Vanier Canada Graduate Scholarships website.

Appendix D: RPA Program multi-year budget commitments (As of April, 2018)

Appendix E: RPA Initiatives (as of April, 2018)

Institute-Strategic Initiatives ($58M)

The diversity and number of Institute-funded Initiatives is extensive and cannot be fully listed.

Roadmap Accelerator Fund ($46M)

• Dementia Research Strategy
• Community-Based Primary Health Care
• e-Health Innovations
• Environment and Health
• Epigenetic
• Inflammation in Chronic Disease
• Pathways to Health Equity for Aboriginal Peoples
• Personalized Health
• Personalized Medicine
• Healthy & Productive Work
• Indigenous Healthy Life Trajectories Initiative/HeLTI – Indigenous
• Canada-China

Ring-Fenced Initiatives ($108M)

• Antimicrobial Resistance Initiative
• Canadian Breast Cancer Initiative
• Canadian Consortium on Neurodegeneration in Aging
• Climate Change
• Drug Safety and Effectiveness Network
• Emerging Health Threats Fund
• Gairdner
• Hepatitis C
• HIV/AIDS Initiative
• Network Environments for Indigenous Health Research
• Canadian Drugs and Substances Strategy
• Pandemic Preparedness
• Strategy for Patient Orientated Research

Enabling Platforms and Other Programs ($30M)

Platforms

• Canada Research Data Centre Network
• Canadian Council on Animal Care
• Canadian Light Source
• Canadian Longitudinal Study on Aging

Other Programs

• Best Brains Travel Exchange
• Canada-European Commission Flagship
• Collaborative Health Research Projects
• Golf Leaf Prizes
• Data Initiatives
• Ethics
• Gairdner
• Global Alliance on Chronic Disease
• Human Frontier Science Program
• President's Fund
• Secretariat on Responsible Conduct of Research

Sun-Setting Programs

• Clinician Training Program
• GSK Chairs
• Knowledge Translation
• Regional Partnership Program
• Small Health Organization Partnerships Program

The EPOP program also includes some unallocated funding whose value varies from $1.8M to -$5M depending on the fiscal year.

Flow-Through Funding ($50M)

• Business-Led Network Centers of Excellence
• Centres of Excellence for Commercialization and Research
• College and Community Innovation
• Industrial Research Chairs for Colleges
• Institute Support Grant
• Networks Of Centres of Excellence

Source: 2018-04-30 Financial Dashboard presented to Science Council at its 133rd meeting, June 13-14, 2018.

Date modified:

2020-02-21