Evaluation of the Institute of Nutrition, Metabolism and Diabetes Research (INMD)

Report of INMD Evaluation Panel

May 2017

At the Canadian Institutes of Health Research (CIHR), we know that research has the power to change lives. As Canada's health research investment agency, we collaborate with partners and researchers to support the discoveries and innovations that improve our health and strengthen our health care system.

Canadian Institutes of Health Research
160 Elgin Street, 9th Floor
Address Locator 4809A
Ottawa, Ontario K1A 0W9

This publication was produced by the Canadian Institutes of Health Research. The views expressed herein do not necessarily reflect those of the Canadian Institutes of Health Research.

INMD Evaluation Panel:

Chair:

Dr. Garret A. FitzGerald, Professor of Medicine and Systems Pharmacology and Translational Therapeutics, University of Pennsylvania, USA

Panel Members:

• Dr. Edith Feskens, Professor, Nutrition and Health over the Life course, Wageningen University, Netherlands
• Dr. Helen Raybould, Professor, Department of Anatomy, Physiology and Cell Biology, University of California, USA
• Dr. Minna Woo, Professor of Medicine and Medical Biophysics, University of Toronto, Canada
• Dr. Vincent Poitout, Professor, Department of Medicine, Université de Montréal, Canada

Thanks to all participants in this evaluation – interview participants and survey respondents – and to Goss Gilroy Inc. and Circum Network Inc. for data collection and analysis.

Additional thanks to the INMD CIHR Evaluation Team: David Peckham, Michael Goodyer, Doaa Saddek, Abigail Forson, Christopher Manuel and Carole Chow.

And special thanks to Dr. Philip M. Sherman, Scientific Director, INMD and Mary-Jo Makarchuk, Assistant Director, INMD.

For more information and to obtain copies, please contact: Evaluation@cihr-irsc.gc.ca.

Table of Contents

1. Executive Summary
2. Overview of the Evaluation
   • Institute of Nutrition, Metabolism and Diabetes
   • Evaluation Objectives
3. Observations and Recommendations
   • Should the INMD be Amended, Merged or Terminated?
     • Context
     • Scientific and Funding Landscape
     • Panel Observations
     • Recommendations
   • Should INMD’s mandate be changed?
     • Context
     • Panel Observations
     • Recommendations
     • Strategic Considerations
   • Observations for the Next Scientific Director
     • Context
     • Panel Observations
   • Other Considerations/Opportunities
4. Evaluation Key Findings
   • Evaluation Objectives
     • Relevance
     • Transformative Impact
     • Convener and Catalyst
     • Operational Effectiveness
5. References
6. Appendices
   • Appendix 1: The INMD Evaluation Panel Members’ Affiliations and Conflict of Interest Declaration
   • Appendix 2: Overview of Data Sources and Methods
   • Appendix 3: Key Figures and Tables
   • Appendix 4: INMD Partnerships

I. Executive Summary

The evaluation of the Institute of Nutrition, Metabolism and Diabetes (INMD) was undertaken by the Canadian Institutes of Health Research (CIHR) as part of the review of the mandate and performance of CIHR Institutes by CIHR’s Governing Council (GC) outlined in the CIHR Act. The evaluation assessed the relevance and performance of INMD to inform GC decisions regarding the role and functioning of the Institute. The evaluation was conducted by the CIHR Evaluation Unit and a team of external evaluation professionals and overseen by a panel of experts in INMD’s mandate areas who reviewed and interpreted the findings and made the final recommendations. The recommendations and observations of the Panel are summarized below.

1. Should the INMD be amended, merged or terminated?

   The rise in obesity rates in Canada has resulted in an increase in the occurrence of health problems (i.e., diabetes and cardiovascular diseases) and higher health care costs in the future. The economic costs associated with diabetes alone are significant. The Panel, therefore, believes that it is crucial that CIHR continue to play a role in supporting innovative research in the areas of nutrition, metabolism and diabetes and that the INMD by virtue of its strong leadership, credible presence and knowledge of the field is well-placed to play this role. The Panel strongly recommends that the INMD not be amended, merged or terminated. The Panel also strongly recommends that the INMD continue as a separate institute within the CIHR.

2. Should the INMD’s mandate be changed?

   The Panel highlighted that INMD’s mandate is appropriate and is reflective of the reality of both public health and scientific opportunities. Given the breadth of the INMD’s mandate and the limited resources currently available, the Panel advises that more discretion be granted to the SD to select and focus on certain aspects of the current mandate. The Panel recommends that the Institute continue with the current mandate. In light of the broadness of INMD’s mandate, the Panel recommends that more of CIHR’s priority-driven research resources be assigned to INMD.

3. Observations for the Next Scientific Director

   As the current Scientific Director of INMD will complete his second term in December 2017, the Panel is not concluding on the renewal decision but does provide some observations to GC regarding the recruitment of the next SD. The current INMD SD has demonstrated strong skills in engaging the research community and is well-regarded by both researchers and stakeholders. The latest changes to Institutes’ budgets that the SDs can directly control could pose a challenge in attracting a new outstanding leader for INMD. The Panel concludes that the job description for the recruitment of INMD’s next SD should be revised to attract an excellent active scientist. To assist the new SD in addressing the broad domain of expertise covered by INMD mandate, the Panel suggests establishing an Institute-specific advisory board to bring domain expertise to the new SD.

II. Overview of the Evaluation

Institute of Nutrition, Metabolism and Diabetes

As one of the 13 CIHR Institutes, the Institute of Nutrition, Metabolism and Diabetes (INMD) has a vision is to position Canada as a leader in the creation of knowledge through health research in relation to diet, digestion, excretion and metabolism that benefits all Canadians and the global community.^{Footnote 1} INMD’s mandate is to support research and to enhance health in relation to diet, digestion, excretion, and metabolism; and to address causes, prevention, screening, diagnosis, treatment, support systems, and palliation for a wide range of conditions and problems associated with hormone, digestive system, kidney, and liver function.

Within its mandate, INMD supports capacity building through partnerships with voluntary health organizations and with health professional associations. In addition, INMD continues to emphasize knowledge translation and exchange, and ethics by expanding boundaries with both established and new partners to co-design and co-fund transformational initiatives.

Evaluation Objectives

The evaluation of the INMD was conducted by CIHR as part of the ongoing assessment of the mandate and performance of the 13 CIHR Institutes. The evaluation assessed the relevance of the mandate of INMD and the performance of the Institute in order to inform decisions regarding the role and functioning of the Institute and the recruitment of the next Scientific Director (SD). The aims of the evaluation are to provide the Governing Council (GC) with valid and reliable findings to inform decisions regarding:

1. Should the INMD be amended, merged or terminated?
2. Should the INMD’s mandate be changed?

As the current SD of INMD will complete his second term in December 2017, the Panel is not concluding on the renewal decision but does provide some observations and considerations to GC regarding the recruitment of the next SD should the Institute be maintained.

The evaluation was overseen by the INMD Evaluation Panel (hereafter referred to as the Panel) comprised of experts in the INMD mandate areas and conducted by the CIHR Evaluation Unit and external evaluation professionals. The names and affiliations of the Panel members are listed in Appendix 1.

The evaluation examined the Institute’s operations within the period 2000-2016, with a specific focus on the period under the leadership of the current SD, Dr. Philip Sherman.^{Footnote 2}

The evaluation of INMD was informed by multiple lines of evidence, including review of documents and data, interviews with INMD and CIHR staff and partners, survey of researchers, and a study of the influence of the publications of CIHR funded research in INMD mandate within and beyond academia. The methods and data sources used are outlined in Appendix 2 and key figures are presented in Appendix 3. While each line of evidence has limitations, there is convergence among them so as to produce key findings.

III. Observations and Recommendations

Should the INMD be Amended, Merged or Terminated?

Context

Over the last decade, obesity rates around the world have increased, with an estimated one billion adults overweight and at least 300 million clinically obese.^{Footnote 3}Canada has seen a steady rise in the number of adults and youth that are overweight or obese and currently more than one-in-four Canadian adults (25.4%) find themselves in this category.^{Footnote 4}According to Statistics Canada about 14 million adults were either overweight or obese in 2014, an increase from 13 million in 2010 (8.6% increase over four years).^{Footnote 5} This rise in obesity rates implies increases in the occurrence of health problems (i.e., diabetes and cardiovascular diseases) and higher health care costs in the future.

Diabetes is among the four most common chronic diseases and the seventh leading cause of death in Canada. The economic costs associated with diabetes alone are significant. It is estimated that the total annual cost of diabetes to the Canadian economy is $9 billion. Approximately 11 million Canadians are living with prediabetes, a condition that significantly increases the risk of several chronic diseases, including type 2 diabetes and cardiovascular disease. Type 2 diabetes is one of the fastest growing diseases, with more than 60,000 new cases reported yearly. Indigenous Peoples are three to five times more likely to develop type 2 diabetes than non-Indigenous Canadians. While the condition is ordinarily seen in older adults, incidence rates of type 2 diabetes among Indigenous children are increasing.

In terms of other chronic diseases, 1 in 10 Canadians is at risk for kidney disease. The two leading causes of kidney disease are diabetes (35%) and renal vascular disease (16%).^{Footnote 6} There is also a relatively high prevalence of digestive disease in Canada. According to the OECD, Canada ranks 7th among surveyed countries as having the highest rates of digestive disease.^{Footnote 7}

Scientific and Funding Landscape

Socioeconomic conditions, culture, as well as genes and environment influence the development of chronic disease. The link between obesity and chronic diseases, mainly diabetes (obesity is a major modifiable risk factor for the development of type 2 diabetes), and cardiovascular disease, is evident. Accordingly, obesity related research that identifies solution-focused interventions at the clinical, policy and population level and emphasizes priority populations (e.g. children, Indigenous Peoples, severely obese individuals) and knowledge translation would help in improving prevention approaches and enhancing weight management strategies.

The link between food and health, including the development and prevention of chronic diseases, is also an established one. A report by the Office of Nutrition Policy and Promotion of Health Canada highlighted the association between features of the food environment and diet-related outcomes, notably:

• Evidence of increased geographic access to non-nutritious food sources among people living in areas of low socioeconomic status pointing to a need for additional research on food environments.
• A relationship between food environment and eating habits, with healthier eating habits among Canadians who select foods based on nutrition labels and those with higher levels of food skills.^{Footnote 8}

The Office of Nutrition Policy and Promotion recognizes that there are knowledge gaps around the determinants of eating behaviors and the current interventions to support healthy eating, stating that related policies and programs require thorough research and evaluation.^{Footnote 9}

Generally, the scientific and research landscape have evolved and there is a trend toward patient-focused research, with an emphasis on patient engagement in the research setting. Furthermore, there has been an increase in research undertaken with a view to specific communities, such as First Nations communities. In addition, there is an increased demand and efforts toward fostering knowledge translation.

Panel Observations

Achieving its mandate

CIHR’s budget has been effectively flat for approximately the last 10 years, so it is declining substantially in real terms. This is exacerbated by the need of health researchers to make many purchases of research materials in US funds.

The Government of Canada’s 2017 Budget did not provide new, untargeted funding for the three federal research funding agencies: CIHR, the Natural Sciences and Engineering Research Council of Canada (NSERC) and the Social Sciences and Humanities Research Council of Canada (SSHRC).^{Footnote 10} The 2017 Budget established the position of Chief Science Advisor and related secretariat, funded the creation of approximately 25 Canada 150 Research Chairs and a number of investments to simplify and target support to Canadian innovators, including: Innovation Clusters and Networks, Impact Canada Fund, Strategic Innovation Fund, Venture Capital Catalyst Initiative, and Innovation Canada.

In terms of funding for CIHR, the 2017 Budget proposes funds for Health Canada, the Public Health Agency of Canada and CIHR to support measures associated with the Canadian Drugs and Substance Strategy ($100M over five years) and National Action Plan on Climate Change Adaptation ($47M over five years). The Panel noted that it is unclear what fraction of the funding might be available for collaborative research with researchers in INMD’s mandate.

In June 2016, the Minister of Science charged an independent advisory panel, chaired by Dr. David Naylor, with conducting a review of federal support for fundamental science in Canada. The final report of Canada’s Fundamental Science Review, released on April 10, 2017, stresses that significant reinvestment in the federal research ecosystem needed over a more predictable and better planned multi-year horizon as well as improved coordination and collaborations between the three federal granting agencies (CIHR, NSERC and SSHRC) and the Canada Foundation for Innovation (CFI).^{Footnote 11} The 2017 Budget document indicates that the federal government will finalize its response to the advisory panel’s recommendations before making any new investments in the federal granting agencies, which could mean Budget 2018.

Until 2014-15, each of the 13 CIHR Institutes received a strategic research budget of $8.6M. As a result of the Institute Modernization, in 2015-16, half of each Institutes’ strategic research budgets ($4.3M per year) was invested in CIHR’s Roadmap Accelerator Fund (RAF) to support multi-Institute and multidisciplinary initiatives align with CIHR’s research priorities patterned along the lines of CIHR’s existing signature and strategic Initiatives. The remaining half of the budget remains under the control of Institutes to direct toward Institute-specific initiatives.^{Footnote 12}

In addition to spending out of INMD budget, investment in INMD mandate research areas could come from the budget of any other CIHR source.^{Footnote 13} The total CIHR investment in the INMD mandate increased steadily between 2000-01 and 2005-06, from $78M to $168M, during a period when CIHR’s overall budget increased steadily, then remained stable until 2010-11 before decreased slightly over the following five years to reach $145M in 2015-16. The decline may be partially explained by the conclusion of two Networks of Centers of Excellence (NCEs) – the Canadian Obesity Network and the Advanced Food and Materials Network – funded in part by CIHR and relevant to INMD’s mandate research areas. For more information about CIHR investments in INMD’s mandate by research areas,^{Footnote 14} see Figure A (Appendix 3).

The Panel noted that offering each of the 13 CIHR Institutes an equal budget is not reflective of the reality of public health, scientific opportunity, and disparities in health conditions, such as diabetes and obesity in the case of INMD. The INMD has a broad mandate, but receives insufficient funding to support its relevant work.

There have also been concerns about recent changes related to Institute support and oversight, including:

• The Institute Advisory Boards (IABs) model was restructured from 13 Institute-specific IAB model to a new model of five IABs aligned with the strategic directions and research priorities of CIHR’s five-year strategic plan, Health Research Roadmap II. The new model aims to boost collaboration across Institutes and within CIHR’s multitude of stakeholder communities by bringing together a wide range of perspectives within each IAB; however, the model is perceived to have diluted the domain expertise required by the SDs. The Institute-specific IABs were important in helping the Institutes to develop and focus on their strategic priorities. Under the new model, SDs may find it necessary to consult with “shadow” or informal advisory boards to obtain advice from and communicate with their Institutes’ research community.
• The change in staff allocation from Ottawa-based Institute staff (OBIS), who were dedicated personnel at CIHR’s central office providing service to one Institute, to Integrated Institute Teams (IIT), which is a matrix resource structure providing specific support across Institutes, is believed to have undermined the importance of corporate memory, continuity and staff loyalty.
• The elimination of “priority announcements”, which made it difficult for Institutes to fund meritorious investigator initiated grants “below the pay line” to fill strategic gaps. Currently, the funding cut is below the 10th percentile.^{Footnote 15} The Panel believes that SDs should be provided with more flexibility to reach below the formal pay line in order to be able to prioritize and fund excellent investigator initiated research and to address strategic gaps in funding.

Recommendations

Should INMD’s mandate be changed?

Context

As outlined in the CIHR Act, the objective of the CIHR is:

to excel, according to internationally accepted standards of scientific excellence, in the creation of new knowledge and its translation into improved health for Canadians, more effective health services and products and a strengthened Canadian health care system…^{Footnote 16}

Among the many activities to achieve its objective, CIHR is responsible for “encouraging innovation, facilitating the commercialization of health research in Canada and promoting economic development through health research in Canada.” And, as divisions within CIHR, the Institutes are expected to contribute to the achievement of CIHR’s overall objective within their mandate through a number of activities, including: “work in collaboration with the provinces to advance health research and to promote the dissemination and application of new research knowledge to improve health and health services.”

In practice, partnership with the private sector and commercialization has been proven to be a challenge for INMD as well as for CIHR and the Institutes more broadly. The problem, however, goes beyond CIHR and its Institutes as it is more related to Canada’s comparatively weak performance in the areas of commercialization and business innovation. Many factors contribute to this including: innovation is not a priority in the taxation system, a dearth of venture capital in Canada and difficulties building bridges between researchers and the private sector within the health sector. It was noted that commercialization of CIHR funded discoveries usually happens outside Canada; more broadly, many inventions start in Canada, but are not supported all the way to the stage of commercialization in Canada.

Despite these challenges, INMD has demonstrated some impressive new areas of activity, including exploratory microbiome science in inflammatory bowel disease (IBD) in Asian immigrants, and some legacy discoveries, such as the discovery of PCSK9 that was later scientifically recognized for its therapeutic promise and commercially developed in the US. More recent current discoveries, such as glucagon like peptide 1 receptor agonists, have attracted commercial partners and have been supported in part by INMD. Little of this experience, however, has been harvested by CIHR to design strategies to engage the private sector. INMD, therefore, currently has no intellectual property reach/prospect/potential that would contribute to its sustainability.

INMD is committed to and supports capacity building through partnerships with other health organizations and professional associations, as well as by providing support to new investigators. As of 2015, INMD has hosted three New Investigator meetings and co-hosted two other such meetings with other CIHR Institutes. The purpose is to help first-time faculty members in the first five years of their appointment to obtain peer reviewed grant funding, to facilitate collaboration among new investigators, to support knowledge translation activities and partnerships, and to provide new investigators with the opportunity to receive mentoring from established Canadian researchers.

INMD works with the Canadian Society of Nephrology and the Kidney Foundation of Canada to increase capacity in the area of nephrology. The Kidney Research Scientist Core Education and National Training Program (KRESCENT) have awarded funding to doctoral and post-doctoral students, and new investigators undertaking research in areas related to kidney health and disease. The main goal of the program is to support and encourage transdisciplinary research. To date, 8 Allied Health Doctoral awards, have been awarded, along with 32 New Investigator awards and 40 postdoctoral fellowships.

In partnership with the Canadian Association of Gastroenterology and others INMD provided funding ($17.4 million) in support of gastroenterological research over an eight year period (2000-2008). Funding included Fellowships Awards (131), Operating Grants (22), and Career Transition Awards (7).

Panel Observations

The Panel remarked that there is an increasing prevalence of obesity and diabetes, an increasing recognition of the importance of metabolic pathways and metabolomics in parsing the origins broadly of disease, and an asymmetric distribution of diabetes and obesity amongst ethnic groups, to the particular disadvantage of Indigenous Peoples of Canada. For these reasons, the Panel believes that the current INMD mandate is appropriate and is reflective of the reality of both: public health and scientific opportunities.

In light of the prevalence and burden of disease within INMD’s mandate, the Panel favors shifting support among the Institutes within CIHR. Presently CIHR divides resources evenly among the 13 Institutes; however, this takes a particularly static view of science and health burden, in other words, of both opportunity and challenge. In the case of INMD, for the convergent reasons outlined above, the Panel feels that more of CIHR’s priority-driven research resources should be assigned to INMD. This is both reflective of the reality of public health and scientific opportunity but also would contribute to making the directorship of the Institute a more attractive proposition.

The Panel noted that commercialization is important yet it is a challenging space for a research institute such as INMD to occupy. Even so, INMD has adapted to respond to emerging therapeutic opportunities, but its scientific and outreach mandates are broad and broadening in the face of a declining budget. The Panel, therefore, concluded that expecting INMD to participate in fostering commercialization, an extremely expensive endeavor, is essentially unrealistic. Furthermore, the Panel noted that it is challenging to assess INMD’s performance in terms of knowledge translation and commercialization given it is a broader CIHR objective.

Recommendations

Strategic Considerations

In light of the Government of Canada’s increased focus on and investment in innovation, the Panel suggests that CIHR, including the INMD, seek advice and consider strategic approaches to engagement with the private sector. For example, an advisory body composed of academics, who have realized commercial success, and leaders in the biotechnology, pharmaceutical and venture capital sectors could advise on how CIHR might leverage the new budgetary allocation to innovation and how it might harvest those public monies assigned to the private sector with a mandate to interact with academia. The advisory board members could possibly be drawn broadly from North America, not restricted to Canada.

Moving forward, the Panel sees the need for a more systematic approach to assess the impact of the investment and activities of CIHR and its Institutes on Canadian health and health systems. The Panel emphasized that CIHR and INMD are supported by government funding; therefore, the assessment of the impact of CIHR’s and the Institute’s investment and activities in its mandate on the health of Canadians is crucial. It is important to note that the CIHR and its Institutes operate in a complicated landscape that makes assessment and attribution difficult: scientific discoveries can take a long time to yield tangible results and there is an inherent difficulty in determining how much of the impact of these discoveries should be attributed to CIHR funding. That said, in addition to assessing the impact of the Institute’s investment through the analysis of the publications resulting from the funded research, capacity building and case studies of impact, the Panel suggests engaging health economists to help identify indicators for the assessment of the economic impact of the Institute’s investments, in terms of how the dollars invested in INMD’s mandate were leveraged: how much economic growth has this investment brought and how funding has attracted other funders.

Observations for the Next Scientific Director

Context

There is widespread appreciation in the INMD research and stakeholder community for the foresight, strategy, energy, effort and soft skills of INMD’s current Scientific Director (SD). In particular, the current SD has successfully engaged in a variety of activities in order to identify, connect and engage with multiple organizations, and this was reflected in strong collaborative partnerships across disciplines. In addition, the current SD has taken on the role of ensuring that the Institute is providing opportunities for the mentorship of new investigators.

The Panel expressed that the new INMD leader would need to be an excellent, active scientist with the following characteristics:

• National and global recognition within the scientific community with high-level administrative experience;
• An ability to listen to and harness the respect of the broad constituency of researchers and stakeholders within INMD’s mandate;
• Scientific and strategic vision and an ability to prioritize and put into practice; and
• Strong communication skills to be able to communicate the mission and importance of the INMD to a broad constituency of stakeholders.

In terms of the context into which the INMD’s new SD is being recruited, there are concerns about a funding crisis in biomedical research in Canada in general as well as the need to rebuild trust between CIHR and the health research community in light of the implementation issues with the recent reforms to the open program and peer review. In addition, the changes resulting from Institute Modernization, most notably the restructuring of Institutes’ budgets that the SDs can directly control, could pose a challenge in attracting an outstanding leader for INMD as a successor of the current Scientific Director. 

Panel Observations

• An aggregate pay increase compared to their present compensation;
• Substantial ongoing support to assuage the hit to their own research program while in post;
• A 2-3 year package of research support to smooth their re-entry into the competitive marketplace on completion of their tenure; and
• A written assurance of their role and importance at the decision making level of CIHR beyond INMD.

The INMD has a strikingly broad constituency, which could be impossible for one leader to cover in terms of domain expertise. One option that may make the position more attractive is to ensure that candidates are aware of the opportunities under the ISG to hire scientific associate director positions as Institute-based employees to broaden the capability of leadership. This could also diversify beyond the dependence on the health and wellbeing of a single individual. The Panel also remarked that given the breadth of the Institute’s mandate, it is important that the Institute have an Institute-specific advisory board to bring domain expertise to the new Scientific Director.

Looking forward, the Panel highlighted that there is currently a funding crisis in biomedical research in Canada and that until this is fixed, morale, recruitment and accomplishment relevant to the health and wealth of the nation will be undermined. Given the political uncertainty around research funding in several advanced economies in today’s world, the Panel remarked that CIHR should also consider the implications of opening the recruitment of the SD to include international applicants in order to benefit from international expertise. This will therefore have implications on the design of an appealing package to attract prominent international scientists.

Other Considerations/Opportunities

The asymmetric burden of diabetes and obesity in Indigenous Peoples represents a global resource for discovery of genes relevant to these diseases. As such, CIHR and INMD should explore future strategic opportunities to collaborate in funding such research with National Institute for Health Research (NIHR) in the UK, the National Institutes of Health (NIH) in the US, and other funding bodies (e.g. the Wellcome Trust, the European Research Council).

Building on the recently announced $30M CIHR and Juvenile Diabetes Research Foundation (JDRF) partnership to fund clinical research to improve the treatment of type 1 diabetes and accelerate the search for a cure, the Panel sees opportunities to develop flexibility by alignment with Diabetes Canada and JDRF to streamline the review process, to shorten the review cycle, augment research funding and to enhance outreach. While recognizing potential complexities, such as in the domain of intellectual property, similar opportunities should be sought in other disease domains and with other charities, foundations and provincial funding agencies.

The Panel notes a particular opportunity with Diabetes Canada to maximize investment in research by integrating the peer review and selection procedures in diabetes with CIHR through the alignment of funds and re-investment of saved resources. On the other hand, Diabetes Canada is likely to be more effective than CIHR and INMD in patient outreach and advocacy and might take the lead in a collaborative effort in that aspect of their activities.

IV. Evaluation Key Findings

Evaluation Objectives

CIHR has conducted this evaluation of the INMD as part of the suite of rolling evaluations of all CIHR Institutes. The aims of the evaluation are to:

1. Provide GC with valid and reliable findings to inform decisions regarding whether the Institute’s role and functioning should be amended, merged, or terminated;
2. Provide CIHR management with valid, insightful, and useful findings regarding the ongoing institute relevance and performance; and
3. Inform decisions regarding the transition of the Institute and recruitment of the next SD.

The evaluation drew on multiple lines of evidence (Appendix 2), including qualitative and quantitative data sources. It collected data from the range of INMD stakeholders and beneficiaries, including researchers, funding partners, and research users, and integrated these with administrative data on expenditures and publications related to the INMD mandates, using a framework that is common to all Institute evaluations. While each line of evidence has limitations, there is convergence among them so as to produce key findings. Overall, we are reasonably confident that the results presented provide an accurate portrait of the INMD’s relevance and performance.

The evaluation was conducted by the CIHR Evaluation Unit and a team of external evaluation professionals and overseen by a panel of experts in INMD’s mandate areas who reviewed and interpreted the findings and made the final recommendations.

Relevance

Ongoing relevance of support to INMD research

The expanded areas of research that have emerged in the wider environment include an increased focus on food-based research, advances in obesity research, as it concerns liver health and chronic disease were also cited, a turn towards the “omics” such as genomics, genetics, and epigenetics, was stated to be a significant area of change and the presence of newer technologies as well as open/big data.

There is a need for an institute that fosters research on food and health to improve nutritional status at the population level, compresses morbidity in relation to chronic disease, and provides support for evidence-informed policies and practice. A research focus on prevention strategies to promote positive environmental changes is warranted.

INMD is operating within a larger context characterized by health realities which speak to the need for an Institute that addresses areas pertaining to nutrition, metabolism, and diabetes. That said, the Institute’s mandate is overwhelmingly challenging, because it covers large global health issues (e.g., obesity and diabetes). INMD fosters an environment that links researchers, scientists, community groups, individuals from around the world in efforts to address these important health concerns. The Institute plays an integral role in developing, promoting and maintaining a research environment that addresses issues related to nutrition, metabolism and diabetes. Research priorities in these areas are supported financially by CIHR more broadly and by INMD specifically. Figure B (Appendix 3) depicts CIHR’s investment in INMD mandate by research priority areas. As Figure B shows, from 2011-12 to 2015-16 the amount invested by CIHR in Food and Health ($120M) represents 42% of total CIHR investment in INMD mandate, compared to 25% in Continuum of Care ($53M); 18% in Environment, Genes and Chronic Disease ($53M); and 15% in Obesity and Healthy Body Weight: Seeking Solutions ($42 M). From 2011-12 to 2015-16, CIHR’s annual investment in Food and Health increased gradually and was always the highest compared to investment in the other research priorities. CIHR’s annual investment in Environment, Genes and Chronic Disease also increased over the same 5-year period compared to investments in the remaining research priorities.

The evaluation found that the relevance of the mandate has been growing more important with the increasing levels of obesity and burden of chronic disease, largely diabetes. The Institute’s mandate covers important areas in the context of health of Canadians: nutrition, metabolism and diabetes all contribute significantly to general health and chronic conditions across the lifespan; diabetes is a huge burden on the health care system in Canada and worldwide. INMD strives diligently to support the research community in the areas of priority, and this is reportedly valued by stakeholder, particularly by non-profit organizations and the public sector.

The institute has been very inclusive, actively seeking out partnership opportunities and reaching out to academia, interest groups, the community, patients and citizens. INMD fosters effective partnerships with all stakeholder groups via reliable collaboration, scientific integrity, and research excellence. The INMD is seen as having been successful in terms of bridging different areas. Stakeholders, in particular non-profit organizations, reported being grateful for their partnership with INMD. Some external stakeholders, mainly non-profit organizations, also stated that they view the Institute as a vehicle for their organization to connect with CIHR.

Appropriateness of the current INMD Mandate and Changes to Institute Name

The INMD mandate is generally perceived to be appropriate and aligned with the strategic direction of CIHR overall. The Institute’s mandate bridges several related organ systems and these are interrelated with disease conditions. All research areas covered by the Institute’s mandate fit well together because of the overlap between them. In this sense the Institute’s mandate represents a continuum not just a theme. This is reflected by the Institute name: metabolism is very general and is the common link, many interests can be included under its umbrella; nutrition is a common background factor; and diabetes is quite specific, but has effect on many organs. Thus, it is by nature that INMD has a broad and diverse mandate while maintaining a clear vision and relevant strategic priorities within that mandate. INMD’s mandate has also been good about tapping into the necessary areas, including environmental impacts on chronic disease. The Institute also demonstrates a clear understanding of where it fits in relation to the other institutes within the various parts of the mandate.

Despite the investment in multiple mandate research areas, some areas (e.g., kidney, liver, metabolism, and Crohn’s, colitis and IBD) remain under-invested in when compared with obesity, nutrition and diabetes, which have been receiving a bigger share of the Institue’s budget. However, the Institute is seen as very inclusive of all relevant research areas.

Overall, the evidence suggested that INMD’s mandate is appropriate and that the work being conducted under the mandate is very relevant and is functioning well. The ability of the mandate to follow environmental changes and developments in technology, however, was flagged as an important factor for continued success, and given the current emphasis on mental health and chronic disease, a suggestion was made to consider how mental health might fit within the scope of INMD’s mandate.

Transformative Impact

Support to Innovative Research and Advancing Knowledge

The INMD is committed to supporting innovative research and advancing knowledge. The Institute makes an effort to identify and develop new initiatives by engaging a wide range of stakeholders from a variety of communities in consultation, workshops, meetings and partner forums. Input from organizations regarding strategic priorities is emphasized by the SD. The majority of researchers surveyed in the course of the INMD evaluation indicated that the research funding they received from CIHR in areas related to INMD mandate contributed very much (60%) or somewhat (36%) to supporting the creation and development of innovative research ideas.

Transformative and impactful research is evident in relation to the Institute’s mandate areas. The lives of patients are directly affected by its research outcomes, as is seen in relation to diabetes, for example, INMD funded research led to a change in a diabetes drug monograph, removing a former contraindication.^{Footnote 17} Additionally, funding supported a Canadian-led international drug trial that resulted in new findings about the effects of medications on glucose levels.^{Footnote 18} Canadian researchers have played a leading role in the areas of glucagon-like peptide 1 receptor agonists and novel potential sources of insulin. In areas related to healthy eating and nutrition, the Institute has produced work on sodium intake that has been commended by the federal government. Impactful research includes that of Dr. Mary L’Abbé, who developed the ‘Big Life Sodium Calculator’, which helps consumers calculate their sodium intake.^{Footnote 19} To date it has been used by a half million consumers worldwide. Some of the research is funded with different CIHR Institutes and targets specific populations. For example, as highlighted by key informant interviewees and corroborated by the SD’s presentation to the Panel, research being conducted on the environment and autoimmune diseases in South Asian communities, and tailoring individual therapy based on genetics and environment.

In terms of impact at the level of the wider research community, bibliometric analysis of INMD publications shows that compared to other CIHR Institutes, INMDs publication rates are higher, which reflects the broad mandate. Two thirds of these publications are of moderate to strong influence^{Footnote 20}, with 10% of publications reviewed showing evidence of having been applied. In another instance, bibliometric analysis of published research in the field of gastroenterology showed that in terms of both, average relative citations and average relative impact factor, grant and award recipients outperformed the global average and also outperformed the Canadian average for the five most published subfields.^{Footnote 21} In addition, INMD enables knowledge translation through all the partnerships it has created and fostered.

The SD is supportive of transformative research and is open to innovations, engages various stakeholders, is committed to client oriented research, and is an active researcher himself. He recognizes the importance of collaboration and is viewed as contributing to the establishment and sustainment of linkages/collaborations. He meets with the community and reaches out to charities and partners regarding research projects and seeks out collaborative research opportunities in an ongoing manner.

Contributions to Building Capacity of the Health Research Enterprise

INMD engages in a number of capacity building activities, including Institute-facilitated capacity building events and other investments to maintain and strengthen research capacity in mandate areas. This is in part fostered by the SD, who is viewed as supportive of young investigators and invested in ensuring they have the proper mentorship and training to help them become independent investigators.

From 2010-11 until 2015-16, INMD spent between 12% and 25% percent of its budget on capacity building, including investments in catalyst/pilot programs, training grants and awards, and development grants (Figure C, Appendix 3). As of 2015-16, 14% of total CIHR funded direct trainees^{Footnote 22} (Figure D, Appendix 3) and 15% of indirect trainees^{Footnote 23} (Figure E, Appendix 3) were funded under the INMD mandate. Additionally, capacity building occurs through the funding researchers receive in relation to INMD mandate areas; 89% of the researchers’ surveyed stated that their funding contributed to supporting the training of researchers and/or practitioners in their research. Similarly, 93% stated that their funding contributed to supporting students and/or trainees.

Partnerships have proven to be a significant contributor to INMD mandate over time as depicted in Figure F and Figure G (Appendix 3). The SD is seen as someone who builds collaborative environments. For instance, a partnership with the Canadian Association of Gastroenterology and others resulted in the funding of 131 fellowships awards, 22 operating grants, and 7 career transition awards. New researchers are supported as well. A partnership with the Kidney Foundation of Canada and the Canadian Society of Nephrology has led to the awarding of eight Allied Health Doctoral awards, along with 32 New Investigator awards and 40 postdoctoral fellowships. Further support for new researchers is evidenced by INMD’s hosting of three New Investigator meetings and co-hosting two with other CIHR Institutes, which contributes to the professional development and mentoring of junior faculty working in the INMD mandate areas.

The 2011 International Review found that INMD has built capacity in the area of Obesity and Healthy Body Weight through funding 10 training awards for doctoral and post-doctoral students, and new investigators, and one Strategic Training Initiative in Health Research (STIHR), which was renewed in 2009 for five years.

Contributions to Achieving Broader Health, Economic and Social Impact

INMD works towards improving health services and the health of Canadians by engaging in Knowledge Translation (KT) at multiple levels and across all areas of the mandate. The strong partnerships and programs with the private, public and not-for-profit sectors allow for knowledge sharing that in evidence-based decision-making and impacts. Indeed, there are opportunities in this regard; the partnerships INMD has established can be leveraged in support of knowledge translation.

The provision of funding to Canadian Kidney Knowledge Translation Generation Network (CANN-NET) helps identify knowledge gaps to be addressed by clinical trials, which in turn has the potential to impact the health of Canadians through new treatments and disease management techniques. Knowledge stemming from the Programmatic Grants in Food and Health will serve to inform decision-making, nutritional practice and guidelines, and contribute to food and public health nutrition policies. Furthermore, 76% of researchers surveyed indicated that the INMD funding they received contributed to supporting KT from their research findings towards improving health services in Canada and 87% indicated this was the case for improving the health of Canadians. Impacts on a global scale are evident as well. For example, one researcher developed a tool that was validated in over 30 countries, which helps doctors assess their patients’ risk of developing chronic kidney disease later in life.

Partnerships that promote KT also contribute to improving the health of Canadians, as well health services available to Canadians. Most recently, INMD and the Canadian Obesity Network (CON) established a strategic research initiative to further understanding of severe obesity, which will serve to inform practice, guidelines, and overall patient/individual care. INMD participates in conferences and offers workshops that promote KT. The “Developing a Research Agenda to Support Sodium Reduction in Canada” workshop highlighted the theme of “transforming knowledge to action” and input from workshop participants was sought as to how this might be undertaken. Recommendations were included in the Sodium Reduction Strategy for Canada (2010).

Scientific Directors help shape the direction of KT; it is a very influential position regardless of an institute’s budget, with great opportunity and ability to influence policy at large. The SD is viewed as being supportive of the advancement of knowledge and KT into policy and practice, which has the potential to impact Canadians directly. This includes working with Health Canada and Public Health Agency of Canada to address health research gaps related to dietary sodium intakes and levels of sugar consumption.

Commercialization, which relates to economic impact, is not viewed as an area where INMD has contributed strongly, although some perceive commercialization as better suited to being under the purview of CIHR instead of INMD. A small minority of researchers surveyed felt that INMD research funding contributes to the commercialization of findings and most funded research is fundamental, far removed from commercialization. This is in line with the Institute’s current investment priorities. Nonetheless, the Institutes are working towards a commercialization strategy that is in the early stages of inception. INMD’s funding was found to play an important role in supporting the creation and development of innovative ideas in research.

Convener and Catalyst

Contribution of Scientific Leadership to the Convener-Catalyst Role

The evaluation found that INMD’s SD was very active in building and maintaining partnerships and collaborations. He has engaged in a variety of activities to identify, connect and engage with multiple organizations, and has participated in events (i.e., workshops, meetings and conferences) where he spoke to INMD’s strategic priorities and to those of CIHR more generally. The SD fostered an interest in CIHR initiatives, such as the Strategy for Patient-Oriented Research (SPOR), encouraging other leaders and partners to participate. He also brought other organizations to meetings and workshops, opening up more opportunities for collaboration and contribution to ideas and formation of priorities.

The INMD’s SD is regarded by stakeholders as a leading source of information in the Institute’s mandate areas and that he is well respected in the wider community, both nationally and internationally. The SD is seen by stakeholders to add credibility to the Institute and will defer to experts when relevant. SDs hold the power to help shape the direction of knowledge translation. This is particularly relevant to an institute like INMD that covers large health issues of global, as well as local relevance.

INMD makes use of communication products and ensures the Institute has a media presence, which helps to establish its profile and visibility among all stakeholders, including the public. Amongst the communications vehicles utilized by the Institute is a monthly newsletter. INMD also has both a web and social media presence that is demonstrative of the Institute’s profile and visibility. Between 2009 and 2015, INMD staff hosted, co-hosted, sponsored or was a guest, presenter or panelist in at least 61 events.

In terms of media presence, INMD was featured in at least 18 news releases or announcements posted on CNW Newswire or on the Government of Canada newsroom. At least 5 media pitches and experts alerts related to INMD were issued during the same time period and furthermore, INMD, including its SD, were directly mentioned in at least 7 media stories.

Partnering to Achieve CIHR and Institute Objectives

The evidence suggested that INMD has been an effective convener and catalyst. The evaluation identified several benefits from the INMD’s partnerships and collaborations with other entities such as expanded networks and the ability to do more with funding. Increasing capacity within specific research areas was offered as a benefit. It was found that partnerships and funding encourage researchers to continue to pursue research in certain areas, some of which may not be as commonly pursued when compared to others. In terms of impact on the health of Canadians, it was found that the research produced as part of partnerships is typically directed at producing positive outcomes for the health of Canadians (e.g., prevention, disease management).

A key success of the Institute has been the creation and fostering of partnerships with other organizations. Accordingly, INMD has a number of partnerships with federal/provincial government, research institutes/centres, not-for-profits (e.g., foundations, health charities), and pharmaceutical/medical companies. A sample of the partners is found in Appendix 4.

The linkages and partnerships that INMD has established with other organizations take a variety of forms. Partner organizations include other CIHR Institutes, government agencies and departments, and not for profit organizations (e.g., health charities). Partnerships are primarily with national organizations, with some collaboration at the international level. While some partnerships were focused on strategic priority areas (e.g., food and health, obesity), others were chronic disease and/or systems-oriented (e.g., diabetes, kidney) that crossed the strategic priority areas.

Operational Effectiveness

Overall, INMD is viewed as operating effectively, while wider reforms at CIHR are viewed less favourably in terms of the resulting impacts.

The Institute is recognized as fostering successful working relationships and the levels of effectiveness, training, and organization are viewed positively. However, additional support and targeted funding was identified as potentially useful. Despite budgetary constraint, INMD’s pursuit of its strategic and operational plans has led to progress and the implementation of initiatives in various areas. In contrast, the implementation of reforms at CIHR are perceived as having been undertaken in a sub-optimal manner and are viewed as not particularly beneficial or useful to the Institutes. Additionally, the reforms are seen as resulting in reduced resources, such as funding. INMD has been successful in demonstrating good work, plus, as noted above, others have also demonstrated good work in its mandate areas – this needs to be leveraged.

INMD receives $1M annually from its Institute Support Grant (ISG) from CIHR. On average, from 2009-10 until 2014-15, INMD spent 70-75% of the funds available under the ISG (see Appendix 3, Figure H). Approximately 75% of the funds were spent on Institute Operations and the remainder was used for Institute Strategic Development. Because INMD does not spend all of its ISG funding annually, the balance is transferred to the next fiscal year and therefore the total annual funds available in this category exceed the $1M allotment to the Institute every year.

V. References

Canadian Institutes of Health Research, Institute of Nutrition, Metabolism and Diabetes (2011). Internal Assessment for 2011 International Review.

Canadian Task Force on Preventive Health Care (2014). Screening, Prevention and Treatment of Overweight/Obesity in Adult Populations.

CIHR Institute of Nutrition, Metabolism and Diabetes Refreshed Strategic Plan, 2015–2018.

CIHR Act.

Budget 2017: Building a Strong Middle Class.

Health Canada, Office of Nutrition Policy and Promotion. (2013). A look at food skills in Canada.

Health Canada, Office of Nutrition Policy and Promotion (2006). Areas of work.

Investing in Canada’s Future: Strengthening the Foundations of Canadian Research. Canada’s Fundamental Science Review (2017).

Organisation for Economic Co-Operation and Development (OECD) (2014). OECD Health Statistics 2014: How does Canada compare?.

Sherman et al. (2013). “Evaluation of funding gastroenterology research in Canada illustrates the beneficial role of partnership”, Canadian Journal of Gastroenterology, 27(12). 717-720.

The Kidney Foundation of Canada (2013). Facing the Facts.

VI. Appendices

Appendix 1: The INMD Evaluation Panel Members’ Affiliations and Conflict of Interest Declaration

Chair:

• Garret A. FitzGerald, Professor of Medicine and Systems Pharmacology and Translational Therapeutics, University of Pennsylvania, USA

Panel Members:

• Edith Feskens, Professor, Nutrition and Health over the Life course, Wageningen University, Netherlands
• Helen Raybould, Professor, Department of Anatomy, Physiology and Cell Biology, University of California, USA
• Minna Woo, Professor of Medicine and Medical Biophysics, University of Toronto, Canada
• Vincent Poitout, Professor, Department of Medicine, Université de Montréal, Canada

Appendix 2: Overview of Data Sources and Methods

Note: These data sources were complemented by telephone consultations, conducted by the INMD Evaluation Panel during the two-day face-to-face meeting, with seven members of the INMD research community who had not been previously interviewed (although some may have completed the researcher survey).

Appendix 3: Key Figures and Tables

• Figure A: CIHR Investment in INMD Mandate by Research Areas
• Figure B: CIHR Investment in INMD Mandate by 2009-2014 Research Priorities
• Figure C: Investment in Capacity Building out of INMD Budget
• Figure D: Percentage of Direct Trainees Supported under INMD Mandate
• Figure E: Percentage of Indirect Trainees Supported under INMD Mandate
• Figure F: Partners’ Contributions to INMD Mandate
• Figure G: Leverage Ratio of Partnership to CIHR Investment in INMD Mandate
• Figure H: Utilization of Institute Support Grant (ISG) Budget

Figure A: CIHR Investment in INMD Mandate by Research Areas

• It is worth noting that from 2000-01 to 2008-09 CIHR spending was higher in endocrinology compared to all other INMD areas, peaking at $51M in 2008-09, this was followed by a rapid decline to 8M in 2015-16 and may be attributed to a change in the method of validation that occurred with the new Scientific Director’s leadership (most notably, new decision rules to exclude of sex-hormones and cancers).

Figure B: CIHR Investment in INMD Mandate by 2009-2014 Research Priorities

• Note that the investment in continuum of care in 2015-16 may be partially explained by the phased approach to implementation of the INMD Strategic Plan based on budget availability despite it not being a priority in the 2015-2018 strategic plan.

Figure C: Investment in Capacity Building out of INMD Budget

• The percentage of spending on capacity building out of INMD budget, dropped from 53% in 2001-02 to 22% in 2004-05, then stabilized over the following three years, with an average 21% annual investment before gradually dropping to 12% in 2009-10. The percentage of spending on capacity building out of INMD budget then remained stable with an annual average investment of 14% until 2013-14 when it increased to 25% before it dropped to reach 16% in 2015-16.

Figure D: Percentage of Direct Trainees Funded under INMD Mandate

• The annual percentage of Direct Trainees funded under INMD mandate in relation to the total number of CIHR funded Direct Trainees increased between from 18% in 2000-01 to 23% in 2004-05 before decreasing gradually to reach 14% in 2015-16.

Figure E: Percentage of Indirect Trainees Supported under INMD Mandate

• The annual percentage of Indirect Trainees funded under INMD mandate in relation to the total number of CIHR funded Indirect Trainees gradually decreased from 28% in 2001-02 to 15% in 2015-16 with slight fluctuations from year to year.

Figure F: Partners’ Contributions to INMD Mandate

• The contributions of academic institutions account for the highest percentage of total partners’ contributions to INMD mandate in the first periods (60%), followed by private sector partners (17%) and non-specified partners (15%). In the second and third periods contributions from not-for-profit organizations accounted for the highest percentage of total partners’ contributions (28% and 26%, respectively), followed by the contributions of federal partners (24% and 25%, for second and third periods respectively).
• Contributions from international partners account for significantly higher percentage of total partners’ contributions in the third period (19%) compared to the previous two periods (2% and 3%). This is largely due to INMD’s participation in the Joint Programme Initiative A Healthy Diet for a Healthy Life (JPI HDHL) with European Union countries. Contributions from municipal/provincial partners account for a higher percentage of total partners contributions in the last period (10%) compared to the previous two periods (0% and 2%), largely due to INMD partnerships to support the Bariatric Care Team Grants.
• For the three periods combined (2000-01 to 2015-16), contributions of academic institutions account for the highest percentage (25%) out of total partners’ contributions to INMD mandate, followed by contributions of not-for-profit organizations (20%).
• As for the first 2 periods of time, the composition of the unspecified category is not identified/specified in CIHR grants and awards database. This category could include any of the Federal, Municipal/Provincial, International, Academic, Not-for Profit, Private Sector partner groups. Accordingly, this data identification problem could possibly affect the accuracy of the interpretation of the percentage of contribution of each partner group highlighted above.

Figure G: Leverage Ratio of Partnership to CIHR Investment in INMD Mandate

Leverage ratio of Partnership = Partners’ contributions to Institute's Mandate ÷ CIHR Investment in Institute’s Mandate

• The leverage ratio of partnership to CIHR investment shows how much was invested in INMD mandate through partner contributions for every dollar of CIHR investment in the INMD’s mandate areas.
• The leverage ratio was highest in 2001-02 (.14), with partner contributions totaling $14M relative to CIHR’s investment of $100M largely due to the small institute budgets at that time (≈$2M) and the “pent up” demand of health charities to partner with CIHR because it was a new organization. This was followed by 2015-16 (ratio = .13), when partner contributions totaled $19M relative to CIHR’s investment of $145M, and then by 2008-09 and 2009-10 (ratio = .12), when partner annual contributions averaged $21MM, relative to CIHR’s average annual investment of $172M.
• The leverage ratio was lowest in 2011-12, when partner contributions totaled $9M relative to CIHR’s investment of $164M.

Figure H: Utilization of Institute Support Grant (ISG) Budget

• INMD has been spending approximately 70%-75% of funds available under ISG. The lowest was 61% in 2012-13 and the highest was 89% in 2015-16. Each year the unspent balance out of the ISG is transferred to the next fiscal year, which is why the total Annual Funds Available exceeds the $1M every year as seen above.

Appendix 4: INMD Partnerships

Date modified:

2017-08-15