Section 3.2: Identifying the Knowledge-to-Action Gaps

Alison Kitson
Templeton College, University of Oxford, Oxford, UK

Sharon E. Straus
Li Ka Shing Knowledge Institute, St. Michael’s Hospital, and Department of Medicine, University of Toronto,
Toronto, ON, Canada

What is a “gap”?

  • There are many examples of underuse and overuse of evidence in decision making and clinical care
  • 1/3 patients do not get treatments of proven effectiveness
  • 1/4 patients get care that is not needed or potentially harmful
  • Up to 3/4 of patients don’t get the information they need for decision making
  • Up to 1/2 of physicians don’t get the evidence they need for decision making
  • Policy makers often don’t use evidence from systematic reviews to inform policies, despite its availability

Measuring a ‘gap’

  • This process starts with identifying outcomes of interest
  • Quality indicators can be considered
    • These are measures that monitor, assess and improve quality of care and organizational functions that affect patient outcomes

Quality indicators

Quality indicators should include:
descriptive statement

  • list of necessary data elements necessary for constructing and reporting the measure
  • detailed specifications on how data elements are to be collected
  • population on whom the indicator is constructed
  • timing of data collection and reporting
  • analytic models used to construct model
  • format in which the results will be presented
  • evidence in support of its use
  • should be valid, reliable and feasible

Little agreement on optimal quality indicators across countries

Developing quality indicators

  • Requires careful consideration of best available evidence  such as that from systematic reviews and use of an appropriate rating process
    • RAND Health advocates use of  modified Delphi method to develop quality indicators:
      • Rounds of anonymous ratings on a risk-benefit scale and in-person discussions between rounds
      • Includes all relevant stakeholders
      • Testing of indicator in real practice settings
  • Could also consider using indicators developed from high-quality evidence-based guidelines

Which quality indicators?

  • Establish process for selecting which gaps to target
  • Consider burden of disease (morbidity, mortality, quality of life, cost)
  • Involve all relevant stakeholders in discussion

How can we measure gaps?

  • Needs assessments
    • systematic processes determine size and nature of gap between current and more desirable knowledge, skills, attitudes, behaviours and outcomes
    • Strategies depends on purpose of the assessment, type of data, and resources that are available
    • Can be subjective or objectively measured
  • Classification of needs:
    • Felt needs (what people say they need)
    • Expressed needs (expressed through action)
    • Normative needs (defined by experts)
    • Comparative needs (group comparisons)
  • When assessing needs, must also consider and specify the stakeholder ‘s perspective i.e. the population, provider organizations or health care providers

Measuring the gap at the population level

  • Can be done using epidemiological data from administrative databases
    • E.g. diagnosis, procedures, laboratory investigations, billing information
    • Databases can range from regional to national
    • Examples include Ontario MOH LTC database of OHIP billings, CIHI data on hospital admissions

Limitations of admin databases

  • All strategies for needs assessments have limitations
  • Administrative databases have limitations including:
    • They were not developed for research so may not contain all the information (such as socioeconomic status)
    • Coding may be incomplete for co morbid conditions etc
    • Only records event for which there are codes
    • May not include the entire population (e.g. prescription database may only contain information on patients who are aged 65 and older because their prescriptions are paid for by the health plan)

Clinical databases

  • These include registries of patients who have undergone specific procedures/have certain diagnoses
    • Examples include databases of people who received a CABG in a health region or who participated in colorectal cancer screening
  • They may complement administrative data because they may have additional data such as information on patient comorbidities
  • Limitations of these databases include inaccurate information and challenges integrating across databases

Measuring the gap at the organization level

  • Multiple strategies can be used including
    • Data abstraction from paper health records and electronic health records
    • Extraction of data from organisational databases (such as those used for hospital accreditation)
  • Chart abstraction should use valid abstraction tools
  • Challenges in use of health records include lack of detail, illegibility (often a challenge in paper records), inability to easily capture data from electronic records

Measuring the gap at the care provider level

  • Can include:
    • Chart audits (paper or electronic)
    • Observation (using standardized patients or videorecording)
    • Competency assessment (using knowledge questionnaires or clinical vignettes)
    • Reflective practice (clinicians highlight learning opportunities or portfolios of their own experience)
    • Surveys, interviews and focus groups (may not accurately reflect true gaps in practice)

Why do gaps exist?

  • Most individuals are efficient processors of routine tasks
  • Do not concentrate on repetitive tasks once they master them
  • Skills for performing repetitive tasks are repressed, allowing us to pay attention to other things
  • Consequently, tasks individuals do most frequently, they think about the least
  • They do not have ways of evaluating the impact of these routine tasks
  • This may result in a “drift” in performance
  • People stop searching for improved conditions when they find a satisfactory result
  • Individuals adapt to slowly changing environments leading them to tolerate extreme variations without becoming aware they are doing so.
  • Opportunities for new ideas or introduction of new knowledge are not recognized
  • Problems become critical situations and catastrophes can result as the system drifts away from its ability to get feedback on routine tasks
  • Most people operate on “automatic pilot”
  • Most individuals will unconsciously adapt to worsening conditions
  • Active strategies must be put into place to counter these natural trends

Future research

  • Testing how routine data can stimulate identification of gaps in service delivery, monitoring changes to practice, and introducing new practices in a reliable and valid way
  • Understanding how local teams can become more autonomous and self-directing to keep vigilant over routine matters
  • Being clearer about how we identify knowledge-to-action gaps in the health care system

Summary

  • Identifying gaps in care is the starting point for knowledge implementation
  • When people are given more freedom to get involved, they more actively engage in finding creative solutions to routine problems and implementing them
Date modified: