QI Series Part 3 | Differentiating Quality Improvement and Clinical Epidemiology

In Education & Quality Improvement by Shawn MondouxLeave a Comment

How do Quality Improvement methods differ from classical clinical research?

This article is the last in a three-part series on Quality Improvement (QI) in health care. The first article, “What is Quality Improvement?”, defined this field of study and provided a brief history of its origins.  The second article defined “quality” and briefly described tools used to achieve improvement. This article seeks to differentiate QI methods and goals from those encountered in clinical epidemiology.

Underlying Principles

In the delivery of medical care, there is a need to know which therapies, interventions, and approaches yield improved clinical outcomes for patients. Classical clinical research seeks to both answer and quantify the effect of an intervention on patient outcomes. In order to ascribe any outcome differences to the intervention, control and intervention groups are established and the variability in the baseline characteristics of each group is minimized through a variety of means.  This way, we can be more certain that the measured difference in effect is due to the application of an intervention. When it is difficult to perfectly match groups, statistical methods are applied to null the effect of variances attributed to elements other than the intervention. If the intervention is the most crucial ingredient, then it means that we assume that the setting in which the intervention was provided and the process through which it was provided have little or nothing to do with the measured outcome.

For those of you who work as clinicians and have seen first-hand the variability in the care provided to patients (the ‘art’ of medicine!), this lack of emphasis on the setting and processes probably doesn’t resonate with you!

In QI projects, we accept rather than attempt to control the real-life variations in the environment when we conduct tests of change. In other words, we simply apply an intervention to the clinical environment and measure the outcomes before, during, and after. The data tools used to conduct these analyses, as well as to determine their significance, are detailed in the following section. QI is interested in HOW we deliver care rather than WHAT care we deliver. It acknowledges that the process of providing care has as much to do with patient outcomes as the care itself. In keeping with this, it explicitly chooses to measure the effect of the practice environment (setting or “culture”) on the outcome by not controlling for it.  

Clinical epidemiology seeks to establish causation in spite of the practice environment while QI seeks to establish causation due to change in the environment.

The nature of the changes sought are also different. QI seeks to conduct small changes to a care process, collect data on this change, further modify the process and repeat this cycle until the desired change is achieved.  This is called the Plan-Do-Study-Act (PDSA) rapid-cycle methodology. It is an incremental and intensely iterative process where the interventions may change several times during the study period. This is in sharp contrast to clinical research in which there is little room for modifying data collection, primary outcomes, or the elements of the intervention once the project is underway. In clinical research, all elements must remain constant until the trial is complete and all patients are enrolled. Due to this, clinical research tends to focus on far-reaching and highly impactful clinical problems. QI is busy conducting small iterative changes. Both fundamentally use the diagnostic process. Often, time and financial investments in QI are significantly smaller.

Data Analysis

The data analysis tools used in clinical epidemiology and QI are fundamentally different. QI uses a process called Statistical Process Control (SPC), a tool first widely used as part of the manufacturing process. These tools take a fundamentally different approach at determining the effect of a particular intervention.

The most common statistical approach used in research is to first estimate the magnitude of the effect of the intervention on outcomes. Based on this estimate, sample sizes would be calculated in order to produce results in which the effect demonstrated would be attributable to chance in less than 5% of scenarios (P < 0.05). This also means that 1 in 20 results produced in a single paper could be due to chance alone.

In SPC, we assume that the process of manufacturing a product (whether this might be a ball-bearing or the time to admission from consult for a medicine patient) is relatively stable over time. Although there will be variation due to inherently variable processes involved in producing a ball-bearing or admitting a patient, this variation is somewhat predictable over time.

So in order to determine significance in QI, we run a stable process for a significant amount of time (or get good retrospective data). With this, we can then draw statistical lines around the graph called “control limits”. Within these limits, variation is simply due to chance. But after a change in process, we look to see what the data shows. If there is a noticeable trend in the data that occurs outside the “control limits”, then our process modification has demonstrated effect. Review these videos on Control Charts for more information (Video 1, Video 2)

 

 

How These Fields Overlap

Although QI is an emerging field and uses effective and novel techniques to demonstrate effects, clinical epidemiology approaches and research remain the standard of evidence in medicine. Although some may view these field as being at odds with one another, our view is that they are very complementary.

The complexities of high-quality clinical epidemiology research are dissuading too many health care practitioners from furthering the field of health care delivery. Not only are trials very expensive, they require enrolment of many subjects to demonstrate effect. This often means multi-centre trials in which the eventual outcome is unknown. QI provides a way in which researchers and funding agencies may seek to establish a better return on investment (ROI) for funding and research time.

We would encourage that broader clinical epidemiology questions be tested as QI interventions at the outset. This allows for many advantages:

  • Refinement of an idea of change and testing it in the clinical environment
  • Demonstrate effect on initially smaller sample sizes
  • Monitor for balancing measures that may be adverse outcomes in a larger trial
  • Allows for slow scaling up and proving of local effects prior to larger investments are made

There are clearly some important areas where QI cannot supplant its well established cousin. Research in which there exists significant equipoise will require a very large trial, especially when the estimated effect is small or incremental. Good examples of this are trials comparing clinical outcomes in patients receiving Normal Saline vs. Ringer’s Lactate or trials comparing novel antiplatelet agents for repeat myocardial infarction or stent restenosis/thrombosis to current regimens such as plavix or ticagrelor. Despite the impressive nature of these project, we must ask if the expenditure of such time and energy is best directed towards studies with potentially small impact.

What if the way we give therapies is more important that what we’re actually giving?

We often worry as health-care providers if we’ve prescribed the ‘best’ antibiotic or the ‘right’ treatment plan, but we more rarely worry about whether the patient has the ability to follow-up with their providers, understand their discharge instructions, or afford the treatment plan. The ‘best’ treatment won’t matter if it is not used, and so we may want to focus less on evidence-based medicine and more so on quality improvement initiatives that help tailor the care to our individual patients.

Ultimately, we must ask ourselves some important questions about the care we deliver. Maybe the most important of these is to ask whether the WAY in which we provide care is as responsible for outcomes as WHAT we give as part of that same care. Maybe the next goal in health care is to hone our processes and refine how we “manufacture” patient care. Once this is largely underway, we may realize that new and important treatment questions will arise. These questions will require larger and larger patient populations to answer, thus making the case of clinical epidemiology once again. As always, the combination of these modalities likely represent the application of an elaborate PDSA cycle where the application of the most effective tools will yield the most beneficial results.

Take Home Points

  • HOW we deliver care may be more important than WHAT care we deliver
  • QI uses different statistical methodologies than clinical epidemiology to determine effect. The goal remains to establish which variation is due to chance and which is due to the intervention, but without controlling for changes in the environment
  • There remains an important and significant opportunity for synergy between the fields of clinical epidemiology and quality improvement

This post was uploaded by Jung-In Choi.

Shawn Mondoux

Dr. Shawn Mondoux is an emergency physician at Hamilton Health Sciences (HHS) and faculty at McMaster University in Hamilton, Ontario. He obtained a masters of Quality Improvement and Patients Safety (QIPS) at the University of Toronto and serves as QI advisor to all projects within the ED. He has a strong interest in clinical QI work as well as the education of PGME learners in QI principles.

Lucas Chartier

Dr. Lucas Chartier is an emergency physician and Deputy Medical Director for the University Health Network (UHN) emergency department (ED). He is also UHN's inaugural Medical Director of Quality and Safety, the ED Lead for the Toronto Central Local Health Integration Network (TC LHIN) and the Chair of the Quality Improvement and Patient Safety (QIPS) Committee of the Canadian Association of Emergency Physicians. With training at McGill (MD CM), University of Toronto (FRCPC) and Harvard (MPH), he now tries to spread the QIPS gospel in all the ways possible!