“… experience from industrial disasters suggests that it's harder than it looks to develop safety measures that don't bite back.”—T. Hartford (1)

An increasing focus on the quality of care has led to a proliferation of process and outcome measures to allow for the measurement of quality. For the year 2012, the Center for Medicare and Medicaid Services has proposed a total of 367 quality measures (2). At face value, all of these quality measures appear reasonable and assess endpoints of definite clinical importance. However, many of these measures can be seen as approaching quality from a fairly univariate and static view. The measures largely assume that deficiencies in quality can be addressed in a fairly straightforward and direct manner without untoward effects. However, the clinical reality is that many of these measures have complex and dynamic interrelationships with other measures of quality such as patient access to care, appropriateness of care, and surveillance rates of diagnostic testing. As such, efforts to improve these measures may lead to unintended consequences. Our understanding of the complete spectrum of unintended consequences remains in its early stages, but some common themes have become increasingly evident.

Quality measures that focus on mortality can lead to the phenomenon of risk avoidance where patients at high risk of adverse outcomes are denied beneficial clinical care (3). With public reporting of percutaneous coronary intervention–related mortality in Massachusetts, the proportion of patients treated with cardiogenic shock fell from 2.28% to 1.29% over the 3-year period from 2003 to 2005 (3). Paradoxically, these highest-risk patients with cardiogenic shock are those that derive the most absolute benefit from treatment (4). Thus, an improvement in the numerical mortality rate within a quality measure may not actually reflect quality of care improvements but may rather be due to denial of needed care to the highest risk patients.

Quality measures that focus on an outcome that is dependent on the level of surveillance may adversely impact providers who are more attentive to diagnosing disease as opposed to those who are less diligent (5). The diagnosis of deep venous thrombosis is directly related to the utilization rate of vascular ultrasound. Thus, differences in the occurrence of deep venous thrombosis may reflect utilization patterns of vascular ultrasound rather than quality differences. In centers with high utilization rates of vascular ultrasound, clinicians may improve their rates of deep venous thrombosis diagnoses by reducing their utilization rates. However, this endeavor can lead to underdiagnosis and undertreatment of patients with resultant harm.

Quality measures can increase the focus on clinical documentation rather than true quality. In patients undergoing primary percutaneous intervention for ST-segment elevation myocardial infarction (STEMI), documentation of a nonsystem reason for delay can allow for exclusion from public reporting. In fact, the proportion of patients with a door-to-balloon time >90 min and a documented reason for delay allowing exclusion from public reporting has increased from 3.7% to 8.1% between the years 2005 and 2010 (6). Some of this increase may reflect more accurate documentation practices, but at its most extreme, clinicians may “game” the system to avoid potential negative professional, reputational, or financial consequences associated with measure reporting (7).

Quality measures may also be proposed and implemented before a complete understanding of their potential solutions. Hospital readmissions after heart failure and myocardial infarction are certainly common and costly; however, our understanding of how clinicians can reduce readmission rates without limiting the provision of appropriate care remains largely unknown (8). Therefore, costly and burdensome efforts to improve the measure may be implemented without any definite evidence of their true impact.

Given the increasing awareness of the risk of unintended consequences of quality improvement, the study by Patel et al. (9) in this issue of the Journal is both timely and of great clinical importance. For more than 2 decades, the provision of pre-hospital electrocardiogram (ECG) for early identification of STEMI has been recommended to reduce the time to treatment (10). Most reports have consistently shown a 15- to 20-min reduction in door-to-balloon time with pre-hospital ECG (11). However, before this study, it was unclear if widespread integration of pre-hospital ECG would lead to increased ambulance scene time, negating the benefits of reduced door-to-balloon times and ultimately causing an increase in overall time to treatment (12). Furthermore, only a very small minority (<5%) of patients with chest pain actually have ST-segment elevation on ECG (13). Therefore, a very large population of patients has to be exposed to this pre-hospital intervention, although only a very small number will actually benefit. For these reasons, a pre-hospital ECG protocol could cause a delay in ambulance scene time when implemented across the very large population of patients presenting with chest pain.

Focusing their attention on the experience within the city of San Diego, these investigators analyzed a very large (N = 21,742) real-life population before and the after the implementation of a pre-hospital ECG program to allow for early diagnosis of STEMI. Of 9,631 patients with chest pain of suspected cardiac origin screened with the pre-hospital ECG protocol, only 303 patients had STEMI identified vividly, thus illustrating the very large population of patients who had to be treated with this intervention. The overall impact of the pre-hospital ECG protocol on scene time (min:s) (19:10 vs. 19:28, p = 0.002) and transport time (13:16 vs. 13:28, p = 0.007) was extremely modest and almost certainly not clinically significant. Furthermore, there appeared to be a small ∼3-min improvement in scene-to-hospital time for STEMI patients, although this finding was confounded by the lack of a STEMI control group. This work conclusively alleviates the concerns of scene delays associated with pre-hospital ECG implementation in a very large real-life population.

The work by this group should encourage other investigators to systematically analyze quality improvement projects for unintended consequences and risks. As a whole, we have assumed a much higher level of certainty of benefit of our quality measures and quality improvement efforts. In fact, the reality is that many of our quality improvement efforts, no matter how well intentioned and designed, will have no or a deleterious impact on quality. Therefore, the implementation of a quality improvement project should be accompanied by a well-designed protocol of clinical follow-up and statistical analysis to assure that a true improvement has occurred. Furthermore, as shown by the work of Patel et al., the results of these quality improvement efforts should be ultimately published for the benefit of all.

The intense focus on improving clinical quality will only increase in the future. Of the many quality measures being considered, some will lead to true clinical benefit to patients, some will have no impact, and some will lead to patient harm. The challenge and opportunity for clinicians today is to identify which of these many quality measures will lead to a true and durable improvement in the quality of patient care.

⁎Editorials published in the Journal of the American College of Cardiology reflect the views of the authors and do not necessarily represent the views of JACC or the American College of Cardiology.