The past decade has been characterized by increased scrutiny of outcomes of surgical and percutaneous cardiovascular procedures. Heightened interest in procedural outcomes has led to the development of regional, state, and national databases for outcome assessment and, in some cases, for public reporting. In particular, the development and validation of risk-adjustment models for coronary artery bypass graft surgery (CABG) mortality (1- 5) has led to the development of report cards for cardiac surgery. More recently, risk-adjustment models for percutaneous coronary intervention (PCI) mortality have been developed and used for the comparison and public reporting of operator- and hospital-specific outcomes (6- 9).

Previous studies have suggested that public reporting of CABG outcomes might result in case selection bias and denial of care to or out migration of high-risk patients to other states (10- 12). However, the potential effect of public reporting on case selection for PCI has not been investigated. Therefore, we compared demographic data, indications, and in-hospital mortality from large multicenter PCI databases in Michigan, where public reporting is not mandated, and in New York, where public reporting of PCI data is mandatory.

Baseline demographic data, clinical data, and indications for PCI were prospectively collected from 11,374 consecutive PCIs in a consortium of 8 hospitals in Michigan from calendar years 1998 to 1999, and 69,048 consecutive PCIs from all 34 hospitals performing PCI in New York from the same time period.

The registry is part of the quality assessment and quality improvement program of hospitals voluntarily participating in the consortium. The registry was approved by the University of Michigan Institutional Review Board and by local institutional review boards. The structure of the consortium, the data collection process, and the data quality assurance process have been described elsewhere (2,13). In brief, data on all consecutive patients undergoing PCI in participating hospitals were prospectively collected using standardized definitions and a standardized data form. The data collection forms were reviewed by the coordinating center for completeness and face validity. In addition, the structure of the database includes diagnostic routines to further determine completeness and validity of the data submitted. All participating sites were audited twice yearly. During the audit, 2% of cases were selected at random for review, and, in addition, hospital records of all patients who died in the hospital or who underwent CABG were audited. During the same audits, cardiac catheterization logs were compared with the database logs to ensure enrollment of consecutive patients. A trained nurse from one of the other participating centers audited the coordinating center.

The study sample included 69,048 consecutive patients undergoing PCI in all 34 New York hospitals from January 1, 1998, to December 31, 1999. Prospectively defined data elements were contributed by each hospital to a central coordinating center for analysis. Data elements included information on demographics, comorbidities, procedural details, complications, and in-hospital outcomes. These data elements are required to be submitted to the Department of Health on every PCI performed in New York State to make up the Coronary Angioplasty Reporting System database.

Intravenous glycoprotein IIb/IIIa inhibitors were considered to be administered when abciximab, eptifibatide, or tirofiban were given during or within 3 h after PCI. Periprocedural myocardial infarction (MI) was defined in the New York registry as transmural MI (new Q waves and a rise in creatine kinase to at least 2.5× the upper limit of normal occurring within 24 h of the PCI) or nontransmural MI (determined according to individual hospital guidelines for the diagnosis of nontransmural MI). In the Michigan registry, periprocedural MI was defined as non-Q-wave MI (any rise in creatine phosphokinase-MB fraction above the individual institution’s normal within 24 h of PCI, without new Q waves on electrocardiogram) and Q-wave MI (development of new Q waves that are 0.03 s in width and/or greater than or equal to one-third of the total QRS complex in contiguous leads and as evidenced by subsequent CPK-MB rise to 3 × the baseline value just before intervention).

Heparin therapy indicates treatment with intravenous heparin within 48 h before the PCI. Nitroglycerin treatment indicates therapy with intravenous nitroglycerin within 24 h of the procedure for ongoing ischemia or left ventricular failure. Diabetes mellitus is defined by treatment with oral hypoglycemic agents or insulin.

Creatinine values in the New York registry were collected as a dichotomous variable (≥2.5 or <2.5), whereas in the Michigan registry they were collected as continuous variables. For the purpose of this analysis, the definition of renal insufficiency was standardized as a serum creatinine ≥2.5 mg/dl. Extracardiac vascular disease was defined as history of stroke or peripheral vascular disease.

Left ventricular ejection fraction was missing in 27.8% of Michigan patients and 12.1% of New York patients. Missing ejection fraction values were imputed (14) using a multivariate model that included age, gender, history of smoking, history of diabetes, history of PCI, history of congestive heart failure, renal failure with dialysis, history of extracardiac vascular disease, acute MI, cardiogenic shock, preprocedure intra-aortic balloon pump, creatinine ≥2.5 mg/dl, and preprocedure intravenous use of nitroglycerin or heparin. In the Michigan data set, creatinine values were missing in 8.4% of cases. Missing creatinine values were assumed to be <2.5 mg/dl.

All procedural decisions, including device selection and adjunctive pharmacotherapy, were made at the discretion of the individual physician performing the PCI. Angiographic assessments were made at the individual hospital and generally were achieved by visual assessment. Cardiac enzymes (creatine kinase and creatine kinase MB isoenzyme or troponin) were obtained by protocol before and at 8 and 24 h after PCI in the New York registry and at the discretion of the operator in the Michigan registry.

The primary end point of this analysis was in-hospital mortality. Univariate associations among risk groups for nominal variables were compared using Pearson chi-square test. The two-tailed Student t test was used for continuous variables. A p < 0.05 was considered significant.

Multivariate logistic regression models were fitted in order to obtain adjusted estimates for the odds ratios (ORs) of in-hospital mortality in the New York data set versus the Michigan data set. Predicted mortality rates for each hospital were also calculated. SAS version 8.2 (SAS Institute, Cary, North Carolina) was utilized for all analyses. Model discrimination was assessed using the c-statistic, and model calibration was assessed using the Hosmer-Lemeshow statistic.

Baseline clinical characteristics are listed in (Table 1). Overall, data from 80,422 consecutive procedures were analyzed. The average age was 63 ± 11.9 years, and 32% of patients were women. Patients in Michigan had a significantly higher incidence of renal insufficiency, diabetes mellitus, chronic obstructive pulmonary disease, extracardiac vascular disease, congestive heart failure, and previous PCI (all p < 0.001). Patients in the New York dataset were slightly older (63.6 ± 11.8 years vs. 62.0 ± 12.0 years, p < 0.0001) and had a higher frequency of hypertension.

Patients undergoing PCI in Michigan were more likely to be given preprocedure intravenous heparin or preprocedure intravenous nitroglycerin (Table 1). Patients in Michigan more often underwent PCI in the setting of cardiogenic shock, cardiac arrest, or acute MI.

The unadjusted in-hospital mortality rate was significantly higher in Michigan than in New York, (1.54% vs. 0.83%; p < 0.0001). Significant differences in mortality rates were also observed in the subgroups of patients with acute MI and cardiac arrest (Table 2).

The unadjusted OR for mortality of New York versus Michigan was 0.54 (95% confidence interval [CI] 0.45 to 0.63; p < 0.0001; c- statistic = 0.55; Table 3). Only small changes in the OR and c-statistic were observed after adjustment for age and gender alone. However, after adjustment for age, gender, other clinical risk factors, and hospital procedure volume, the survival advantage in the New York dataset was no longer significant (adjusted OR for mortality 1.05, 95% CI 0.84 to 1.31; p = 0.70; c-statistic = 0.88; Hosmer-Lemeshow chi-square 9.6, p = 0.29) (Tables 3, 4). To determine the potential effect of the imputed missing ejection fraction values on the result, a model excluding ejection fraction as an explanatory variable was developed. Removal of ejection fraction from the model did not change the result of the analysis (adjusted OR for mortality 1.09, 95% CI 0.87 to 1.36; p = 0.45; c-statistic = 0.88; Hosmer-Lemeshow chi-square 11.8, p = 0.15).

Predicted mortality rates based on case mix in the 8 Michigan hospitals and in the 34 New York hospitals are shown in (Figure 1). The median predicted mortality rate in the Michigan hospitals was 1.63% when compared with a median predicted mortality rate of 0.76% in the group of New York hospitals (p = 0.0002). In addition, in the group of New York hospitals, there was only one hospital that had a predicted mortality rate equal to the median of the Michigan hospitals. Substantial differences were also observed in the frequency of PCI for cardiogenic shock among the two groups (Figure 2).

In this analysis of contemporaneous PCI, we evaluated differences in case mix and in-hospital mortality rates between two large, quality-controlled regional PCI registries. One registry operates in Michigan, a state without public reporting of clinical outcomes. The other registry was derived in New York, a state with mandatory public reporting of clinical outcomes after CABG and PCI. We found significant differences in comorbidities and indications for PCI between the two registries. Overall, the patient population in the Michigan registry had a significantly higher frequency of comorbidities. The majority of these comorbidities have been identified in previous studies as independent risk factors for PCI mortality. In addition, a significantly higher frequency of PCI in the setting of cardiogenic shock, acute MI, and cardiac arrest was observed in the Michigan registry compared with the New York registry. Each one of these indications for PCI has also been previously identified as an independent predictor of in-hospital death and has been included in validated risk-adjustment models for PCI mortality (6- 7). As a result of the higher incidence of comorbidities and high-risk indications for PCI, we found a strikingly higher (almost two-fold) unadjusted in-hospital mortality rate in the Michigan registry compared to the New York registry.

The higher in-hospital mortality rate observed in Michigan was explained by the differences in comorbidities and indications for PCI. As shown by logistic regression modeling and analysis of the c-statistic, the variable New York versus Michigan alone had little discriminatory power in explaining differences in mortality rates (c-statistic = 0.55). Some additional discriminatory power was added by age and gender. However, the inclusion of comorbidities and indications for PCI significantly improved model discrimination (c-statistic = 0.88). In the final model, the variable New York versus Michigan was no longer an independent predictor of in-hospital mortality.

The most obvious differences between the New York and Michigan registries were the differences in case mix and the presence or absence of public reporting. Although we do not have any direct proof, a case selection bias driven by the fear of public reporting of higher mortality rates in New York was one possible explanation for the observed differences in case mix and mortality rates.

The publication in 1987 by the Health Care Financial Administration of mortality statistics for CABG has led to the development of regional and state registries for outcome assessment and risk-adjustment. Over the past decade, New York has assumed an important leadership role in outcome assessment for patients undergoing CABG and PCI. Data collection for this effort started in the late 1980s and has continued uninterrupted. In 1991, under the Freedom of Information Act, New York was ordered by the Supreme Court to release mortality statistics for CABG to the newspaper Newsday (1). Those statistics were published in the December 18, 1991, issue of Newsday, and, since then, public reporting of hospital- and operator-specific outcomes has become part of practicing medicine in New York. Over the past decade, there has been a progressive decline in CABG mortality in New York (15- 16). More recently, a decline in PCI mortality has also been observed. The debate over the cause of these declines is ongoing. Advocates of public reporting support the hypothesis that public exposure to outcome data leads to internal quality improvement efforts aimed at reducing the mortality rates from revascularization procedures (15- 16). On the other hand, some studies have shown that mandatory reporting of outcome data can lead to gaming of health care claims by the addition of comorbidities not previously identified as clinically important, to denial of care to high-risk patients, and to out migration of high-risk patients to other states where public reporting does not occur (10- 12). In addition, a recent analysis of secular trends in CABG mortality in the Northeast showed declining mortality rates in states where public reporting does not occur similar to the ones observed in New York over the same time period. These findings call into question the extent to which public reporting was responsible for sustaining improvement in surgical mortality over time (17- 18). Others have suggested that the decline in CABG mortality rates in New York State may have been caused by changes in the way surgical centers collected data over the time period (3). It seems likely that the reduction in CABG mortality is attributable to a combination of all these factors.

Public report cards are often developed from adequate mortality data but without adequate risk-adjustment or by using claims data that lack sufficient detail to perform a thorough risk-adjustment (19- 20). Our analysis argues that quality-controlled clinical data with appropriate risk-adjustment for demographics, comorbidities, and treatment variables is necessary for meaningful outcomes information. It also supports the hypothesis that appropriate risk-adjustment can account for significantly different mortality rates (a two-fold difference in this study).

It is likely that public reporting of outcomes information will become more commonplace in the health care system. Although public reporting has not yet been linked to substantial shifts in hospital volumes due to doctor or hospital “shopping,” as it becomes more popular, patients, payers, insurers, and other interested parties will begin to pay more attention to the information provided (21- 24). The derivation of accurate outcomes information will assume great importance to health care institutions and providers as report cards gain increasing acceptance and prominence. Although making accurate outcomes information accessible has the potential to improve healthcare, our study suggests that public reporting of outcome data might also have an unintended effect on case selection, leading to a tendency toward not intervening on higher-risk patients. More studies will be needed to determine the full effect of public reporting on quality and access to care.

There are several important limitations of the current study. First, this analysis compared data from two registries collected in two different states. An assumption was made that the patient populations across the two states were comparable and that the case mix differences observed in the two registries were due to selection bias rather than to true differences in comorbidities. This statement is indirectly supported by statistical data from the American Heart Association that shows that the age-adjusted coronary heart disease death rates per 100,000 are 210 in Michigan and 240.4 in New York. Thus, it is unlikely that fewer patients with MI or cardiogenic shock underwent PCI in New York because the population suffered fewer MIs than the population in Michigan. In addition, we were unable to assess whether a different approach existed between the two states relating to the possible futility of procedures performed. In a previous analysis, we have shown that mortality rates in patients with multiple comorbidities are high regardless of the results of the procedure performed, and that these high mortality rates are due to the natural history of the disease rather than to the procedure itself (6). Whether PCI in these very high-risk patients is futile, and whether the differences between the two registries were due to a futility assessment in New York remains to be determined. However, numerous prior randomized trials and registry studies have shown a beneficial effect of PCI in patients with acute MI (25).Therefore, the finding that the frequency of PCI in patients with acute MI in New York was roughly half the frequency of PCI for acute MI in Michigan is a concern. Adding to this concern are the results of a recent survey of interventional cardiologists practicing in New York State in 1998 to 2000 (26). In that survey, the majority of respondents (79%) agreed that the publication of mortality statistics influenced their decision to intervene on critically ill patients, such as patient with cardiogenic shock, and 83% agreed that patients who might benefit from angioplasty may not receive the procedure as a result of public reporting. Thus, the observed differences in our study may represent an unintended and adverse consequence of public reporting of outcomes data.

There are important variations in case mix between patients undergoing PCI in Michigan compared to New York. These differences explain the significant discrepancies in unadjusted mortality rates and suggest a propensity in New York toward not intervening on higher-risk patients. A fear of public reporting resulting in case selection bias in New York is one possible explanation for these differences.