For many years, the American College of Cardiology (ACC) and the American Heart Association (AHA) have jointly published and updated guidelines to be considered by referring physicians when determining what treatments to recommend. “Indications” have been provided for a variety of treatments, including percutaneous coronary intervention (PCI) and coronary artery bypass graft (CABG) surgery (1- 2) for patients with coronary artery disease (CAD).

Recently, the ACC and AHA, in conjunction with the Society for Cardiovascular Angiography and Interventions (SCAI), the Society of Thoracic Surgeons (STS), the American Association for Thoracic Surgery (AATS), and the American Society of Nuclear Radiology (ASNC), released appropriate use criteria (AUC) for coronary revascularization to serve as a supplement to the earlier ACC/AHA guideline documents (3). The process of appropriateness development, which was described in an earlier publication (4), derives from methods developed by the RAND Corporation many years ago that are outlined in a more recent release (5). It consists of identifying clinical scenarios that are intersections of clinical characteristics that are essential in determining the appropriate intervention for a given patient. Earlier versions of appropriateness criteria for CABG surgery and PCI were published and tested many years ago (6- 10).

The new appropriateness criteria use the broad categories 1) acute coronary syndrome (ACS); 2) no ACS/no prior CABG surgery; and 3) no ACS/prior CABG surgery as the first means of separating patients into “scenarios” that are used to identify whether patients are appropriate for revascularization. Within these 3 indications, patients are classified into scenarios on the basis of number of vessels diseased/proximal left anterior descending artery (PLAD) disease, findings on noninvasive testing, amount of anti-ischemic medical therapy received, and Canadian Cardiovascular Society (CCS) angina class. In keeping with the RAND methodology, patients are rated as appropriate, uncertain, or inappropriate for revascularization in the first part of the document, and then selected high-risk patients who were judged to be appropriate for revascularization are rated in the same manner with regard to PCI versus CABG surgery.

A recent publication using data from the National Cardiovascular Data Registry (NCDR) found that, for nonacute conditions, 50% of PCI patients were classified as appropriate, 38% as uncertain, and 12% as inappropriate (11). The investigators also found substantial variation across hospitals, with an interquartile range of 6.0% to 16.7%.

The purpose of our study is to use information from New York State's CABG surgery and PCI registries to examine the extent to which actual practice in New York conforms to the appropriateness recommendations of the ACCF/SCAI/STS/AATS/AHA/ASNC (henceforth called ACCF criteria) regarding the use of revascularization. Subsequent analyses examine the same question for each of the specific scenarios in the ACCF criteria, and examine the variation across hospitals in the state in conforming to the criteria.

The databases used in the study were New York State's Cardiac Surgery Reporting System and the Percutaneous Coronary Interventions Reporting System. These registries were developed in 1989 and 1991, respectively, for the purpose of collecting information on all New York patients undergoing CABG surgery and PCI in nonfederal hospitals in the state. They contain information on demographics, comorbidities, left ventricular function, hemodynamic state, vessels diseased, hospital and operator identifiers, and in-hospital adverse outcomes. As of July 2009, they also contain information on the extent of anti-ischemic medical therapy used by patients and noninvasive test findings, both of which are included in the scenarios used in the ACCF document. The stress test information includes standard exercise stress tests, stress echocardiogram, stress testing with single-photo emission computed tomography, and with cardiac magnetic resonance. Also, low-, intermediate-, and high-risk findings are defined and recorded. Definitions for these data elements are identical in the 2 systems. Data are audited and cross-checked against the department's acute care hospital discharge database, the Statewide Planning and Research Cooperative System to ensure accuracy and completeness.

All 40 nonfederal hospitals in which CABG surgery was performed and 58 nonfederal hospitals in New York in which PCI was performed during the study period were included in the study. Patients in the study included all patients who underwent the procedures in these hospitals between July 1, 2009, and December 31, 2010, except patients who had ACS and/or had previous CABG surgery. A total of 10,460 isolated CABG patients and 33,970 PCI patients met these criteria. Of these patients, a rating could not be determined for 2,292 CABG surgery patients and 9,425 PCI patients, leaving a total of 8,168 CABG surgery patients and 24,545 PCI patients for whom a rating could be determined. Reasons for not being able to determine ratings will be discussed in the results section.

Each of the numbered indications in (Table 2) of the ACCF document used in the study (indications 12 to 47) is further classified in the document according to CCS class (asymptomatic, class I or II, class III or IV), except indications 18, 19, and 21. Since indication 21 requires fractional flow reserve or intravascular ultrasonography findings, and the New York databases do not include intravascular ultrasonography information in sufficient detail, it is not included in the study, and all 3 of its scenarios are missing.

The AUC rating assignments are determinations of whether each CABG surgery patient and each PCI patient met the ACCF appropriateness criteria for revascularization. The recommendations for each patient were categorized as appropriate, uncertain, or inappropriate. Reasons for the inability to determine ratings were also explored.

For each procedure (CABG, PCI), the number of patients receiving the procedure was summed across all scenarios and classified according to each of the 3 ratings so that the percentage of patients undergoing each of the procedures who were appropriate, uncertain, and inappropriate could be determined. Ratings were examined for each procedure for each of the 103 scenarios so that the scenarios most responsible for the overall results could be ascertained.

Other analyses were conducted to determine the range across hospitals in uncertain and inappropriate ratings for each procedure after eliminating hospitals with relatively low volumes. The extent to which variations in rates of inappropriate (and rates of inappropriate/uncertain, and inappropriate/uncertain/unrated) PCIs was examined by using multivariable hierarchical regression with the GLIMMIX (SAS Institute, Cary, North Carolina) procedure to compute the hospital random effects variation, which was then used determine the median rate ratio (12). The median rate ratio is the likelihood of patients with identical clinical characteristics undergoing an inappropriate PCI at 1 randomly chosen hospital relative to another randomly chosen hospital. A value of 1 denotes no variation. Also, the relationship between the inappropriateness rate and hospital volume was investigated using Spearman's correlation coefficient.

For patients who could not be rated, reasons were explored. All tests for statistical significance were 2-sided, with a p < 0.05 significance level, and all statistical analyses were performed using SAS version 9.2 (SAS Institute).

There were totals of 10,460 CABG surgery patients and 33,970 PCI patients in the New York registries who met the definitions described in the methods (no ACS, no prior CABG surgery). Of these patients, hospitals had sufficient information to rate 8,168 CABG patients and 24,545 PCI patients. Of the CABG surgery patients who could be rated, 7,372 (90.3%) were appropriate for revascularization according to the ACCF criteria, 91 (1.1%) were inappropriate, and 705 (8.6%) were uncertain (Table 1). Of the 92 CABG surgery patients who were inappropriate for revascularization, 51 (55.4%) were asymptomatic with 1- or 2-vessel CAD without involvement of PLAD disease, had intermediate-risk findings on noninvasive testing, and were receiving no or minimal anti-ischemic medical therapy; and another 20 (21.7%) were CCS class I or II with 1- or 2-vessel CAD without PLAD disease, had low-risk findings on noninvasive testing, and were receiving no or minimal anti-ischemic medical therapy. Of the CABG patients rated as uncertain for revascularization, 113 (16.0%) were CCS class I or II with 1- or 2-vessel CAD without PLAD disease with intermediate-risk findings on noninvasive testing and receiving no or minimal anti-ischemic medical therapy, 131 (18.5%) were CCS class III or IV with 1- or 2-vessel CAD without PLAD disease with intermediate risk findings on noninvasive testing and receiving no or minimal anti-ischemic medical therapy, and 154 (21.7%) were CCS class I or II with 1- or 2-vessel CAD without PLAD disease and had no stress test done, placing them in a scenario that does not mention anti-ischemic medical therapy (Table 2).

Of the PCI patients who could be rated, 8,856 (36.1%) were appropriate, 3,508 (14.3%) were inappropriate, and 12,181 (49.6%) were uncertain (Table 1). A total of 34 of the 58 hospitals approved to perform PCI in New York report data to the NCDR. Patients in these hospitals comprised 57% of the PCI patients in (Table 1), and their rates of appropriate, uncertain, and inappropriate findings in our registry were 36%, 47%, and 17%, respectively, compared with 36%, 53%, and 11% for hospitals not in the NCDR. Why these differences exist is beyond the scope of this study. It should be noted that 2 differences between our study and the NCDR study with regard to data definitions and assignment of patients to acute versus nonacute scenario groups involved the definition of CCS class (NCDR requests the highest class, and our definition requests the “typical” class) and the definition of acute myocardial infarctions, which we defined based on a symptom onset time of <24 h and NCDR appears to define on the basis of 7 days. When we removed the 1,688 myocardial infarction patients with symptom onset times between 24 h and 7 days from the nonacute group that our study was based on, we obtained identical percentages of appropriate, inappropriate, and uncertain to the nearest integer.

Of the 3,508 PCI patients who were inappropriate for revascularization according to the ACCF criteria, 1,583 (45.1%) were asymptomatic with 1- or 2-vessel CAD without PLAD disease, had intermediate-risk findings on noninvasive testing, and were receiving no or minimal anti-ischemic medical therapy (Table 3). Another 1,203 PCI patients (34.3% of all patients inappropriate for PCI according to ACCF criteria) were CCS class I or II with 1- or 2-vessel CAD without PLAD disease, had low-risk noninvasive test findings, and were receiving no or minimal anti-ischemic medical therapy; and another 488 (11.6%) were identical to the previous group except that they were asymptomatic instead of CCS class I or II. Among the PCI patients who were uncertain for revascularization, 5,019 (41.4%) were CCS class I or II with 1- or 2-vessel CAD without PLAD disease and had no stress test done, placing them in a scenario with no mention of anti-ischemic medical therapy in the criteria; and another 3,132 patients (25.8%) were CCS class I or II with 1- or 2-vessel CAD without PLAD disease, had intermediate-risk noninvasive test findings, and were receiving no or minimal anti-ischemic medical therapy.

For the 46 hospitals with PCI volumes >400, the range in percent of PCI patients who were inappropriate for revascularization according to the ACCF criteria was from 1% to 40% (Figure 1). The median rate ratio was 1.93 (95% confidence interval: 1.64 to 2.20), indicating that there was a large variation across hospitals in the chance of patients with identical clinical characteristics undergoing an inappropriate PCI. The value of Spearman's correlation coefficient for the relationship between total hospital volume and hospital inappropriateness rate was very weak (R = 0.03). For CABG surgery, the range of inappropriate cases according to ACCF criteria was from 0.0% to 6%.

As indicated in (Table 4), of the 9,425 PCI cases for which a rating could not be determined, 2,834 (30.0%) were in scenario 18 (1- or 2-vessel disease/no PLAD disease/no noninvasive testing/asymptomatic), which was not rated by the ACCF because the writing group considered the likelihood of the clinical scenario was so low that rating should not be performed. These cases would appear to be particularly inappropriate for revascularization because there is no expectation of survival benefit and no possibility of quality of life improvement. If these cases were added to the numerator and denominator in the computation of the percentage of rated cases that are inappropriate, the percentage of inappropriate PCIs would rise from 14.3% to 23.2%. Another 3,171 (33.6%) PCI patients were reported as having a positive finding on noninvasive testing, but were classified in a scenario requiring information as to whether the finding was high risk or not, and were missing such information. Yet another 3,078 patients (32.7%) were in scenarios that required noninvasive testing, and the hospital reported that there was no noninvasive testing done or no evidence of results in its medical records. We were able to query the PCI hospitals and confirm that they did not have information regarding the risk level of positive stress tests when it was needed for assigning appropriateness. Also, for cases with stress tests reported as missing when they were needed for appropriateness ratings, we audited 127 medical records and found only 8 (6%) had stress test information in the record.

Of the 2,292 CABG surgery cases for which a rating could not be determined, 1,619 (70.6%) were also in scenarios that required noninvasive testing, and the hospital reported that there was no noninvasive testing done or no evidence of results in its medical records. Another 571 cases (24.9%) had a positive stress test result but required an unavailable risk level to assess appropriateness.

This study assessed the referral patterns for CABG surgery and PCI in New York State as a function of the scenarios in the recent appropriateness criteria developed by the ACCF/SCAI/STS/AATS/AHA/ASNC to assess conformance to the ACCF criteria. This was done for the portion of the ACCF appropriateness criteria that related to appropriateness of revascularization relative to medical therapy, not the appropriateness of PCI versus CABG surgery. The study was also restricted to the part of the ACCF criteria relating to patients without ACS or prior CABG surgery because there is reason to believe that this is the group of patients for whom practice may be most at odds with the appropriateness criteria (12).

Recent studies have examined guidelines adherence among patients undergoing PCI (13- 14) or CABG surgery (15), or both PCI and CABG surgery (16), and 2 of them have also demonstrated a direct relationship between guidelines adherence and better outcomes (13- 14).

One of these studies was an earlier study in New York that used ACC/AHA guidelines that preceded their appropriateness criteria (1- 2) to show that among patients who were indicated for CABG surgery and not PCI, a total of 50% were recommended for CABG surgery and 34% were recommended for PCI, whereas among patients indicated for PCI and not CABG surgery, 93% were recommended for PCI, and only 5% were recommended for CABG surgery (16).

A recent study similar to ours examined the appropriateness of PCI (not guidelines adherence) using the same ACCF appropriateness criteria used in our study, and found that only 50% of the PCIs for nonacute conditions in the NCDR were appropriate (12). Our study found that of the New York patients undergoing PCI without ACS or prior CABG surgery for whom a rating could be determined, 8,856 (36.1%) were appropriate, 3,508 (14.3%) were inappropriate, and 12,181 (49.6%) were uncertain. These results are similar to, but somewhat more discordant with, the ACCF criteria than the other recent findings, which were that for patients who underwent PCI for nonacute conditions, 50.4% were appropriate, 11.6% were inappropriate, and 38.0% were uncertain.

The overall findings of high use of PCI are consistent with the earlier study in New York described, which found that many patients indicated only for CABG surgery had received a recommendation for PCI (16). It is also consistent with the findings of another study, which compared utilization of CABG surgery and PCI in New York and Ontario. In that study, Ko et al. (17) found that among non–acute myocardial infarction patients in 2004 through 2006, the age- and sex-adjusted population-based rate of PCIs was 2.5 times higher (95% confidence interval: 2.2 to 2.5) in New York, whereas there was no significant difference in the age- and sex-adjusted population-based rates of CABG surgery. Caveats of this study are that the number of patients in each region with CAD, and patient preferences, are unknown.

As noted earlier, one respect in which this study differs from the study by Chan et al. (11) is that our study also includes an assessment of the appropriateness of revascularization for CABG surgery patients. Findings were that 90% of all patients undergoing CABG surgery were judged to be appropriate for revascularization, only 1% were inappropriate, and 9% were uncertain. Thus, there was a very high correspondence between choice of CABG surgery and the ACCF criteria for revascularization. We hypothesize that a reason why CABG surgery patients were found to be rarely inappropriate in comparison with PCI is that a relatively large percentage of CABG surgery was performed on patients with severe CAD (3-vessel disease and left main disease). Also, CABG utilization may be curtailed as a result of the greater invasiveness and perceived (whether correct or not) risks of CABG.

Thus, in concert, these 3 studies consistently suggest an overuse of PCI, with seemingly no significant overuse of CABG surgery. The practice pattern variation was quite large, and the treatment recommended to patients without ACS was very much dependent on the hospital in which they were treated. This is very important because it means that even if interventional cardiologists disagree in general with the ACCF ratings, as a group they do not seem to have consistency with regard to how they choose patients for PCI.

A major difference between the ACCF criteria and actual practice appears to be the importance of patients with 1- or 2-vessel disease without PLAD disease being on maximal anti-ischemic medical therapy, having high-risk findings on noninvasive stress testing, and/or having CCS class II or IV (without unstable angina). The only scenarios deemed appropriate for revascularization by the ACCF had at least 1 of these 3 criteria for patients with 1- or 2-vessel disease without PLAD disease. In practice, many patients without any of these criteria underwent PCI.

Another important finding of our study is that of the 33,970 PCI patients without ACS or prior CABG surgery, an appropriateness rating could not be determined for 9,425 (27.7%) of the patients. The primary reasons for this were that either there was no medical record documentation when needed regarding noninvasive test results, or the documented results of the noninvasive tests were not specific enough to determine whether the results were highly positive.

A caveat of the study is that we were unable to assess the degree of underuse of CABG surgery and PCI according to the ACCF criteria because the New York registries only contain data for patients who underwent those procedures. Also, it is important to note that the appropriateness criteria that were examined in this study were published in February 2009, and the study was based on data from July 2009 through 2010. It is possible that some clinicians were not aware of the criteria during part or all of the study period. Furthermore, as pointed out in the document presenting the ACCF appropriateness criteria, some PCIs classified as inappropriate may have been performed on patients with conditions not covered by the criteria who were best served by PCI (3). It is expected that these would be rare circumstances that would not account for the substantial interhospital variations in inappropriateness found in our study.

Our study: 1) reinforces earlier findings regarding the relatively low percentage of patients undergoing PCI for reasons deemed appropriate by the ACCF; 2) provides new evidence that the ACCF criteria indicate very high levels of appropriateness for CABG surgery; and 3) demonstrates that in addition to the many PCI patients deemed inappropriate or uncertain for the procedure, there are many more for whom proper judgment of appropriateness cannot be made for lack of supporting information. Our intent is to share these findings with clinicians and to work collaboratively to reduce inappropriate clinical decisions and variations in hospital and cardiologist practice patterns.

The authors would like to thank New York State's Cardiac Advisory Committee for their encouragement and support of this study; and Rosemary Lombardo, Cynthia Johnson, Erika Ihara, and the cardiac surgery departments and cardiac catheterization laboratories of the participating hospitals for their tireless efforts to ensure the timeliness, completeness, and accuracy of the data submitted.