CLINICAL STUDIES
Frequency of major adverse cardiac events within one month of coronary angioplasty: a useful measure of operator performance
Joseph Lindsay, Jr., MD, FACCa,
Ellen E. Pinnow, MSa and
Augusto D. Pichard, MD, FACCa
a Section of Cardiology, the Washington Hospital Center, Washington, DC, USA
Manuscript received February 9, 1999;
revised manuscript received June 14, 1999,
accepted August 25, 1999.
Reprint requests and correspondence: Dr. Joseph Lindsay, Jr, Section of Cardiology, Washington Hospital Center, 110 Irving St. Northwest, Washington, DC 20010
JML2{at}MHG.edu
 |
Abstract
|
|---|
OBJECTIVES
To test one-month outcomes in a single center for their statistical power to corroborate conclusions derived from large multicenter databases.
BACKGROUND
Only with large, multicenter databases has it been possible to demonstrate more frequent occurrences of complications in patients treated by "low-volume operators." Critics feel that such analyses mask excellent performance by many "low-volume operators."
METHODS
In a high-volume cardiac catheterization laboratory in a large, nonuniversity teaching hospital, baseline clinical and angiographic characteristics were collected for a consecutive series of 1,029 patients treated by 37 percutaneous transluminal coronary intervention (PTCI) operators over a four-month period. One-month follow-up was obtained in 967 (94%) patients who form the basis for this analysis.
RESULTS
Only the group of operators performing <50 cases annually had a major adverse cardiac event (MACE) (death, myocardial infarction or symptom-driven revascularization) rate at one month significantly greater than predicted from baseline characteristics. (Observed rate: 15.1%, expected: 9.7%, 95% confidence interval [CI]: 4.7%, 14.6%.) The difference was driven by the significantly more frequent rate at which repeat revascularization was performed in patients treated by that group of operators (observed: 13.8%, expected: 7.1%, 95% CI: 2.8%, 11.4%).
CONCLUSIONS
As is true of analyses of large multicenter databases, lower volume operators as a group have less good outcomes than those performing more. The greater statistical power provided by one-month MACE rate offers advantages over the use of in-hospital complications for the analysis of operator performance.
|
Abbreviations and Acronyms
| | CABG | = coronary artery bypass surgery | | MACE | = major adverse cardiac event (death, myocardial infarction, symptom-driven revascularization) | | PTCI | = percutaneous transluminal coronary intervention |
|
Despite strong evidence for an inverse relationship between operator volume and outcomes of percutaneous transluminal coronary intervention (PTCI), controversy remains. The debate has focused on the recent report of an expert panel of the American College of Cardiology (1). That group recommended that, to maintain competence in this procedure, an operator should perform a minimum of 75 PTCI procedures annually. Four published analyses of large multicenter databases (2–5) form the basis for their recommendation. Each analysis assessed the relationship of operator volume to in-hospital complications in several thousands of patients.
Many cardiologists have questioned the fairness of applying this standard to individual physician credentialling on several grounds. First, they point out that the excellent performance of many operators performing fewer than 75 cases annually may be obscured in such large conglomerates of data. Individual "low-volume operators" may have excellent results particularly in "high-volume" laboratories (2,6). Second, in a related argument, critics suggest that multicenter analyses may obscure a variety of laboratory-dependent factors that may influence outcome. Finally, critics contend that proper risk adjustment (a fundamental necessity in outcomes analysis) may not be effectively accomplished using multicenter data since consistent and complete collection of baseline clinical and angiographic characteristics of patients may be problematic.
Statistical obstacles also stand in the way of fair assessment of the performance of "low-volume operators" when the rate of in-hospital complications is the outcome variable of interest. Since these events occur with a frequency of less than 5% (7), only the large number of patients that typically are available from multicenter studies provides the statistical power to detect differences in performance between groups of operators (8,9). The necessity for risk adjustment adds to the statistical dilemma since multivariate analysis requires large data sets (10).
We sought to overcome the statistical problems inherent in the use of infrequent in-hospital complications as the outcome variable by replacing it with one-month adverse events. Furthermore, we felt that an experience collected in a single center would both eliminate the possibility of variability in operator performance related to differences in institutional practice inherent in multicenter databases and allow a more consistent collection of baseline characteristics for risk adjustment.
We believe one-month event rates to be a valid index of operator performance for two reasons: first, since both angiographic (11,12) and intravascular ultrasound data (13) support a conclusion that neither restenosis nor progression of disease is common within one month of PTCI, it is reasonable to conclude that adverse events occurring within that month are related either to the procedure or to patient selection. Second, from the standpoint of patient satisfaction, one-month results may be a more ideal criterion for successful PTCI than merely freedom from in-hospital complications. Unlike previous analyses, we included symptom-driven repeat PTCI within the month as an adverse event. Certainly the patient is entitled to expect to be free of the need for additional revascularization for at least a month.
|
Methods
|
|---|
The study was approved by the Institutional Review Board of the Medlantic Research Institute. Each patient provided verbal permission before a telephone questionnaire was undertaken.
Patients.
One thousand twenty-nine consecutive patients underwent PTCI during the four-month period from September 1 to December 31, 1996 in the catheterization laboratory of this institution. The 967 (94.0%) in whom one-month follow-up information could be obtained form the basis of this report. If a patient underwent more than one PTCI during this time frame, the initial one was taken as the index procedure.
Operators.
The procedures were carried out by 37 different operators. Each had been credentialled to perform PTCI after review by the Peer Review Committee of the laboratory. Each met the criteria for training and experience established by that committee. They were grouped with regard to the number of PTCI procedures personally performed during the 12-month period from July 1, 1996–June 30, 1997. Operator groups were selected prior to the analysis with reference to the American College of Cardiology threshold of 75 cases annually and with the intention of having (with the exception of the very high volume operators) a relatively equal distribution of patients (Table 1). The rate of follow-up was slightly lower (87%) in the patients of operators performing 75–199 cases annually. It was 93% in patients of those performing <50 and 95% in those of operators in the other two groups. Thus, the conclusions of the analysis are not likely to be affected.
Data collection.
As is routine in this laboratory, the operating cardiologist or an assistant completed a report form describing the baseline clinical characteristics of each patient and the location and complexity of each target lesion. Target lesions were assigned a complexity score [AHA/ACC classification (14)] by the operating cardiologist based on visual inspection. To facilitate the analysis of patient-based outcomes, in any instance of multilesion treatment each patient was assigned complexity category based on the most complex lesion targeted. Moreover, patients were assigned to the saphenous vein graft group if a stenosis in such a conduit was treated.
To collect information regarding the subsequent hospital course, independent chart review was conducted by specially trained quality assurance nurses to identify the following hospital complications: death from any cause, same admission coronary artery bypass surgery (CABG), myocardial infarction or an additional PTCI for recurrent symptoms.
Telephone follow-up was initiated in March 1997 and completed in December 1997. Information regarding the patient’s course subsequent to hospital discharge was obtained from contact with the patient or with his or her physician. Contact was attempted no sooner than six months following the index PTCI. The occurrence of death or heart attack and the need for CABG or repeat PTCI was recorded. All additional PTCI’s were performed at this institution and records were, therefore, available for review. Only those performed for recurrent unstable symptoms were tallied.
Definition of terms.
Baseline variables were identified by the operator or his assistant. Diabetes was noted when a patient provided that history. Renal failure was entered when the admission serum creatinine  2 mg/dl.
Outcome variables were identified by quality assurance nurses or by telephone contact. Any death, whether procedure-related or not, was tallied. Myocardial infarction was noted when a clinical diagnosis was made in-hospital of Q-wave or non-Q-wave infarction or when, on telephone contact, the patient reported a hospital admission for "heart attack" subsequent to discharge.
Two combined outcome variables were created to provide a sufficient number of events for multivariate analysis. First, a major adverse cardiac event (MACE) was tallied when death from any cause, myocardial infarction, CABG or additional PTCI was recorded. Second, after univariate analysis suggested that the observed differences between operator groups was accounted for by the frequency of additional revascularization, the outcome variable, additional revascularization, was created to include any patient having CABG or an additional symptom-driven PTCI during follow-up.
Statistical analysis.
All data were entered into a computerized database. Demographic, baseline clinical, procedural and outcome variables were analyzed using standard statistical methods. Continuous variables are presented as the mean ± standard deviation and comparisons were made by means of analysis of variance. Discrete variables are presented as percentages and were compared by means of contingency tables and chi-square analysis with Yates correction. Fisher’s exact test was employed when appropriate. A p < 0.05 was considered significant.
Stepwise multivariate logistic regression (15) was used to identify baseline characteristics associated with each of the two combined outcome variables. Those univariately associated at a p < 0.2 level were tested in the multivariate models. From the independent risk factors determined by multivariate analysis, the expected risk of an event was calculated for each patient from the formula:
where: E = expected probability of an event in an individual patient, beta = coefficient describing the relationship of the independent to the dependent variable (outcome), X = presence of an independent variable in the patient, and C = constant.
The expected event rate for patients treated by each operator group was compared with the observed event rate.
|
Results
|
|---|
Baseline characteristics.
The baseline clinical and angiographic characteristics of the patients are listed in Table 2. Differences in those treated by different operator groups reflect the varying practice patterns of the physicians within the group. A distinct trend across volume groups was apparent only for the performance of multilesion PTCI.
Patients treated by operators who performed >200 cases annually differed in several respects from those of the other three groups reflecting the referral nature of the practice of the highest volume operators. Operators in that group treated a greater percentage of patients with prior CABG and prior PTCI, more patients with Type-C lesions, targeted more saphenous vein graft stenoses and performed more multilesion procedures. On the other hand, the patients of these highest volume operators less frequently had unstable angina at presentation.
Unadjusted outcomes.
As expected, the overall MACE rate at one month was nearly double that encountered in-hospital (9.9% vs. 5.1%, Table 3). Interesting differences may be seen in the nature of the adverse events during the two time frames. Death or myocardial infarction not associated with any additional revascularization accounted for 20 (40.8%) of the 49 in-hospital complications but only 5 (10.6%) of the 47 MACE incidents between hospital discharge and one month. Indeed, symptom-driven repeat PTCI was involved in 42 (89.4%) of the 47 MACE incidents occurring after hospital discharge.
Table 4 lists the frequency of MACE at one month sorted by operator volume group. The rate for operators with an annual volume <50 was 15.1%, higher than that (9.1%) of the other three groups (p = 0.040). This difference was determined by the greater frequency with which additional revascularization procedures were required in the lowest volume group (13.7% vs. 6.3%, p = 0.004). There was no difference among the groups for the frequency of death or myocardial infarction.
Risk-adjusted outcomes at one month.
Table 5 lists the baseline variables independently associated with each of the two selected combined outcome events after adjustment by means of stepwise logistic regression. Any prior revascularization procedure and myocardial infarction within 48 hours of the index PTCI were independent predictors of both MACE and need for additional revascularization. The presence of cardiogenic shock at the index procedure was an independent predictor only of MACE. The c-statistic for accuracy of the model was 0.65 for one-month MACE and 0.64 for need for additional revascularization by one month.
Table 6 and Figure 1 depict the expected and observed event rates for the operator groups. Only operators in the lowest volume group had outcomes significantly different from those expected. The frequencies of both MACE and repeat revascularization in that group were greater in patients treated by that group of operators. Indeed, the observed frequency of repeat revascularization was nearly twice the expected rate.
View larger version (11K):
[in this window]
[in a new window]
|
Figure 1 The observed one-month event rate for each operator group is indicated by a star and is displayed against the expected rate with its 95% confidence intervals for both MACE (A) and additional revascularization (B). Notice that only the outcomes of the group of operators performing <50 cases annually fall outside the 95% confidence limits of the expected rate for their patient population. MACE = major adverse cardiac event.
|
|
Intergroup variation in treatment strategy.
We sought to identify differences in procedural strategies that might account for the differences in the observed outcomes. Distinct differences were apparent (Table 7). Operators performing >200 cases annually employed stents (70.5% vs. 47.6%, p < 0.001) and debulking devices (atherectomy or laser angiography) (28.2% vs. 11.4%, p < 0.001) more frequently than operators in the other three groups. The frequency of MACE was similar for patients receiving stents as compared with those who did not (p = 0.407). Those in whom stents were deployed had fewer repeat revascularizations, but the difference did not reach statistical significance (7.2% vs. 10.5%, p = 0.099). When stent use and operator volume were added to the multivariate model, the latter continued to be an independent predictor of both outcome variables while stenting was not. The frequency of neither MACE (11.5% vs. 12.4%, p = 0.780) nor need for repeat revascularization (7.2% vs. 8.8%, p = 0.533) was different with regard to use of a debulking device.
Abciximab was employed in 84 (8.7%) of patients. The group of operators performing between 50 to 74 cases annually used the agent more often than the other three groups (15.7% vs. 8.2%, p = 0.018). Abciximab use was associated with more frequent MACE (20.2% vs. 11.5%, p = 0.030) but not repeat revascularization by one month (p = 0.573). The increased frequency of MACE in patients treated with abciximab undoubtedly relates to the tendency of operators in this laboratory to limit the use of this agent to "high-risk" patients or as a "bail out" strategy. Thus, no systematic variation in the use of these treatment strategies accounts for the difference in outcome between operator groups.
|
Discussion
|
|---|
These data indicate that the use of one-month MACE rates following PTCI provides a valid basis for analysis of operator outcomes. The fact that our findings are consistent with those reported from the analysis of very large, multicenter data sets (2–5) supports this conclusion. We used data from 967 patients to demonstrate that, as a group, physicians performing 50 cases annually had better outcomes than the group performing <50. The difference was determined by the greater frequency with which additional revascularization was carried out. In fact, the observed rate of an additional PTCI or CABG within a month of the index procedure was nearly double that predicted by the multivariate model in patients treated by physicians performing <50 cases annually (Table 6).
Unlike the large multicenter analyses (2–5), we tallied the need for an additional PTCI during the first month after the index procedure as an adverse event. Counting such events substantially enhanced the statistical power of the analysis. Indeed, almost 90% of events between hospital discharge and one-month involved repeat PTCI. Thus, the greater frequency of PTCI between hospital discharge and one month in patients treated by the operator group with the lowest annual volume in large measure accounts for the differences observed. We believe the inclusion of such end points to be justified since we tallied only those undertaken because of renewed or continued symptoms.
We are aware of only one other instance in which one-month outcomes were used to assess operator performance. That study also found a better performance by "high-volume operators." Kastrati et al. (16) used a 30-day combined variable of death, myocardial infarction and CABG to analyze operator performance with stent implantation. In their study, higher volume operators had better outcomes.
Potential advantages of the use of one-month MACE.
Importantly, the greater statistical power provided by one-month outcomes allows analysis of the variation between operator groups in a single institution. Utilization of results from a single laboratory overcomes many of the criticisms of the use of large, multicenter databases. Many of the potential reasons for variability in operator performance are eliminated since all procedures are performed in uniform environment. Moreover, risk adjustment will almost certainly be more consistent.
Shook et al. (17) in a previous study of the relationship between operator volume and outcome from a single institution reviewed 2,350 PTCI procedures performed over a three-year period. They found that more emergency CABGs were required and more in-hospital morbidity was encounted in patients treated by the "low-volume operator" group.
It does not necessarily follow that all operators performing <50 cases annually have less than optimal results. In our analysis we found many physicians in the lowest volume group with quite good outcomes (Fig. 2). There was a wide scatter reflecting the small sample size for these operators. What is needed, therefore, is a way to evaluate the performance of individual operators. Such a means of evaluation is particularly important to directors of cardiac catheterization laboratories and divisions of cardiology since it is on their shoulders that the expert panel of the American College of Cardiology placed the responsibility for operator evaluation (1).
View larger version (13K):
[in this window]
[in a new window]
|
Figure 2 Scatter plots are shown demonstrating individual operator frequency of MACE (A) and of additional revascularization (B) at one month. Note the wide scatter in event rates for the operators performing <50 cases annually. MACE = major adverse cardiac event.
|
|
The greater statistical power provided by one-month MACE rates may offer a partial solution to this problem. A sample-size calculation indicates that, given an event rate of 9.9%, a 2.5-fold increase over the laboratory standard can be detected with a sample size of about 40 patients (alpha = 0.05, power = 0.80). Using in-hospital complications (typically about 5%), a sample size of 90 is needed. Thus, while not ideal, one-month MACE is potentially useful to evaluate even lower volume operators.
Appropriateness of the use of one-month outcome.
Our assumption that the frequency of one-month MACE largely reflects procedure-related factors is supported not only by the previously cited reports of angiographic and intracoronary ultrasound observations (11–13) but also by our own data set. All of the difference in performance between operator groups is determined during the first month. The frequency of MACE was similar in all groups between one and six months (Fig. 3). It is in the latter time frame that restenosis rather than procedural factors will account for the vast majority of events.
View larger version (37K):
[in this window]
[in a new window]
|
Figure 3 Bar graphs depict the frequency of MACE (A) and additional revascularization (B) for the operator groups at one month and between one and six months. The difference between groups is apparent at one month. Event rates between one and six months are similar for all groups. MACE = major adverse cardiac event.
|
|
Laboratory environment.
The cardiac catheterization laboratory at this institution provides an advantageous environment in which to address operator volume and outcomes. In many respects it is a microcosm of the practice of interventional cardiology in the U.S. The laboratory is open to all qualified cardiologists, and operators are, therefore, heterogeneous with regard to experience and annual volume. Twenty-three of the 37 operators reviewed for this analysis had been active in interventional cardiology for more than 8 years and in one instance for more than 20 years. Thirteen of the remaining 14 were graduates of interventional fellowship programs. Nearly three-quarters of the physicians perform fewer than the 75 cases annually proposed by the expert panel, but the majority of patients are treated by individuals who do >200. Moreover, the laboratory is unquestionably high-volume. Nearly 4,000 PTCI procedures are performed annually. Philosophically and operationally, the more experienced operators and support staff are committed to assisting all physicians in the safe and effective care of their patients. Thus, the performance of low-volume operators is likely to be enhanced by participation in this high-volume laboratory.
Study limitations.
This cohort study was conducted as part of an ongoing quality assurance program in this laboratory. It was intended to investigate the practicality of using one-month outcomes on a routine basis. Our resources were not adequate to allow either collection of all of the 29 baseline variables identified by Block et al. (18) as having an association with adverse outcomes after PTCI or for independent review of all angiograms. We used twelve baseline variables that are relatively free of potential for operator bias (Table 2). Importantly, only a few angiographic characteristics of target lesions were included since independent review was not available. None proved to be an independent predictor of one-month events.
All outcome variables are "hard" end points with the exception of the occurrence of myocardial infarction. For that variable, problems in definition exist. The frequency of in-hospital non-Q-wave infarction may have been influenced by the frequency with which the operating cardiologist obtained serum indexes of myocardial injury. Moreover, the identification of myocardial infarction after hospital discharge depended on patient recall and may have included omissions or misidentification of infarction. Fortunately, the occurrence of myocardial infarction did not contribute to the observed differences in outcome between operator groups.
While the risk-adjustment analysis yielded results that are intuitively reasonable, some of its aspects could be improved. Certain of the baseline variables discarded as not independently related were either not precisely defined prior to data collection (i.e., unstable angina) or were subject to observer bias (i.e., lesion complexity). Moreover, other potentially important outcome predictors were not analyzed (i.e., ejection fraction) because of incomplete data. Thus, the multivariate model needs validation and potentially, modification.
We cannot exclude the possibility that one or more operators performed PTCI in other laboratories and were, as a consequence, misclassified as "low-volume." Such a misclassification could, theoretically, either enhance or detract from the observed performance of the <50 case group. Unfortunately, reliable data on operator performance at competing institutions were not available. Based on our knowledge of community practice, however, we do not believe that any possible misclassification adversely affected the observed performance of the "low-volume group."
Conclusion.
Risk-adjusted one-month MACE rates provide data relating operator volume and outcome of PTCI that is consistent with the results of the analyses of large multicenter databases. Moreover, the greater statistical power provided by the overall rate of MACE and, particularly, of the need for performance of additional revascularization by one month appear to offer an option for analyzing individual physicians with interventional practices of 40 to 50 procedures annually.
|
Footnotes
|
|---|
This study was supported by a grant from the Medlantic Research Institute, Washington, DC.
|
References
|
|---|
- Technology and Practice Executive Committee of the American College of CardiologyHirshfeld JW, Ellis SG, Faxon DP. Recommendations for the assessment and maintenance of proficiency in coronary interventional procedures. Statement of the American College of Cardiology. J Am Coll Cardiol. 1998;31:722–743[Free Full Text]
- Jollis JG, Peterson ED, Nelson CL, et al. Relationship between physician and hospital coronary angioplasty volume and outcome in elderly patients. Circulation. 1997;95:2485–2491[Abstract/Free Full Text]
- Ellis SG, Weintraub W, Holmes D, et al. Relation of operator volume and experience to procedural outcome of percutaneous coronary revascularization at hospitals with high interventional volumes. Circulation. 1997;95:2479–2484[Abstract/Free Full Text]
- Hannan EL, Racz M, Ryan TJ, et al. Coronary angioplasty volume-outcome relationships for hospitals and cardiologists. JAMA. 1997;279:892–898
- McGrath PD, Wennberg DE, Malenka DJ, et al. Operator volume and outcomes in 12,988 percutaneous coronary interventions. J Am Coll Cardiol. 1998;31:570–576[Abstract/Free Full Text]
- Klein LW, Schaer GL, Calvin JE, et al. Does low individual operator coronary interventional procedure volume correlate with worse institutional procedural outcome? J Am Coll Cardiol. 1997;30:870–877[Abstract]
- Lindsay J Jr, Pinnow EE, Pichard AD. New devices enhance hospital results of coronary angioplasty. Cathet Cardiovasc Diagn. 1998;43:1–6[CrossRef][Medline]
- Lindsay J Jr, Pinnow EE, Popma JJ, Pichard AD. Obstacles to outcomes analysis in percutaneous transluminal coronary revascularization. Am J Cardiol. 1995;76:168–172[CrossRef][Medline]
- Ellis SG, Omoigui N, Bittl JA, et al. Analysis and comparison of operator-specific outcomes in interventional cardiology. Circulation. 1996;93:431–439[Abstract/Free Full Text]
- Concato J, Feinstein AR. Monte Carlo methods in clinical research: applications in multivariable analysis. J Invest Med. 1997;45:394–400[Medline]
- Serruys PW, Luijten HE, Beatt KJ, et al. Incidence of restenosis after successful coronary angioplasty: a time-related phenomenon. Circulation. 1988;77:361–371[Abstract/Free Full Text]
- Nobuyoshi M, Kimura T, Nosaka H, et al. Restenosis after successful percutaneous transluminal coronary angioplasty: serial angiographic follow-up in 229 patients. J Am Coll Cardiol. 1988;12:616–623[Abstract]
- Kimura T, Kaburagi S, Tamura T, et al. Remodeling of human coronary arteries undergoing angioplasty or atherectomy. Circulation. 1997;96:475–483[Abstract/Free Full Text]
- Ryan TJ, Faxon DP, Gunnar RM, et al. Guidelines for percutaneous transluminal coronary angioplasty. A report of the American College of Cardiology/American Heart Association Task Force on the assessment of diagnostic and therapeutic cardiovascular procedures. J Am Coll Cardiol. 1988;12:529–545[Medline]
- SAS users guide: statistics, version 6. Cary, NC: SAS Institute; 1991.
- Kastrati A, Neumann F-J, Schomig A. Operator volume and outcome of patients undergoing coronary stent placement. J Am Coll Cardiol. 1998;32:970–976[Abstract/Free Full Text]
- Shook TL, Sun G-W, Burstein S, Eisenhauer AC, Matthews RV. Comparison of percutaneous transluminal coronary angioplasty outcome and hospital costs for low-volume and high-volume operators. Am J Cardiol. 1996;77:331–336[CrossRef][Medline]
- Block PC, Peterson EC, Krone R, et al. Identification of variables needed to risk adjust outcomes of coronary interventions: evidence-based guidelines for efficient data collection. J Am Coll Cardiol. 1998;32:275–282[Abstract/Free Full Text]
This article has been cited by other articles:
|
|
|
|
 
B. A. Vakili, R. Kaplan, and D. L. Brown
Volume-Outcome Relation for Physicians and Hospitals Performing Angioplasty for Acute Myocardial Infarction in New York State
Circulation,
October 30, 2001;
104(18):
2171 - 2176.
[Abstract]
[Full Text]
[PDF]
|
|
|
|
Top
Abstract
Methods