|
|
||||||||||
|
J Am Coll Cardiol, 2002; 39:1104-1112 © 2002 by the American College of Cardiology Foundation |
* San Francisco Heart Institute at Seton Medical Center, Daly City, California 94015, USA
Manuscript received August 13, 2001; revised manuscript received November 29, 2001, accepted January 9, 2002.
* Reprint requests and correspondence: Richard E. Shaw, PhD, FACC, San Francisco Heart Institute at Seton Medical Center, 1900 Sullivan Avenue, Daly City, California 94015, USA.
RichardShaw{at}dochs.org
| Abstract |
|---|
|
|
|---|
BACKGROUND: The 19982000 American College of CardiologyNational Cardiovascular Data Registry (ACCNCDR) dataset was used to overcome limitations of prior risk-adjustment analyses.
METHODS: Data on 100,253 PCI procedures collected at the ACCNCDR between January 1, 1998, and September 30, 2000, were analyzed. A training set/test set approach was used. Separate models were developed for presentation with and without acute myocardial infarction (MI) within 24 h.
RESULTS: Factors associated with increased risk of PCI mortality (with odds ratios in parentheses) included cardiogenic shock (8.49), increasing age (2.61 to 11.25), salvage (13.38) urgent (1.78) or emergent PCI (5.75), pre-procedure intra-aortic balloon pump insertion (1.68), decreasing left ventricular ejection fraction (0.87 to 3.93), presentation with acute MI (1.31), diabetes (1.41), renal failure (3.04), chronic lung disease (1.33); treatment approaches including thrombolytic therapy (1.39) and non-stent devices (1.64); and lesion characteristics including left main (2.04), proximal left anterior descending disease (1.97) and Society for Cardiac Angiography and Interventions lesion classification (1.64 to 2.11). Overall, excellent discrimination was achieved (C-index = 0.89) and application of the model to high-risk patient groups demonstrated C-indexes exceeding 0.80. Patient factors were more predictive in the MI model, while lesion and procedural factors were more predictive in the analysis of non-MI patients.
CONCLUSIONS: A risk adjustment model for in-hospital mortality after PCI was successfully developed using a contemporary multi-center registry. This model is an important tool for valid comparison of in-hospital mortality after PCI.
| ||||||||||||||||||||||||
There have been numerous efforts in recent years to incorporate risk adjustment methodology to evaluate differences in mortality rates for interventional procedures (614). These efforts have been limited by inconsistent definitions of the factors used in the models, small sample sizes, limited geographic representation, inclusion of programs that do not necessarily represent the standard of practice across the country, and patient samples that do not reflect contemporary percutaneous coronary intervention (PCI) practice. The goal of the current study was to analyze the initial experience of the American College of CardiologyNational Cardiovascular Data Registry (ACCNCDR) to develop a risk-adjusted model for mortality associated with PCI. The experience of this registry is described in a companion publication (15). The strengths of the ACCNCDR registry experience include the use of standardized data definitions, data completeness procedures, geographic and institutional diversity, a large sample size, and analysis of contemporary PCI practice. These features offer a significant advantage over previous efforts to develop a risk-adjustment model for in-hospital mortality after PCI.
| Methods |
|---|
|
|
|---|
Data elements entered into the mortality risk model included patient demographic data, cardiac risk factors, coronary revascularization status, anginal status, non-coronary disease processes, angiographic findings and procedural variables (Tables 1 through 5). A variable was constructed combining the lesion codes for the ACC/AHA type A-B-C lesion class along with the presence or absence of an occlusion, based on the work of Krone et al. (16). This classification scheme, referred to as the Society for Cardiac Angiography and Interventions Lesion Class (SCAI LC), produces four categories (I, II, III, IV): Inontype C/patent; IItype C/patent; IIInontype C/occlusion; and IVtype C/occlusion. In a preliminary analysis, this classification system was highly correlated with PCI outcomes. Congestive heart failure was not included in the model, because there was a problem with one of the software vendor packages that allowed out-of-range values to be included in the database that could not be interpreted.
|
|
|
|
|
Risk model development. Univariate analyses were used to identify patient demographic and risk factor (Table 1), cardiac history and anginal status (Table 2), non-coronary disease processes (Table 3), angiographic (Table 4) and procedural (Table 5) factors significantly associated with mortality that were included in the regression model. Of the 32 variables evaluated, hypertension, previous myocardial infarction (MI), previous CABG and lesion in a graft did not achieve a significance level of <0.01 and were not included in the regression model. Hypercholesterolemia was also omitted from the model because of its counterintuitive relationship to the mortality outcome, perhaps related to the advanced sickness of many of the patients treated. Others (8,9) also have noted this peculiarity.
A standard training set/test set approach was used. A randomly generated training set was used to develop the regression model, and the test set consisting of the remaining patients was used to assess the performance of the model against observed mortality results. After the risk factors were determined and their regression weights calculated from the training set, the standard probability formula was applied to the test set to determine the risk of mortality for each patient. The ROC curves were generated for the training set and the test set.
The model was validated in two ways. First, the dataset was ordered using the values for each patients probability of mortality generated from the regression model. The dataset was then divided into deciles of risk, and the observed mortality rate was calculated for each decile. The observed versus the expected mortality was plotted and evaluated using the R-square statistic. The model was also validated by identifying patient subgroups that were known to have high mortality rates. In addition, separate logistic regression models were generated for patients presenting with acute MI within 24 h of PCI and those presenting without acute MI within 24 h.
| Results |
|---|
|
|
|---|
The training set consisted of 50,123 PCI procedures randomly selected from the overall patient population. In this group, 707 deaths occurred (1.4%). Multivariate logistic regression analysis identified PCI indication for shock, increasing age, the need for urgent or emergent PCI, pre-procedure placement of an IABP, decreasing LVEF, acute MI within 24 h of hospital admission, diabetes, renal failure, chronic lung disease, treatment approaches including use of thrombolytics and non-stent devices, and lesion characteristics including presence of left main or proximal left anterior descending (LAD) disease and SCAI lesion class as factors independently associated with in-hospital mortality (Table 6) with a C-index of 0.89, demonstrating excellent model discrimination. The Hosmer-Lemeshow statistic was not significant, indicating little departure from a perfect fit.
|
|
|
|
|
| Discussion |
|---|
|
|
|---|
Prior risk modeling has been limited by inconsistent definitions, small sample sizes, lack of institutional diversity, restricted geographic representation and patient samples that do not reflect contemporary PCI practice. Hannan et al. (6) published results from the New York State mandated registry for PCI. This represented one of the first efforts to develop a mortality risk model using data from several centers. The patient sample that was included, however, reflected an early era in the development of interventional technology and pharmaceutical treatment. More recently, Kimmel et al. (7) analyzed the experience in 10,622 first-time PCI procedures from the Society for Cardiac Angiography and Interventions Registry. OConnor et al. (8) reported on the development of a risk-adjusted mortality model using 15,331 patients who underwent PCI over a three-year period at six regional centers in northern New England. Although the data definitions and collection protocols were well-established and produced high-quality data in both of these studies, the sample sizes were relatively small for model development. In the Kimmel et al. (7) study, the geographic distribution was broad, but the number of centers was small, while the northern New England experience represented a very narrow geographic region. Neither of these studies included patient samples comparable to contemporary PCI cohorts, wherein the proportion of patients receiving stents and glycoprotein IIb/IIIa inhibitors exceeds 75%. Two more recent analyses (13,14) have included patient samples that reflect the widespread use of stents and glycoprotein IIb/IIIa inhibitors, although both were analyses of a single centers experience.
Block et al. (9) combined the experience of eight registries in a pooled meta-analysis of 158,273 cases to identify factors associated with risk of Q-wave MI, emergent cardiac surgery and death. This effort was helpful in providing a broad view of data elements across a large sample of patients. It was limited for the purpose of model development, however, in that source data were not available to apply standard statistical analyses. Other efforts, such as the multi-center study of Ellis et al. (10), the National Heart, Lung, and Blood Percutaneous Transluminal Coronary Angioplasty registries (11) and the New Approaches in Coronary Intervention Registry (12), have utilized risk-adjustment techniques. However, the centers involved in these studies were highly selective and not necessarily representative of a broad assessment of PCI experience.
Comparison with models from other studies.
The variables that were generated from the ACCNCDR mortality model are consistent with a number of other studies published from local databases and registries. OConnor et al. (8) identified increasing age, cardiogenic shock, urgent and emergent procedures, LVEF, pre-procedure IABP placement and attempt of type C lesions as significant predictors in their model. They also noted that heart failure and creatinine levels
2.0 mg/dl were predictors in their model, but these two variables were not included in the analysis of the ACCNCDR experience, although the definition of renal failure in the current analysis was based on similar creatinine levels. The ACCNCDR data analysis included usage of devices, which was not addressed in the OConnor et al. (8) model. The only factor common to both studies that was significant in the OConnor et al. (8) model and that fell out of the ACCNCDR model was the presence of peripheral vascular disease. In an analysis of patients undergoing more contemporary PCI, Resnic et al. (13) identified similar factors, including cardiogenic shock, class 3 or 4 heart failure, left main intervention, tachycardia, chronic renal insufficiency, age
75 years, type B2 or C lesions, acute MI, unstable angina and stent use (a negative relationship to mortality).
Block et al. (9) pooled data from eight different sources and identified a number of factors associated with in-hospital death. Variables they identified that overlap with the current analysis include age, LVEF, acute MI, procedure acuteness, cardiogenic shock, use of IABP, diabetes, renal failure and lesion type. There is remarkable consistency for many of the factors across all databases.
Other studies have focused on the unique relationship of lesion factors to adverse outcomes. Ellis et al. (14) found 10 lesion factors that were related to ischemic complications after PCI. The most significant factors were a non-chronic total occlusion and degenerated saphenous vein graft. In the current analysis, the SCAI LC lesion classification combined the effect of type C and occlusion. However, treatment of a lesion in a vein graft did not have a significant relationship to in-hospital mortality. The assessment of vein graft irregularities in the Ellis et al. (14) study probably represents a more comprehensive analysis of the condition of the saphenous vein graft, which may have enhanced the predictive utility of this variable. Zaacks et al. (22) found that focused characteristics of lesions (presence of thrombus, inability to protect a side branch and degenerated vein graft lesions) were more likely to be related to complications, while the more general classification of lesions using the ACC/AHA A-B-C system were more predictive of procedural success. Several lesion factors that are consistent with other studies emerged in the current analysis as predictors of mortality. One of the most important aspects of the current analysis, however, is the development of separate models for acute MI and non-MI patients. This analysis showed that lesion factors were more highly predictive of in-hospital mortality for non-MI patients than for acute-MI patients.
The utility of risk models. It is important to emphasize that the development of predictive models is as much an art as it is a science. Models are dependent on the quality and accuracy of the data and the relative rate of the outcome event being studied. If the quality of the data is suspect, the modeling process is unpredictable. Likewise, when the outcomes assessed occur infrequently, as in PCI mortality, the modeling process is even more challenging. These limitations of modeling must be kept in mind when evaluating and applying the results of the models presented herein as well as other models developed for PCI outcomes.
Study limitations. There were variables in the ACCNCDR that could not be reliably used for the current analysis. The variable for assessing the status of congestive heart failure had uninterpretable data resulting from a software vendor problem. This problem was corrected, and the heart failure variable will be available for future analyses. The logistic regression approach has an upper limit of predictive capability, with a C-statistic of around 0.87 (23). Techniques such as neural networks are capable of achieving indexes to 0.93 and may play a role in future modeling efforts. It is also not clear to what extent these models built on national datasets can be generalized to local datasets. Comparisons of several models used in cardiac surgery have demonstrated similar predictive capabilities (24,25), but application from one setting to another has limitations (26). These same issues are present for risk-adjusted models developed for PCI mortality.
Perhaps the most significant limitation of the current study is the lack of a systematic approach to auditing the data. Although many consistency checks were instituted in the data collection process, the extent to which these data reflect clinical reality at each institution is not known. It is imperative that future databases used for institutional evaluation be subjected to valid and objective audit processes.
| Conclusions |
|---|
|
|
|---|
| Acknowledgments |
|---|
| References |
|---|
|
|
|---|
This article has been cited by other articles:
![]() |
M. A. Cavender and E. M. Ohman What Do You Need to Know Before Performing a Percutaneous Coronary Intervention? Circulation, August 5, 2008; 118(6): 609 - 611. [Full Text] [PDF] |
||||
![]() |
M. Singh, E. D. Peterson, S. Milford-Beland, J. S. Rumsfeld, and J. A. Spertus Validation of the Mayo Clinic Risk Score for In-Hospital Mortality After Percutaneous Coronary Interventions Using the National Cardiovascular Data Registry Circ Cardiovasc Intervent, August 1, 2008; 1(1): 36 - 44. [Abstract] [Full Text] [PDF] |
||||
![]() |
S. V. Rao, F.-S. Ou, T. Y. Wang, M. T. Roe, R. Brindis, J. S. Rumsfeld, and E. D. Peterson Trends in the Prevalence and Outcomes of Radial and Femoral Approaches to Percutaneous Coronary Intervention: A Report From the National Cardiovascular Data Registry J. Am. Coll. Cardiol. Intv., August 1, 2008; 1(4): 379 - 386. [Abstract] [Full Text] [PDF] |
||||
![]() |
B Kunadian, J Dunning, R Das, A P Roberts, R Morley, A J Turley, D Twomey, J A Hall, R A Wright, A G C Sutton, et al. External validation of established risk adjustment models for procedural complications after percutaneous coronary intervention Heart, August 1, 2008; 94(8): 1012 - 1018. [Abstract] [Full Text] [PDF] |
||||
![]() |
P. Sousa, A. S. Uva, F. Pinto, and on behalf of the investigators of PCI registry of Risk-adjustment model in health outcomes evaluation: a contribution to strengthen assessment towards quality improvement in interventional cardiology Int. J. Qual. Health Care, July 11, 2008; (2008) mzn029v1. [Abstract] [Full Text] [PDF] |
||||
![]() |
L. J. Shaw and J. Narula Cardiovascular Imaging Quality--More Than a Pretty Picture! J. Am. Coll. Cardiol. Img., March 1, 2008; 1(2): 266 - 269. [Full Text] [PDF] |
||||
![]() |
T. B. Ferguson Jr On the Evaluation of Intervention Outcome Risks for Patients With Ischemic Heart Disease Circulation, January 22, 2008; 117(3): 333 - 335. [Full Text] [PDF] |
||||
![]() |
S. B. King III, T. Aversano, W. L. Ballard, R. H. Beekman III, M. J. Cowley, S. G. Ellis, D. P. Faxon, E. L. Hannan, J. W. Hirshfeld Jr, A. K. Jacobs, et al. ACCF/AHA/SCAI 2007 Update of the Clinical Competence Statement on Cardiac Interventional Procedures: A Report of the American College of Cardiology Foundation/American Heart Association/American College of Physicians Task Force on Clinical Competence and Training (Writing Committee to Update the 1998 Clinical Competence Statement on Recommendations for the Assessment and Maintenance of Proficiency in Coronary Interventional Procedures) J. Am. Coll. Cardiol., July 3, 2007; 50(1): 82 - 108. [Full Text] [PDF] |
||||
![]() |
C. K. Haan, S. O'Brien, F. H. Edwards, E. D. Peterson, and T. B. Ferguson Trends in emergency coronary artery bypass grafting after percutaneous coronary intervention, 1994-2003. Ann. Thorac. Surg., May 1, 2006; 81(5): 1658 - 1665. [Abstract] [Full Text] [PDF] |
||||
![]() |
A D Grayson, R K Moore, M Jackson, S Rathore, S Sastry, T P Gray, I Schofield, A Chauhan, F F Ordoubadi, B Prendergast, et al. Multivariate prediction of major adverse cardiac events after 9914 percutaneous coronary interventions in the north west of England Heart, May 1, 2006; 92(5): 658 - 663. [Abstract] [Full Text] [PDF] |
||||
![]() |
P. S. Douglas and R. G. Brindis President's Page: A Question of Quality: Why National Benchmarking? J. Am. Coll. Cardiol., March 7, 2006; 47(5): 1076 - 1078. [Full Text] [PDF] |
||||
![]() |
R. G. Brindis and G. J. Dehmer Continuous Quality Improvement in the Cardiac Catheterization Laboratory: Are the Benefits Worth the Cost and Effort? Circulation, February 14, 2006; 113(6): 767 - 770. [Full Text] [PDF] |
||||
![]() |
M. Moscucci, E. K. Rogers, C. Montoye, D. E. Smith, D. Share, M. O'Donnell, A. Maxwell-Eward, W. L. Meengs, A. C. De Franco, K. Patel, et al. Association of a Continuous Quality Improvement Initiative With Practice and Outcome Variations of Contemporary Percutaneous Coronary Interventions Circulation, February 14, 2006; 113(6): 814 - 822. [Abstract] [Full Text] [PDF] |
||||
![]() |
C. Wu, E. L. Hannan, G. Walford, J. A. Ambrose, D. R. Holmes Jr, S. B. King III, L. T. Clark, S. Katz, S. Sharma, and R. H. Jones A Risk Score to Predict In-Hospital Mortality for Percutaneous Coronary Interventions J. Am. Coll. Cardiol., February 7, 2006; 47(3): 654 - 660. [Abstract] [Full Text] [PDF] |
||||
![]() |
W. S. Weintraub Evaluating the Risk of Coronary Surgery and Percutaneous Coronary Intervention J. Am. Coll. Cardiol., February 7, 2006; 47(3): 669 - 671. [Full Text] [PDF] |
||||
![]() |
K D Dawkins, T Gershlick, M de Belder, A Chauhan, G Venn, P Schofield, D Smith, J Watkins, H H Gray, and Joint Working Group on Percutaneous Coronary Inter Percutaneous coronary intervention: recommendations for good practice and training Heart, December 1, 2005; 91(suppl_6): vi1 - vi27. [Abstract] [Full Text] [PDF] |
||||
![]() |
H. V. Anderson, R. E. Shaw, R. G. Brindis, L. W. Klein, C. R. McKay, M. A. Kutcher, R. J. Krone, M. J. Wolk, S. C. Smith Jr, and W. S. Weintraub Relationship Between Procedure Indications and Outcomes of Percutaneous Coronary Interventions by American College of Cardiology/American Heart Association Task Force Guidelines Circulation, November 1, 2005; 112(18): 2786 - 2791. [Abstract] [Full Text] [PDF] |
||||
![]() |
R. Vanholder, Z. Massy, A. Argiles, G. Spasovski, F. Verbeke, N. Lameire, and for the European Uremic Toxin Work Group (EUTox) Chronic kidney disease as cause of cardiovascular morbidity and mortality Nephrol. Dial. Transplant., June 1, 2005; 20(6): 1048 - 1056. [Abstract] [Full Text] [PDF] |
||||
![]() |
M. Singh, C. S. Rihal, R. J. Lennon, K. N. Garratt, and D. R. Holmes Jr Comparison of Mayo Clinic risk score and American College of Cardiology/American Heart Association lesion classification in the prediction of adverse cardiovascular outcome following percutaneous coronary interventions J. Am. Coll. Cardiol., July 21, 2004; 44(2): 357 - 361. [Abstract] [Full Text] [PDF] |
||||
![]() |
G. J. Dehmer, J. W. Hirshfeld, W. J. Oetgen, K. Mitchell, A. Wells Simon, M. Elma, M. A. Kellett Jr, R. G. Brindis, on behalf of the American College of Cardiology Fo, CathKIT Task Force Members, et al. CathKIT: improving quality in the cardiac catheterization laboratory J. Am. Coll. Cardiol., March 3, 2004; 43(5): 893 - 899. [Full Text] [PDF] |
||||
![]() |
D. A. Halon, S. Adawi, I. Dobrecky-Mery, and B. S. Lewis Importance of increasing age on the presentation and outcome of acute coronary syndromes in elderly patients J. Am. Coll. Cardiol., February 4, 2004; 43(3): 346 - 352. [Abstract] [Full Text] [PDF] |
||||
![]() |
D. E. Cutlip, K. K. L. Ho, R. E. Kuntz, and D. S. Baim Risk assessment for percutaneous coronary intervention: our version of the weather report? J. Am. Coll. Cardiol., December 3, 2003; 42(11): 1896 - 1899. [Full Text] [PDF] |
||||
![]() |
M. J. Sarnak, A. S. Levey, A. C. Schoolwerth, J. Coresh, B. Culleton, L. L. Hamm, P. A. McCullough, B. L. Kasiske, E. Kelepouris, M. J. Klag, et al. Kidney Disease as a Risk Factor for Development of Cardiovascular Disease: A Statement From the American Heart Association Councils on Kidney in Cardiovascular Disease, High Blood Pressure Research, Clinical Cardiology, and Epidemiology and Prevention Hypertension, November 1, 2003; 42(5): 1050 - 1065. [Full Text] [PDF] |
||||
![]() |
M. J. Sarnak, A. S. Levey, A. C. Schoolwerth, J. Coresh, B. Culleton, L. L. Hamm, P. A. McCullough, B. L. Kasiske, E. Kelepouris, M. J. Klag, et al. Kidney Disease as a Risk Factor for Development of Cardiovascular Disease: A Statement From the American Heart Association Councils on Kidney in Cardiovascular Disease, High Blood Pressure Research, Clinical Cardiology, and Epidemiology and Prevention Circulation, October 28, 2003; 108(17): 2154 - 2169. [Full Text] [PDF] |
||||
![]() |
L. W. Klein, P. Block, R. G. Brindis, C. R. McKay, B. D. McCallister, M. Wolk, W. Weintraub, and ACC-NCDR Registry Percutaneous coronary interventions in octogenarians in the American College of Cardiology-National Cardiovascular Data Registry: Development of a nomogram predictive of in-hospital mortality J. Am. Coll. Cardiol., August 7, 2002; 40(3): 394 - 402. [Abstract] [Full Text] [PDF] |
||||
| ||||||||||||||||||||||||||||||||||||||||||||||||||