|
|
||||||||||
|
J Am Coll Cardiol, 1999; 34:692-697 © 1999 by the American College of Cardiology Foundation |











* University of Michigan Medical Center, Ann Arbor, Michigan, USA
St. Vincent Hospital, Darlinghurst, Australia
Cleveland Clinic, Cleveland, Ohio, USA
Dartmouth-Hitchcock Medical Center, Lebanon, New Hampshire, USA
Manuscript received August 28, 1998; revised manuscript received April 7, 1999, accepted May 16, 1999.
Reprint requests and correspondence: Dr. Mauro Moscucci, Heart Care Program, University of Michigan Medical Center, B1, Room F245, 1500 East Medical Center Drive, Ann Arbor, Michigan 48109-0022
| Abstract |
|---|
|
|
|---|
We sought to validate recently proposed risk adjustment models for in-hospital percutaneous transluminal coronary angioplasty (PTCA) mortality on an independent data set of high risk patients undergoing PTCA.
BACKGROUND
Risk adjustment models for PTCA mortality have recently been reported, but external validation on independent data sets and on high risk patient groups is lacking.
METHODS
Between July 1, 1994 and June 1, 1996, 1,476 consecutive procedures were performed on a high risk patient group characterized by a high incidence of cardiogenic shock (3.3%) and acute myocardial infarction (14.3%). Predictors of in-hospital mortality were identified using multivariate logistic regression analysis. Two external models of in-hospital mortality, one developed by the Northern New England Cardiovascular Disease Study Group (model NNE) and the other by the Cleveland Clinic (model CC), were compared using receiver operating characteristic (ROC) curve analysis.
RESULTS
In this patient group, an overall in-hospital mortality rate of 3.4% was observed. Multivariate regression analysis identified risk factors for death in the hospital that were similar to the risk factors identified by the two external models. When fitted to the data set, both external models had an area under the ROC curve >0.85, indicating overall excellent model discrimination, and both models were accurate in predicting mortality in different patient subgroups. There was a trend toward a greater ability to predict mortality for model NNE as compared with model CC, but the difference was not significant.
CONCLUSIONS
Predictive models for PTCA mortality yield comparable results when applied to patient groups other than the one on which the original model was developed. The accuracy of the two models tested in adjusting for the relatively high mortality rate observed in this patient group supports their application in quality assessment or quality improvement efforts.
| ||||||||||||||||
| Methods |
|---|
|
|
|---|
Statistical analysis. Data are expressed as the mean value ± SD or as a percentage. Statistical analysis was performed using Stata (Stata Corporation, College Station, Texas). To assess the consistency of variable selection between different models and different patient groups, univariate predictors of in-hospital death were identified using logistic regression analysis. Independent predictors of death in the hospital were identified using stepwise multivariate logistic regression analysis of a model containing univariate factors (p < 0.05). The internal model was tested for goodness-of-fit using the Hosmer-Lemeshow statistic (16).
Validation of external models. Predicted probabilities of in-hospital death for individual patients were calculated using 1) a risk adjustment logistic regression model developed at the Cleveland Clinic (17) from a multi-institutional data base including 12,985 procedures (model CC) and 2) a risk adjustment logistic regression model developed by the Northern New England (NNE) Cardiovascular Disease Study Group (18) from a multi-institutional data base including 15,331 procedures (model NNE) (Table 1). The predicted probabilities for the whole population and for various subgroups were then calculated.
|
| Results |
|---|
|
|
|---|
|
Validation of external models. Predicted probabilities of in-hospital death estimated using the two external models are shown in Table 3. There was a trend for model CC to underestimate the risk of death for each patient subgroup. Receiver operating characteristic curve analysis revealed an area under the ROC curve of 0.88 ± 0.03 for model NNE and an area of 0.85 ± 0.03 for model CC (p = 0.09) (Fig. 1). The overall calibration was excellent for both models, and the Hosmer-Lemeshow statistic was not significant, indicating little departure from a perfect fit (Fig. 2).
|
|
|
| Discussion |
|---|
|
|
|---|
In agreement with what has been shown by other investigators, higher mortality rates were observed in elderly patients, in patients presenting with AMI, in women, in patients with multivessel disease and in patients undergoing emergency revascularization procedures (1014). In particular, the variables identified in the internal logistic regression model as independent predictors of in-hospital death were similar to the variables identified by the two external models, supporting overall consistency in variable selection across different patient groups. In addition, these variables were identical to variables included in a list that was recently developed from combining eight large multicenter data bases (28), and which has been proposed as a tool for efficient data collection to risk adjust outcomes of coronary interventions.
Validation of the two external models. Both external models had a high area under the ROC curve (e.g., >0.8), indicating that they had excellent discrimination in this high risk patient group. The area under the ROC curve of model NNE of 0.88 observed in this study was in fact similar to the area under the ROC curve observed in an internal validation set of 6,037 patients during the development of this model (area 0.88). The area under the ROC curve for model CC was also similar to the area under the ROC curve reported during its development (17) (area 0.84). The comparison of the two models on different subsets of patients and on the overall patient population suggested that the NNE model performed slightly better than model CC in predicting mortality, although the difference of the area under the ROC curve was not statistically significant.
Differences in model performance. The slight difference in model performance can be explained on the basis of differences in variables entered in each model and their coefficients. Age, AMI, cardiogenic shock and lesion severity were variables common to both models, whereas EF, emergent or urgent priority, renal insufficiency, history of congestive heart failure, history of peripheral vascular disease and intraaortic balloon pump requirement were variables present in the NNE model only. The slight difference in model performance may also reflect baseline differences between the cohorts from which each model was derived and the present data set. For example, our patient group was characterized by a high incidence of cardiogenic shock and AMI (3% and 14%, respectively). In the data base used for the development of model CC, the overall incidence of AMI was 6.9% and the overall mortality rate was 1.3%. In the data base used for the development of the NNE model, the overall incidence of cardiogenic shock was only 0.6% and the observed mortality rate was 1.08%. Although the observed mortality in the group of patients with cardiogenic shock in this study is similar to mortality rates reported by other registry studies and in the recently completed SHOCK (SHould we emergently revascularize Occluded Coronaries for cardiogenic shocK?) trial (29,30), both models had a tendency to underestimate the mortality rate in this group of patients.
Preprocedural morbidity and PTCA mortality. It is important to underscore the fact that the high mortality rate observed in the group of patients presenting with cardiogenic shock is more likely a reflection of the natural history of the disease rather than a consequence of the procedure itself. In support of this statement, the mortality rate observed in our group of patients presenting with cardiogenic shock was similar to the mortality rate observed in the PTCA group of the SHOCK trial, and it was lower than the mortality rate observed in the medical treatment arm of this study (30).
The issue of preprocedural morbidity becomes rather important when trying to differentiate between "procedure-related deaths" and "disease-related deaths." As of today, death after PTCA is a relatively rare event, and most of the deaths are related to preprocedural morbidity rather than to the procedure itself. Few deaths will be related to an interaction between preprocedural morbidity and the procedure itself, and very few of them will be secondary to the procedure alone. The value of risk adjustment resides in its ability to identify and to "correct" for those comorbidities that affect the relation between observed outcomes and the procedure itself.
Implications of this analysis. The important finding of this study is that despite some differences in variable selection, the two external models performed well in predicting mortality in this patient group. In particular, both models were rather accurate in adjusting for comorbidities and accounting for the relatively high mortality rate observed, thus adding supportive mathematic evidence to the initial impression of a patient group at higher risk of death in the hospital. Thus, our study shows that models that have been developed from large multi-institutional data bases can perform well also when applied to smaller, high risk, single-center patient populations. This finding is particularly relevant because models that are most effective in mathematically accounting for variation in outcomes allow physicians and institutions greater confidence that the relation between observed outcomes and procedures is influenced by explanatory variables related to case-mix rather than by other unrelated variables. In addition, the same models ensure that sheer chance is being minimized as the explanation for changing rates of outcomes during quality improvement efforts. Thus, for internal quality assessment and quality improvement purposes that typically utilize such formulas to generate expected versus observed outcomes over a given time frame (31), institutions should identify the model that best fits their patient group and their data collection process.
Study limitations. A perceived potential limitation of this analysis is that we did not validate the internal model on an independent data set. However, the internal model was generated only for comparison purposes and to determine consistency of variable selection across different patient groups, rather than to develop a new local model. Second, both the CC model and the NNE model were validated on a relatively small number of patients from a single center. Therefore, we should be cautious in generalizing these findings. Finally, although elective or bailout stenting was available at our institution during the study period, we cannot exclude that a more widespread use of elective stenting could result in a further decrease in the observed complications rates. Thus, risk adjustment models should be viewed as a moving target, and they should be refined over time according to further advancements in medical technology.
Conclusions. Despite differences in variable selection, predictive models for PTCA mortality yield comparable results when applied to patient groups other than the one on which the original model was developed. The accuracy of the two models tested in adjusting for the relatively high mortality rate observed in this patient group supports their application in quality assessment or quality improvement efforts.
| References |
|---|
|
|
|---|
This article has been cited by other articles:
![]() |
B. Kunadian, J. Dunning, A. P Roberts, R. Morley, D. Twomey, J. A Hall, A. G C Sutton, R. A Wright, D. F Muir, and M. A d. Belder Cumulative funnel plots for the early detection of interoperator variation: retrospective database analysis of observed versus predicted results of percutaneous coronary intervention BMJ, April 26, 2008; 336(7650): 931 - 934. [Abstract] [Full Text] [PDF] |
||||
![]() |
H. S. Gurm, D. E. Smith, J. S. Collins, D. Share, A. Riba, A. J. Carter, T. LaLonde, E. Kline-Rogers, M. O'Donnell, H. Changezi, et al. The relative safety and efficacy of abciximab and eptifibatide in patients undergoing primary percutaneous coronary intervention insights from a large regional registry of contemporary percutaneous coronary intervention. J. Am. Coll. Cardiol., February 5, 2008; 51(5): 529 - 535. [Abstract] [Full Text] [PDF] |
||||
![]() |
A Oudot, P G Steg, N Danchin, G Dentan, M Zeller, P Sicard, P Buffet, Y Laurent, L Janin-Manificat, I L'Huillier, et al. Impact of chronic oral anticoagulation on management and outcomes of patients with acute myocardial infarction: data from the RICO survey Heart, August 1, 2006; 92(8): 1077 - 1083. [Abstract] [Full Text] [PDF] |
||||
![]() |
A D Grayson, R K Moore, M Jackson, S Rathore, S Sastry, T P Gray, I Schofield, A Chauhan, F F Ordoubadi, B Prendergast, et al. Multivariate prediction of major adverse cardiac events after 9914 percutaneous coronary interventions in the north west of England Heart, May 1, 2006; 92(5): 658 - 663. [Abstract] [Full Text] [PDF] |
||||
![]() |
W. S. Weintraub Evaluating the Risk of Coronary Surgery and Percutaneous Coronary Intervention J. Am. Coll. Cardiol., February 7, 2006; 47(3): 669 - 671. [Full Text] [PDF] |
||||
![]() |
C. R. Narins, A. M. Dozier, F. S. Ling, and W. Zareba The Influence of Public Reporting of Outcome Data on Medical Decision Making by Physicians Arch Intern Med, January 10, 2005; 165(1): 83 - 87. [Abstract] [Full Text] [PDF] |
||||
![]() |
F. S. Resnic, K. H. Zou, D. V. Do, G. Apostolakis, and L. Ohno-Machado Exploration of a Bayesian Updating Methodology to Monitor the Safety of Interventional Cardiovascular Procedures Med Decis Making, August 1, 2004; 24(4): 399 - 407. [Abstract] [PDF] |
||||
![]() |
M. A. Qureshi, R. D. Safian, C. L. Grines, J. A. Goldstein, D. C. Westveer, S. Glazier, M. Balasubramanian, and W. W. O'Neill Simplified scoring system for predicting mortality after percutaneous coronary intervention J. Am. Coll. Cardiol., December 3, 2003; 42(11): 1890 - 1895. [Abstract] [Full Text] [PDF] |
||||
![]() |
M. Moscucci, K.A.A. Fox, C. P. Cannon, W. Klein, J. Lopez-Sendon, G. Montalescot, K. White, R.J. Goldberg, and for the GRACE Investigators Predictors of major bleeding in acute coronary syndromes: the Global Registry of Acute Coronary Events (GRACE) Eur. Heart J., October 2, 2003; 24(20): 1815 - 1823. [Abstract] [Full Text] [PDF] |
||||
![]() |
D. R. Holmes, F. Selzer, J. M. Johnston, S. F. Kelsey, R. Holubkov, H. A. Cohen, D. O. Williams, and K. M. Detre Modeling and Risk Prediction in the Current Era of Interventional Cardiology: A Report From the National Heart, Lung, and Blood Institute Dynamic Registry Circulation, April 15, 2003; 107(14): 1871 - 1876. [Abstract] [Full Text] [PDF] |
||||
![]() |
K. J. Jenkins, K. Gauvreau, J. W. Newburger, T. L. Spray, J. H. Moller, and L. I. Iezzoni Consensus-based method for risk adjustment for surgery for congenital heart disease J. Thorac. Cardiovasc. Surg., January 1, 2002; 123(1): 110 - 118. [Abstract] [Full Text] [PDF] |
||||
![]() |
M. Moscucci, E. Kline-Rogers, D. Share, M. O'Donnell, A. Maxwell-Eward, W. L. Meengs, P. Kraft, A. C. DeFranco, J. L. Chambers, K. Patel, et al. Simple Bedside Additive Tool for Prediction of In-Hospital Mortality After Percutaneous Coronary Interventions Circulation, July 17, 2001; 104(3): 263 - 268. [Abstract] [Full Text] [PDF] |
||||
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| HOME | SUBSCRIPTIONS | CURRENT ISSUE | PAST ISSUES | CARDIOSOURCE | SEARCH | HELP | FEEDBACK |