|
|
||||||||||
|
J Am Coll Cardiol, 2006; 47:654-660, doi:10.1016/j.jacc.2005.09.071
(Published online 28 December 2005). © 2006 by the American College of Cardiology Foundation |




,*
* University at Albany, State University of New York, Albany, New York, USA
St. Josephs Hospital, Syracuse, New York, USA
St. Vincents Hospital and Medical Center, New York, New York, USA
Mayo Clinic, Rochester, Minnesota, USA
|| Fuqua Heart Center/Piedmont Hospital, Atlanta, Georgia, USA
¶ University Hospital of Brooklyn, Brooklyn, New York, USA
# North Shore-LIJ Health System, Manhasset, New York, USA
** Mt. Sinai Medical Center, New York, New York, USA

Duke University Medical Center, Durham, North Carolina, USA.
Manuscript received August 25, 2005; revised manuscript received September 16, 2005, accepted September 20, 2005.
* Reprint requests and correspondence: Dr. Robert H. Jones, Duke Clinical Research Institute, P.O. Box 17969, Durham, North Carolina 27715. (Email: jones060{at}mc.duke.edu).
| Abstract |
|---|
|
|
|---|
BACKGROUND: Risk scores predicting adverse outcomes after PCI have been developed from a single or a small group of hospitals, and their abilities to be generalized to other patient populations might be affected.
METHODS: A logistic regression model was developed to predict in-hospital mortality for PCI using data from 46,090 procedures performed in 41 hospitals in the New York State Percutaneous Coronary Intervention Reporting System in 2002. A risk score was derived from this model and was validated using 2003 data from New York.
RESULTS: The risk score included nine significant risk factors (age, gender, hemodynamic state, ejection fraction, pre-procedural myocardial infarction, peripheral arterial disease, congestive heart disease, renal failure, and left main disease) that were consistent with other reports. The point values for risk factors range from 1 to 9, and the total risk score ranges from 0 to 40. The observed and recalibrated predicted risks in 2003 were highly correlated for all PCI patients as well as for those in the higher-risk subgroup who suffered myocardial infarctions within 24 h before the procedure. The total risk score for mortality is strongly associated with complication rates and length of stay in the 2003 PCI data.
CONCLUSIONS: The risk score accurately predicted in-hospital death for PCI procedures using future New York data. Its performance in other patient populations needs to be further studied.
| ||||||
Risk scores provide useful statistical information in a more clinically useful form than corresponding logistic regression models. Most risk scores have been developed from data from a single or a small group of hospitals, and their abilities to be generalized to other patient populations is suspect (58). In theory, the generalizability of risk scores could be improved when they are derived from a larger patient population. The objective of this study is to develop a risk score system for the prediction of in-hospital mortality for PCI using the data from the New York State Percutaneous Coronary Intervention Reporting System, which contains all PCI procedures performed in the state.
| Methods |
|---|
|
|
|---|
Analysis. A logistic regression model was developed to predict in-hospital mortality using a cross-validation strategy. The first step of the model development was to examine the bivariate relationship between mortality and each pre-procedural risk factor using 2002 PCI data. The risk factors examined included age, gender, body surface area, hemodynamic state, ventricular function, vessels diseased, a variety of comorbidities, and previous interventions. Risk factors that were significantly (p < 0.05) related to mortality were chosen as candidate variables that were used to develop the logistic regression model. Next, the 2002 PCI data were split into two groups of the same population size, with the groups having identical in-hospital mortality rates and similar prevalences of all candidate variables. A stepwise logistic regression model was developed using one group, and significant risk factors (p < 0.1) were used as candidate variables to develop another stepwise logistic regression model in the other group. The variables that remained significant (p < 0.1) were used as candidate variables to fit a logistic regression model using the whole 2002 PCI database. All variables with p values <0.05 were included in the final model. To evaluate the fit of the final logistic regression model, the C-statistic was used to measure discrimination (12) and the Hosmer-Lemeshow test was used to measure calibration (13).
The risk score was developed based on the final logistic regression model using the method described by Sullivan et al. (14). The first step was to break the only continuous variable, age, into four groups: 55 years or younger, 56 to 64 years, 65 to 74 years, and 75 years and older. This was done because the spline functions used for age in the previously mentioned logistic regression model established that the risk of mortality was essentially flat for patients who were under 55 years old, and that risk increased linearly for patients older than 55 years. The youngest age group was used as the base category. Reference values for the other three groups were defined as the differences between their mid-point values and 55. For the oldest age group, the mid-point was between 75 and the 99% percentile for age (88 years), which was used to minimize the influence of extreme values. Then the distance of each age group from the base age category in regression coefficient units was computed by multiplying its reference value by the regression coefficient of age (0.0635). For instance, the distance for the age group of 56 to 64 years was 0.0635 x 5 = 0.3175.
All other risk factors in the model were categorical variables, and the distance between a variable and its base (reference) category in regression coefficient units was equal to the size of the coefficient. Spline functions confirmed that the risk of mortality was essentially constant for patients with ejection fractions above 30% and linear for patients with ejection fractions below 30%, and this led to the choice of three categories (one the base category with a zero score) for ejection fraction in the risk index. The definitions of all risk factors are included in the .
The constant of the scoring system was defined as the increase of risk in regression units associated with an increase of five years in age (i.e., 0.3175). This constant corresponded to one point in the risk score system. For each risk factor, its distance from the base category in regression coefficient units was divided by this constant and rounded to the nearest integer to get its point value. For example, the point value for the risk factor "left main coronary artery disease" was calculated as 0.8456/0.3175 and rounded to 3.
A patients total risk score was calculated by adding up the points for all existing risk factors. For each risk score, the predicted risk of in-hospital mortality (p) was calculated using the method described by Sullivan et al. (14),
|
|
|
The correspondence between the risk score and the probability of complications as well as length of stay among patients undergoing PCI in 2003 was also examined. Complications collected in the data system include stroke, myocardial infarction (MI), acute occlusion in the targeted lesion or in a significant side branch, vessel injury at the catheter entry site requiring intervention, renal failure, emergency cardiac surgery, stent thrombosis, and emergency PCI.
All statistical analyses except the hierarchical logistic regression analyses were conducted in SAS version 9.1 (SAS Institute, Cary, North Carolina).
| Results |
|---|
|
|
|---|
Table 2 presents the point scores for all risk factors in the logistic regression model. Point scores ranged from 1 for the age group 56 to 64 years and for female gender to 9 for shock and for acute MI with stent thrombosis. The minimum total risk score was 0 for a patient without any risk factors listed in Table 2, and the maximum possible score is 40. The predicted probabilities of in-hospital death for each risk score ranged from 0.05% for a patient with a score of 0 to 99.36% for a patient with the highest possible score of 40 (Table 3).
|
|
Figure 1 presents the distribution of patients and the correspondence between observed and predicted risks by risk score group using 2002 PCI data. About half of the patients had total scores of 4 or less with a predicted risk of death <0.2%. Only 0.5% of the patients had total scores of at least 19, and the highest observed score was 31. In general, the observed and predicted risks were very close to each other even for total scores of at least 13 as evidenced by the fact that the predicted risks were within the 95% CIs of the observed risks. Figure 2 shows that the correspondence between observed and predicted risks was also good for the 4,974 patients who suffered acute MI, a higher-risk subgroup that had an observed in-hospital mortality rate of 2.87%.
|
|
|
|
|
|
| Discussion |
|---|
|
|
|---|
The risk score developed in this study was designed to be a handy tool to predict the risk of in-hospital mortality for PCI based on a patients pre-procedural risk factors. The point score assigned to each risk factor was derived from a well-fit logistic regression model, which included risk factors consistent with other published models (18). For example, all risk factors used in this study to compute the risk score were also included in at least one of four other recent risk score systems. Renal failure was included in all four other systems; age, shock, and history of MI were included in three other systems (58). Also, shock and acute MI had the highest scores in the other systems that included them (58).
It should be noted that the risk factors in the New York PCI risk score are very similar to the ones used in a coronary artery bypass graft surgery risk score based on New York data (15). Seven of the nine variables (age, female gender, hemodynamic state, ejection fraction, pre-procedural MI, peripheral arterial disease, and renal failure) in the PCI score are the same. The PCI risk score also includes congestive heart failure and left main disease, and the coronary artery bypass graft surgery risk score also includes chronic obstructive pulmonary disease, extensively calcified ascending aorta, and previous open heart operations.
When this risk score (or any risk score) is used in other patient populations for purposes of predicting mortality for individual patients, we recommend, if possible, recalibrating the predicted risk to achieve optimal accuracy. This is because underlying outcomes may be quite different between the development patient population and other populations. Recalibration requires multiplying the predicted risk for each risk score presented in Table 3 by the ratio of the observed mortality rate in a new population and its expected mortality rate predicted by the 2002 New York logistic regression model. The expected mortality rate could also be approximated by using the predicted risks in Table 3 if computing the predicted probability using a logistic regression model is not feasible. It was demonstrated in this study that the observed and recalibrated predicted risks were close to each other for each risk score in the 2003 PCI data for all PCI patients as well as for those who suffered acute MIs before the procedure.
It was also observed that the risk score proposed in this study was positively correlated with the complication rate and the length of stay in the 2003 PCI data. This finding indicates it can be used as an indicator of the chance of experiencing complications and longer lengths of stay even though it was derived from a logistic regression predicting in-hospital death for PCI patients.
It is worth emphasizing that, although a good risk score can predict adverse outcomes after PCI procedures well on average, it is not designed to precisely predict a single patients risk. In addition, an individual patient may have other medical problems not included in the model that are rare but associated with significant risk (e.g., hypoxic coma from a cardiac arrest or newly diagnosed leukemia). Therefore, a risk score should be used as a tool to help physicians and patients to make informed decisions, but not to foresee a patients specific outcome.
We look forward to attempts to test the generalizability of the risk score developed in this study by testing it on other patient populations, and to attempt to compare its accuracy with various other risk scores (58).
| Appendix |
|---|
|
|
|---|
| Acknowledgments |
|---|
| References |
|---|
|
|
|---|
Related articles in JACC:
This article has been cited by other articles:
![]() |
T. B. Ferguson Jr On the Evaluation of Intervention Outcome Risks for Patients With Ischemic Heart Disease Circulation, January 22, 2008; 117(3): 333 - 335. [Full Text] [PDF] |
||||
![]() |
M. Singh, B. J. Gersh, S. Li, J. S. Rumsfeld, J. A. Spertus, S. M. O'Brien, R. M. Suri, and E. D. Peterson Mayo Clinic Risk Score for Percutaneous Coronary Intervention Predicts In-Hospital Mortality in Patients Undergoing Coronary Artery Bypass Graft Surgery Circulation, January 22, 2008; 117(3): 356 - 362. [Abstract] [Full Text] [PDF] |
||||
![]() |
S. R. Dixon, C. L. Grines, and W. W. O'Neill The Year in Interventional Cardiology J. Am. Coll. Cardiol., July 17, 2007; 50(3): 270 - 285. [Full Text] [PDF] |
||||
![]() |
S. B. King III, T. Aversano, W. L. Ballard, R. H. Beekman III, M. J. Cowley, S. G. Ellis, D. P. Faxon, E. L. Hannan, J. W. Hirshfeld Jr, A. K. Jacobs, et al. ACCF/AHA/SCAI 2007 Update of the Clinical Competence Statement on Cardiac Interventional Procedures: A Report of the American College of Cardiology Foundation/American Heart Association/American College of Physicians Task Force on Clinical Competence and Training (Writing Committee to Update the 1998 Clinical Competence Statement on Recommendations for the Assessment and Maintenance of Proficiency in Coronary Interventional Procedures) J. Am. Coll. Cardiol., July 3, 2007; 50(1): 82 - 108. [Full Text] [PDF] |
||||
![]() |
R. H. Jones The Year in Cardiovascular Surgery J. Am. Coll. Cardiol., May 8, 2007; 49(18): 1887 - 1898. [Full Text] [PDF] |
||||
![]() |
E. L. Hannan, C. Wu, E. V. Bennett, R. E. Carlson, A. T. Culliford, J. P. Gold, R. S.D. Higgins, C. R. Smith, and R. H. Jones Risk Index for Predicting In-Hospital Mortality for Cardiac Valve Surgery Ann. Thorac. Surg., March 1, 2007; 83(3): 921 - 929. [Abstract] [Full Text] [PDF] |
||||
![]() |
H. M. Krumholz and F. A. Masoudi The Year in Epidemiology, Health Services Research, and Outcomes Research J. Am. Coll. Cardiol., November 7, 2006; 48(9): 1886 - 1895. [Full Text] [PDF] |
||||
![]() |
A Siotia and J Gunn Risk scoring for percutaneous coronary intervention: let's do it! Heart, November 1, 2006; 92(11): 1539 - 1540. [Full Text] [PDF] |
||||
![]() |
Risk Scoring for In-Hospital Mortality After PCI and CABG Journal Watch Cardiology, March 16, 2006; 2006(316): 6 - 6. [Full Text] |
||||
![]() |
W. S. Weintraub Evaluating the Risk of Coronary Surgery and Percutaneous Coronary Intervention J. Am. Coll. Cardiol., February 7, 2006; 47(3): 669 - 671. [Full Text] [PDF] |
||||
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| HOME | SUBSCRIPTIONS | CURRENT ISSUE | PAST ISSUES | CARDIOSOURCE | SEARCH | HELP | FEEDBACK |