|
|
||||||||||
|
J Am Coll Cardiol, 2001; 37:992-997 © 2001 by the American College of Cardiology Foundation |




* Institute for Clinical Evaluative Sciences, Toronto, Ontario, Canada
e Department of Medicine and Public Health Sciences, University of Toronto, Toronto, Ontario, Canada
Manitoba Centre for Health Policy and Evaluation, Department of Community Health Sciences, Faculty of Medicine, University of Manitoba, Winnipeg, Manitoba, Canada
National Bureau of Economic Research, Stanford, California, USA
Center for Primary Care and Outcomes Research, Department of Medicine, Stanford University, Stanford, California, USA
Manuscript received March 31, 2000; revised manuscript received August 2, 2000, accepted December 14, 2000.
Reprint requests and correspondence: Dr. Jack V. Tu, Institute for Clinical Evaluative Sciences, G-106, 2075 Bayview Avenue, Toronto, Ontario, M4N 3M5, Canada
tu{at}ices.on.ca
| Abstract |
|---|
|
|
|---|
To develop and validate simple statistical models that can be used with hospital discharge administrative databases to predict 30-day and one-year mortality after an acute myocardial infarction (AMI).
BACKGROUND
There is increasing interest in developing AMI "report cards" using population-based hospital discharge databases. However, there is a lack of simple statistical models that can be used to adjust for regional and interinstitutional differences in patient case-mix.
METHODS
We used linked administrative databases on 52,616 patients having an AMI in Ontario, Canada, between 1994 and 1997 to develop logistic regression statistical models to predict 30-day and one-year mortality after an AMI. These models were subsequently validated in two external cohorts of AMI patients derived from administrative datasets from Manitoba, Canada, and California, U.S.
RESULTS
The 11-variable Ontario AMI mortality prediction rules accurately predicted mortality with an area under the receiver operating characteristic (ROC) curve of 0.78 for 30-day mortality and 0.79 for one-year mortality in the Ontario dataset from which they were derived. In an independent validation dataset of 4,836 AMI patients from Manitoba, the ROC areas were 0.77 and 0.78, respectively. In a second validation dataset of 112,234 AMI patients from California, the ROC areas were 0.77 and 0.78 respectively.
CONCLUSIONS
The Ontario AMI mortality prediction rules predict quite accurately 30-day and one-year mortality after an AMI in linked hospital discharge databases of AMI patients from Ontario, Manitoba and California. These models may also be useful to outcomes and quality measurement researchers in other jurisdictions.
| ||||||||||||||
for patient case-mix differences between institutions. A few statistical models that have already been implemented using administrative databases to predict AMI mortality are not widely used because they require many variables and complex models and often have not been validated in other jurisdictions (3,5,6).
In Ontario, we published the first hospital-specific AMI "report card" in Canada in 1999 (7). This report contained information on the 30-day and one-year risk-adjusted mortality rates for 52,616 AMI patients at 167 hospitals in Ontario between April 1, 1994, and March 31, 1997. As part of the development of the Ontario AMI report, we created a simple 11-variable prediction rule using the secondary diagnosis fields in the Ontario hospital discharge data bases to adjust for regional and interinstitutional differences in AMI case mix. To evaluate the potential usefulness of this model to clinicians and researchers in other jurisdictions, we tested the model in two completely independent datasets of AMI patients in another Canadian province, Manitoba, and a large area of the U.S., the state of California. This study describes the derivation and validation of the "Ontario AMI mortality prediction rules."
| Methods |
|---|
|
|
|---|
Two independent cohorts of AMI patients were created using similar methods from administrative databases in Manitoba at the Manitoba Centre for Health Care Policy and Evaluation, and in California from the California Office of Statewide Health Planning and Development hospital discharge database.
In the California dataset, the "most responsible diagnosis" is specified as the "principal" diagnosis, and secondary diagnosis codes do not distinguish between complications and comorbid conditions.
Inclusion/exclusion criteria. Similar inclusion/exclusion criteria were used to create the linked AMI datasets in the three jurisdictions. Patients were included in the AMI cohorts if they were admitted with a "most responsible" diagnosis (ICD-9 code 410) of AMI in Ontario or Manitoba, or a "principal" diagnosis of AMI in California. Previous studies have shown the similarities of these two types of diagnoses (8). The exclusion criteria included patients admitted to a noncardiac surgical service, those admitted as transfers from another acute care facility, those admitted with an AMI in the previous year, those discharged alive with a total length of stay of less than four days and those whose AMI was coded as an in-hospital complication. Transferred patients were only counted once based on their first admission, with subsequent admissions linked to the first one. The rationale for these criteria are described elsewhere (9). After these inclusion/exclusion criteria were applied, 52,616 patients were left in the Ontario AMI cohort. A comparable cohort was created using the Manitoba hospital discharge data over the same time frame and yielded a total of 4,386 AMI patients. The California cohort for the same time period consisted of 112,234 AMI patients.
Potential risk factors for prediction model development. Forty-three potential candidate variables in addition to age and gender were considered for inclusion in the AMI mortality prediction rules (Table 1). These candidate variables were taken from a list of risk factors used to develop previous report cards in the California Hospital Outcomes Project and Pennsylvania Health Care Cost Containment Council AMI "report card" projects (3,5). Each of these comorbidities was created using appropriate ICD-9 codes from the 15 secondary diagnosis fields in OMID. The Ontario discharge data are based on ICD-9 codes rather than ICD-9-CM codes used in the U.S., so the U.S. codes were truncated. Some risk factors used in these two projects do not have an ICD-9 coding analog (e.g., infarct subtype, race) and therefore were not included in our analysis. The frequency of each of these 43 comorbidities was calculated, and any comorbidity with a prevalence of <1% was excluded from further analysis. Comorbidities that the authors felt were not clinically plausible predictors of AMI mortality were also excluded. The remaining variables were then entered into a multivariate logistic regression model and backward stepwise regression was used to eliminate variables until only variables significant at the p < 0.05 level were left in the final model. The discrimination of the resulting models was calculated by measuring the area under the receiver operating characteristic (ROC) curve (10).
|
| Results |
|---|
|
|
|---|
|
Regression coefficients. Table 3 shows the logistic regression coefficients and associated odds ratio (OR) with 95% confidence intervals for both the 30-day and one-year mortality prediction rules. The presence of shock at hospital admission was the strongest predictor of mortality at 30 days (OR = 22.31, 95% CI, 19.30 to 25.79) followed by age >75 years (OR = 12.24, 95% CI, 10.18 to 14.71).
|
|
|
| Discussion |
|---|
|
|
|---|
Other AMI prediction rules. Several prediction rules have been developed by other investigators to predict AMI mortality using administrative databases. However, each of these models has a number of limitations. Normand et al. (6) developed a 40-variable prediction rule using the U.S. Medicare claims database. However, the performance of this model was only 0.72 in terms of its ROC curve area for two-year mortality in the dataset from which it was derived. The developers of the Pennsylvania and California report cards have also developed prediction rules using their administrative databases. However, in both jurisdictions two separate models are required. The Pennsylvania group requires one model for direct admissions and a separate model for transferred-in patients (3). For the California report card, separate models were developed for patients with no prior hospital admissions versus those with prior hospital admissions (5). In contrast, the Ontario AMI mortality prediction rules can be applied to all AMI patients and predict both 30-day and one-year mortality after an AMI using the same set of predictor variables.
Clinical prediction rules. Although AMI outcome prediction rules developed using clinical data remain the ultimate "gold standard" for risk adjustment, it is very expensive and time-consuming to collect these data on a population basis. For these reasons, it is likely that most report cards on AMI care will continue to be developed using routinely collected hospital discharge databases. A recent study using data abstracted from the charts of elderly AMI patients in the U.S. Cardiovascular Cooperative Project yielded a prediction rule with an ROC curve area of 0.79 for 30-day mortality (12), which is only slightly superior to that which we were able to achieve using administrative data. Furthermore, we were also able to demonstrate that this disease-specific prediction rule predicts 30-day AMI mortality with a higher ROC curve area than the Charlson comorbidity index, which is the most commonly used method of adjusting for comorbid conditions using administrative databases (13).
Strengths and limitations of the Ontario rules. The Ontario AMI prediction rules have several strengths. First, they use a relatively small number of variables that can be easily generated using the appropriate ICD-9 codes from hospital discharge databases. The variables in the model are clinically sensible and are similar to those found in other studies. Second, although factors, such as blood pressure at presentation and type of infarct, are not included in the model, other variables in the current model (i.e., shock, congestive heart failure) may be correlated with these factors and contribute to the models overall predictive performance. Third, the model has been externally validated in two completely different jurisdictions from which it was derived, which represents a rigorous test of its potential generalizability. Fourth, the rules predict both 30-day and one-year mortality, whereas most other models were designed only to predict short-term mortality.
The Ontario AMI mortality prediction rules also have their limitations. First, we were not able to directly compare their predictions against those that would occur with a prediction rule derived from clinical data. Second, it remains to be established whether risk-adjusted mortality rates calculated using this rule are a marker of better quality in-hospital care (e.g., higher rates of use of aspirin, beta-blockers, thrombolytics). These types of studies are planned in the future.
In summary, we have developed and validated the Ontario AMI mortality prediction rules: simple logistic regression models that predict 30-day and one-year mortality after an AMI using variables that can be easily generated from hospital discharge administrative databases. The models are easy to use, have clinical sensibility and have been externally validated. These models were recently used to generate the first hospital-specific AMI report card in Ontario, Canadas largest province. They have also been recently adopted for use in the Technological Change in Health Care (TECH) project, which involves the comparison of AMI care using administrative databases from 16 countries around the world (14). We believe the models will also likely prove useful to clinicians and outcomes researchers in other jurisdictions around the world.
| Acknowledgments |
|---|
| Footnotes |
|---|
| References |
|---|
|
|
|---|
This article has been cited by other articles:
![]() |
J. S. Alpert A Plethora of Prognostic Pearls Circulation, September 23, 2008; 118(13): 1312 - 1313. [Full Text] [PDF] |
||||
![]() |
S. M. Singh, P. C. Austin, A. Chong, and D. A. Alter Coronary Angiography Following Acute Myocardial Infarction in Ontario, Canada Arch Intern Med, April 23, 2007; 167(8): 808 - 813. [Abstract] [Full Text] [PDF] |
||||
![]() |
D. T. Ko, H. M. Krumholz, Y. Wang, J. M. Foody, F. A. Masoudi, E. P. Havranek, J. J. You, D. A. Alter, T. A. Stukel, A. M. Newman, et al. Regional Differences in Process of Care and Outcomes for Older Acute Myocardial Infarction Patients in the United States and Ontario, Canada Circulation, January 16, 2007; 115(2): 196 - 203. [Abstract] [Full Text] [PDF] |
||||
![]() |
J. N. Rasmussen, A. Chong, and D. A. Alter Relationship Between Adherence to Evidence-Based Pharmacotherapy and Long-term Mortality After Acute Myocardial Infarction JAMA, January 10, 2007; 297(2): 177 - 186. [Abstract] [Full Text] [PDF] |
||||
![]() |
K. A A Fox, O. H Dabbous, R. J Goldberg, K. S Pieper, K. A Eagle, F. Van de Werf, A. Avezum, S. G Goodman, M. D Flather, F. A Anderson Jr, et al. Prediction of risk of death and myocardial infarction in the six months after presentation with acute coronary syndrome: prospective multinational observational study (GRACE) BMJ, November 25, 2006; 333(7578): 1091 - 1091. [Abstract] [Full Text] [PDF] |
||||
![]() |
P. C. Austin The impact of unmeasured clinical variables on the accuracy of hospital report cards: a monte carlo study. Med Decis Making, September 1, 2006; 26(5): 447 - 466. [Abstract] [PDF] |
||||
![]() |
G. H. Gislason, S. Jacobsen, J. N. Rasmussen, S. Rasmussen, P. Buch, J. Friberg, T. K. Schramm, S. Z. Abildstrom, L. Kober, M. Madsen, et al. Risk of Death or Reinfarction Associated With the Use of Selective Cyclooxygenase-2 Inhibitors and Nonselective Nonsteroidal Antiinflammatory Drugs After Acute Myocardial Infarction Circulation, June 27, 2006; 113(25): 2906 - 2913. [Abstract] [Full Text] [PDF] |
||||
![]() |
J. N Rasmussen, S. Rasmussen, G. H Gislason, P. Buch, S. Z Abildstrom, L. Kober, M. Osler, F. Diderichsen, C. Torp-Pedersen, and M. Madsen Mortality after acute myocardial infarction according to income and education. J. Epidemiol. Community Health, April 1, 2006; 60(4): 351 - 356. [Abstract] [Full Text] [PDF] |
||||
![]() |
B A Williams, R S Wright, J G Murphy, E S Brilakis, G S Reeder, and A S Jaffe A new simplified immediate prognostic risk score for patients with acute myocardial infarction Emerg. Med. J., March 1, 2006; 23(3): 186 - 192. [Abstract] [Full Text] [PDF] |
||||
![]() |
D. T. Ko, J. V. Tu, F. A. Masoudi, Y. Wang, E. P. Havranek, S. S. Rathore, A. M. Newman, L. R. Donovan, D. S. Lee, J. M. Foody, et al. Quality of Care and Outcomes of Older Patients With Heart Failure Hospitalized in the United States and Canada Arch Intern Med, November 28, 2005; 165(21): 2486 - 2492. [Abstract] [Full Text] [PDF] |
||||
![]() |
C. A. Beck, H. Richard, J. V. Tu, and L. Pilote Administrative Data Feedback for Effective Cardiac Treatment: AFFECT, A Cluster Randomized Trial JAMA, July 20, 2005; 294(3): 309 - 317. [Abstract] [Full Text] [PDF] |
||||
![]() |
W. D. Leslie, S. A. Tully, M. S. Yogendran, L. M. Ward, K. A. Nour, and C. J. Metge Prognostic Value of Lung Sestamibi Uptake in Myocardial Perfusion Imaging of Patients With Known or Suspected Coronary Artery Disease J. Am. Coll. Cardiol., May 17, 2005; 45(10): 1676 - 1682. [Abstract] [Full Text] [PDF] |
||||
![]() |
D. T. Ko, P. C. Austin, B. T. B. Chan, and J. V. Tu Quality of Care of International and Canadian Medical Graduates in Acute Myocardial Infarction Arch Intern Med, February 28, 2005; 165(4): 458 - 463. [Abstract] [Full Text] [PDF] |
||||
![]() |
W. D. Leslie, S. A. Tully, M. S. Yogendran, L. M. Ward, K. A. Nour, and C. J. Metge Prognostic Value of Automated Quantification of 99mTc-Sestamibi Myocardial Perfusion Imaging J. Nucl. Med., February 1, 2005; 46(2): 204 - 211. [Abstract] [Full Text] [PDF] |
||||
![]() |
D. S. Lee, P. C. Austin, J. L. Rouleau, P. P. Liu, D. Naimark, and J. V. Tu Predicting Mortality Among Patients Hospitalized for Heart Failure: Derivation and Validation of a Clinical Model JAMA, November 19, 2003; 290(19): 2581 - 2587. [Abstract] [Full Text] [PDF] |
||||
![]() |
P. C. Austin, D. A. Alter, and J. V. Tu The Use of Fixed-and Random-Effects Models for Classifying Hospitals as Mortality Outliers: A Monte Carlo Assessment Med Decis Making, November 1, 2003; 23(6): 526 - 539. [Abstract] [PDF] |
||||
![]() |
D. A. Alter, J. V. Tu, P. C. Austin, and C. D. Naylor Waiting times, revascularization modality, and outcomes after acute myocardial infarction at hospitals with and without on-site revascularization facilities in Canada J. Am. Coll. Cardiol., August 6, 2003; 42(3): 410 - 419. [Abstract] [Full Text] [PDF] |
||||
![]() |
D. A. Alter, Y. Khaykin, P. C. Austin, J. V. Tu, and J. E. Hux Processes and Outcomes of Care for Diabetic Acute Myocardial Infarction Patients in Ontario: Do physicians undertreat? Diabetes Care, May 1, 2003; 26(5): 1427 - 1434. [Abstract] [Full Text] [PDF] |
||||
![]() |
J. V. TU and C. CAMERON Impact of an acute myocardial infarction report card in Ontario, Canada Int. J. Qual. Health Care, March 1, 2003; 15(2): 131 - 137. [Abstract] [Full Text] [PDF] |
||||
![]() |
D. A. Alter, C. D. Naylor, P. C. Austin, B. T.B. Chan, and J. V. Tu Geography and service supply do not explain socioeconomic gradients in angiography use after acute myocardial infarction Can. Med. Assoc. J., February 4, 2003; 168(3): 261 - 264. [Abstract] [Full Text] [PDF] |
||||
![]() |
Y Khaykin, P C Austin, J V Tu, and D A Alter Utilisation of coronary angiography after acute myocardial infarction in Ontario over time: have referral patterns changed? Heart, December 1, 2002; 88(5): 460 - 466. [Abstract] [Full Text] [PDF] |
||||
![]() |
D. A. Alter, C. D. Naylor, P. C. Austin, and J. V. Tu Biology or bias: practice patterns and long-term outcomes for men and women with acute myocardial infarction J. Am. Coll. Cardiol., June 19, 2002; 39(12): 1909 - 1916. [Abstract] [Full Text] [PDF] |
||||
![]() |
P. Kaul, L. D. Saunders, L. L. Roos, G. Kephart, W. A. Ghali, R. Walld, and J. Warren Trends in Utilization of Coronary Artery Bypass Surgery and Associated Outcomes: Alberta, Manitoba, and Nova Scotia American Journal of Medical Quality, May 1, 2002; 17(3): 103 - 112. [Abstract] [PDF] |
||||
![]() |
W. B. Campbell, J. V. Tu, P. C. Austin, and B. T. B. Chan Relationship of Physician Volume to Mortality After Acute Myocardial Infarction JAMA, October 3, 2001; 286(13): 1574 - 1575. [Full Text] [PDF] |
||||
![]() |
D. A. Morrow, E. M. Antman, L. Parsons, J. A. de Lemos, C. P. Cannon, R. P. Giugliano, C. H. McCabe, H. V. Barron, and E. Braunwald Application of the TIMI Risk Score for ST-Elevation MI in the National Registry of Myocardial Infarction 3 JAMA, September 19, 2001; 286(11): 1356 - 1359. [Abstract] [Full Text] [PDF] |
||||
![]() |
J. V. Tu, P. C. Austin, and B. T. B. Chan Relationship Between Annual Volume of Patients Treated by Admitting Physician and Mortality After Acute Myocardial Infarction JAMA, June 27, 2001; 285(24): 3116 - 3122. [Abstract] [Full Text] [PDF] |
||||
![]() |
D. A. Alter, C. D. Naylor, P. C. Austin, and J. V. Tu Long-term MI Outcomes at Hospitals With or Without On-site Revascularization JAMA, April 25, 2001; 285(16): 2101 - 2108. [Abstract] [Full Text] [PDF] |
||||
![]() |
J. G. Jollis Measuring the effectiveness of medical care delivery J. Am. Coll. Cardiol., March 15, 2001; 37(4): 998 - 1000. [Full Text] [PDF] |
||||
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| HOME | SUBSCRIPTIONS | CURRENT ISSUE | PAST ISSUES | CARDIOSOURCE | SEARCH | HELP | FEEDBACK |