CORRESPONDENCE: RESEARCH CORRESPONDENCE
Comparing Hospital Performance in Door-to-Balloon Time Between the Hospital Quality Alliance and the National Cardiovascular Data Registry
Brahmajee K. Nallamothu, MD, MPH, FACC,
Yongfei Wang, MS,
Elizabeth H. Bradley, PhD,
Kalon K.L. Ho, MD, SM, FACC,
Jeptha P. Curtis, MD, FACC,
John S. Rumsfeld, MD, PhD, FACC,
Frederick A. Masoudi, MD, MSPH, FACC and
Harlan M. Krumholz, MD, SM, FACC*
* 333 Cedar Street, Room I-456 SHM, P.O. Box 208088, New Haven, Connecticut 06520-8088 (Email: harlan.krumholz{at}yale.edu).
To the Editor:
Data on hospital performance in door-to-balloon (DTB) times for patients with ST-segment elevation myocardial infarction (STEMI) are now collected and reported by numerous organizations, including the Centers for Medicare and Medicaid Services (CMS), The Joint Commission, and the American College of Cardiology (1–3). Despite proliferation of these efforts to report DTB time, its consistency across data sources has not been examined. Accordingly, we compared performance rankings for hospitals on DTB time using 2 prominent national data sources: 1) the Hospital Quality Alliance (HQA, a consortium of organizations including CMS and the Joint Commission; and 2) the National Cardiovascular Data Registry’s (NCDR) CathPCI Registry.
Data collected on DTB time from HQA and NCDR between January and December 2005 were used for this analysis. Data from HQA reflected 1,407 U.S. hospitals performing percutaneous coronary intervention (PCI) and 51,887 patients; data from NCDR reflected 456 hospitals and 19,697 patients. We excluded hospitals reporting fewer than 20 patients (n = 338 for HQA; n = 118 for NCDR) and those not reporting in all 4 quarters of 2005 (n = 19 for HQA; n = 62 for NCDR). Of the remaining, 35 hospitals could not be matched using unique identifiers, yielding 241 for direct comparison.
In 2005, DTB time in HQA was assessed as the time from hospital arrival to angioplasty balloon inflation in patients with STEMI or new left bundle branch block on the electrocardiogram performed closest to hospital arrival and in whom successful PCI was performed within 24 hours of admission. When use of an angioplasty balloon was not documented, the earliest time recorded for a wire or other alternative device reaching a lesion was used instead. In NCDR, DTB time was assessed using slightly different criteria than those described above. Specifically, it was measured as the time from hospital arrival to the first attempt at "reperfusion" in patients with STEMI clinically identified as undergoing primary PCI. For patients with evidence of normal coronary flow before use of an angioplasty balloon, reperfusion time was defined as the time until flow was re-established either by a guidewire or alternative intracoronary device. In both data sources, patients were excluded if transferred in from another hospital or fibrinolytic therapy was given before PCI.
We ranked hospitals into quintiles of performance separately in the HQA and NCDR data using several specifications of DTB time to reflect the various ways in which hospital performance for this measure has been previously reported (4). We used the following specifications: 1) mean DTB time; 2) median DTB time; 3) proportion of patients with DTB times within 90 min; and 4) proportion of patients with DTB times within 120 min. We calculated weighted kappa coefficients for each comparison and also reported the proportion of hospitals that remained within the same quintile or a similar quintile (i.e., the same or nearest quintile above or below) between data sources.
In the 241 matched hospitals, there were 51.3 (±22.9) patients per hospital in HQA with DTB times and 57.9 (±28.0) patients per hospital in NCDR with DTB times, reflecting modest differences in patient selection criteria between data sources. A comparison of patient populations between HQA and NCDR demonstrated similar comparability in the mean patient age (60.8 years vs. 60.6 years, respectively; p = 0.31) and mean percentage of women (29.4% vs. 29.0%, respectively; p = 0.62). Hospital median DTB times were longer in HQA compared with NCDR: 93.8 min versus 87.7 min (p < 0.001). The mean proportion of patients with DTB times within 90 min was 47.9% for HQA and 53.0% for NCDR.
Agreement between hospital rankings based on DTB time in HQA and NCDR was only fair to moderate overall (kappa coefficients ranging from 0.32 to 0.56 depending on DTB specification) (Table 1). The poorest agreement was for mean DTB time and the greatest agreement was for the proportion of patients with DTB within 90 min. However, even for the proportion of patients with DTB times within 90 min, only 47.7% of hospitals were in the same quintile for HQA and NCDR (Table 1). Agreement was slightly better at the extremes of hospital performance, where 58 (60.4%) of the 96 hospitals in the top and bottom quintiles of hospital rankings for HQA overlapped with NCDR.
View this table:
[in this window]
[in a new window]
|
Table 1 Summary of Comparability Between HQA and NCDR Data Using Rankings of Different Specifications of DTB Time
|
|
In summary, we found only fair to moderate agreement between DTB times reported to HQA and NCDR by the same group of hospitals, leading to substantial differences in rankings of hospital performance between data sources. Our results were consistent across different specifications of DTB time, although mean DTB times showed the least agreement between data sources, possibly owing to their greater susceptibility to extreme or outlier values. We believe discrepancies between data sources may arise from differences in: 1) patient selection for the measurement; 2) definitions for determining specific time points when measuring DTB time; and 3) data abstraction, quality checks, and standards (Table 2).
View this table:
[in this window]
[in a new window]
|
Table 2 Comparison Between DTB Time Assessment in HQA and NCDR Data Based on Patient Selection, Definitions for Determining Specific Time Points, and Data Abstraction, Quality Checks, and Standards in 2005
|
|
Our findings highlight an important problem that may result when multiple data sources are available to assess hospital performance. Even minor differences in the approach to measurement may substantially affect rankings of individual hospitals. This point is highlighted by placing our findings in the framework of the Medicare Premier Demonstration Project for pay-for-performance, which rewards or withholds payments to the top and bottom quintiles of hospital rankings. Under this framework, hospital rankings for nearly 40% of hospitals in the top and bottom quintiles differed between HQA and NCDR, leading to the potential for variable payments depending on which data source was used for ranking.
As the measurement of hospital performance expands, it becomes critical that the methods for collecting data are as reliable and aligned as possible. Inconsistency between data sources can lead to confusion and mistrust, undermining quality improvement efforts. This is especially required as measures move from internal quality improvement efforts to public reporting and pay-for-performance. Better alignment of data sources could also avoid duplication of efforts and has been recommended by the Institute of Medicine.
The present study should be interpreted in the context of these limitations. First, we matched only 241 facilities in the 2 data sources. Second, we only examined comparability across 2 data sources; however, the data sources we chose represent influential national programs. Third, we were unable to match patients directly across data sources. This prevented us from identifying whether the primary reason for differences between data sources was secondary to the selection of eligible cohorts or to abstraction techniques. Finally, this study cannot judge which of the data sources was more accurate. Both appear to have their strengths and weaknesses. As measures evolve (e.g., DTB time in HQA has already had major changes since 2005), further work should assess strategies for harmonization across data sources.
 |
Appendix
|
|---|
For a table outlining the comparison between DTB time assessment in HQA and NCDR data based on: 1) patient selection; 2) definitions for determining specific time points; and 3) data abstraction, quality checks, and standards in 2005, please see the online version of this article.
|
Footnotes
|
|---|
Please note: the authors are grateful to Ms. Jo DeBuhr, Mr. Tony Hermann, and Ms. Kristi Mitchell for their careful review of earlier drafts of the manuscript and insightful comments.
|
References
|
|---|
1. Brindis RG, Fitzgerald S, Anderson HV, Shaw RE, Weintraub WS, Williams JF. The American College of Cardiology-National Cardiovascular Data Registry (ACC-NCDR): building a national clinical data repository J Am Coll Cardiol 2001;37:2240-2245.[Free Full Text]2. Jha AK, Li Z, Orav EJ, Epstein AM. Care in U.S. hospitals—the Hospital Quality Alliance program N Engl J Med 2005;353:265-274.[Abstract/Free Full Text] 3. Williams SC, Schmaltz SP, Morton DJ, Koss RG, Loeb JM. Quality of care in U.S. hospitals as reflected by standardized measures, 2002–2004 N Engl J Med 2005;353:255-264.[Abstract/Free Full Text] 4. Krumholz HM, Anderson JL, Brooks NH, et al. ACC/AHA clinical performance measures for adults with ST-elevation and non–ST-elevation myocardial infarction: a report of the American College of Cardiology/American Heart Association Task Force on Performance Measures (Writing Committee to Develop Performance Measures on ST-Elevation and Non–ST-Elevation Myocardial Infarction) Circulation 2006;113:732-761.[Free Full Text]
This article has been cited by other articles:
|
|
|
|
 
F. A. Masoudi, R. O. Bonow, R. G. Brindis, C. P. Cannon, J. DeBuhr, S. Fitzgerald, P. A. Heidenreich, K. K.L. Ho, H. M. Krumholz, C. Leber, et al.
ACC/AHA 2008 Statement on Performance Measurement and Reperfusion Therapy: A Report of the ACC/AHA Task Force on Performance Measures (Work Group to Address the Challenges of Performance Measurement and Reperfusion Therapy)
J. Am. Coll. Cardiol.,
December 9, 2008;
52(24):
2100 - 2112.
[Full Text]
[PDF]
|
|
|
|
|
|
|
F. A. Masoudi, R. O. Bonow, R. G. Brindis, C. P. Cannon, J. DeBuhr, S. Fitzgerald, P. A. Heidenreich, K. K.L. Ho, H. M. Krumholz, C. Leber, et al.
ACC/AHA 2008 Statement on Performance Measurement and Reperfusion Therapy: A Report of the ACC/AHA Task Force on Performance Measures (Work Group to Address the Challenges of Performance Measurement and Reperfusion Therapy)
Circulation,
December 9, 2008;
118(24):
2649 - 2661.
[Full Text]
[PDF]
|
|
|
|
|
|
|
M. S. Kim, T. Y. Wang, D. Dai, A. J. Klein, E. D. Peterson, J. S. Rumsfeld, and J. C. Messenger
Association of Previous Coronary Artery Bypass Graft Surgery With Door-to-Balloon Time and In-Hospital Outcomes: A Report From the National Cardiovascular Data Registry (NCDR)
J. Am. Coll. Cardiol.,
November 11, 2008;
52(20):
1665 - 1670.
[Full Text]
[PDF]
|
|
|
|
Top
Appendix
References