EXPEDITED PUBLICATION: EDITORIAL COMMENT
The Emperor’s New ClothesAnother Cypher Versus Taxus Post-Hoc Meta-Analysis*
Dean J. Kereiakes, MD, FACC1,*
The Christ Hospital Heart and Vascular Center, The Lindner Center for Research and Education, Cincinnati, Ohio.
* Reprint requests and correspondence: Dr. Dean J. Kereiakes, The Lindner Center for Research and Education, 2123 Auburn Avenue, Suite 424, Cincinnnati, Ohio 45219. (Email: lindner{at}fuse.net).
A meta-analysis combines information from multiple independent prior studies in order to provide a more powerful estimate of specific treatment effects. Although this strategy may add statistical power by pooling smaller groups into a larger one, the meta-analytic approach remains fraught with challenges in terms of the quality of the constituent trials in addition to the technical limitations of statistical agglomeration (1,2). Disadvantages of the meta-analytic approach include the potential abrogation of many scientific requirements of reproducibility, suitable extrapolation, or even fair comparison (3). While the validity of probability statements derived from a meta-analysis is increased when the analysis involves the prospectively defined primary outcomes of the constituent trials, publication bias; trial selection; as well as heterogeneity of study protocols, subjects, and methods may still compromise the robustness of the meta-analytic findings (2). Moreover, the simple fact that component trials are randomized neither overcomes these multiple potential methodologic limitations nor validates the inclusion of trial results that reside outside of the broader data set (outliers), whose incorporation into a meta-analysis may distort its conclusions.
While meta-analyses may be designed prospectively into a drug/device developmental program, they are more commonly constructed post hoc from multiple randomized controlled trials as well as observational (registry) studies. Challenges that must be addressed include the selection of appropriate trials, the assignment of appropriate weighting based on robustness of trial design, and, finally, the prospect of making valid pooled comparisons across existing differences in trial methodology. In the context of these inherent limitations perhaps, not surprisingly, precedent meta-analyses may fail to accurately predict the outcomes of subsequent large randomized controlled clinical trials 35% of the time (4). In the absence of an adequately powered randomized, controlled clinical trial, the previously published meta-analysis would have resulted in either the adoption of an ineffective treatment or the rejection of an effective therapy in 32% and 33% of instances, respectively. Such variability in the validity of meta-analytic conclusions has prompted their designation as "Modern Day Statistical Alchemy" (3).
In this context and in this issue of the Journal, Schömig et al. (5) provide a post-hoc meta-analysis of randomized comparative trials involving the Cypher sirolimus-eluting stent (SES) (Cordis Inc., Miami Lakes, Florida) and the Taxus paclitaxel-eluting stent (PES) (Boston Scientific Corp., Natick, Massachusetts) for the treatment of a broad spectrum of patients presenting with symptomatic coronary artery disease (5). From this analysis, the authors conclude that: 1) SES are superior to PES in reducing the risk of both reintervention (enhanced efficacy) and stent thrombosis (enhanced safety); and that 2) the risks of both death and nonfatal myocardial infarction were similar between the devices. How should we interpret these findings and what relevance do they have for current clinical practice?
This work expands and extends similar observations made by this group from 6 trials involving 3,667 patients with clinical follow-up to 1 year (6). In the current meta-analysis, the authors have added 10 additional trials (to the original 6 trials) involving a total of 8,695 patients followed for a mean of 9 to 37 months. Although the majority of constituent trial subjects are described as "unselected," 2 trials enrolled only patients who presented with ST-segment elevation acute myocardial infarction while single additional trials enrolled only medically treated diabetic patients or patients presenting with bare-metal stent (BMS) restenosis, respectively. While each trial was randomized, they are mostly single-center, unblinded (investigators were aware of which device was being deployed) studies in which the clinical end points under analysis were determined by the investigators rather than being adjudicated by an independent clinical events committee. Although individual patient-level data from 5,562 patients were available (and analyzed separately), these data were largely acquired by post-hoc completion of electronic data sheets in 11 of the 16 trials (with neither source document audit nor independent adjudication). No summary scores were utilized to rank the quality of individual constituent trials (7), and no systematic approach was taken to reconcile differences in individual trial protocols. For example, routine late coronary angiographic follow-up was protocol-mandated in 10 of the 16 trials, and the protocol-prescribed duration of dual (aspirin plus thienopyridine) antiplatelet therapy varied widely (from 2 to 12 months for SES and from 6 to 12 months for PES). Finally, although only 8 trials were designed and powered to evaluate a clinical end point, the authors used all 16 to evaluate a primary efficacy outcome of target lesion revascularization (TLR) or target vessel revascularization (TVR) and a primary safety outcome of stent thrombosis (across variable protocol definitions). Although the authors could discern "no significant interaction between the treatment effect (TLR or TVR) and the inclusion of follow-up angiography in the study protocol," several points should be made.
First, the repeat revascularization curves in Figure 1B of the report by Schömig et al. (5) show the impact of routine follow-up angiography. These curves are roughly parallel before 6 months and then again after 8 to 9 months of follow-up is complete, but the abrupt inflection that occurs between 6 and 8 months reflects the impact of protocol-mandated angiography on revascularization through the well-described "oculo-stenotic reflex" (8). This is particularly true in the ISAR-DESIRE (Intracoronary Stenting and Angiographic Results: Drug-Eluting Stents for In-Stent Restenosis) (9), ISAR-DIABETES (Intracoronary Stenting and Angiographic Results: Do Diabetic Patients Derive Similar Benefit From Paclitaxel-Eluting and Sirolimus-Eluting Stents) (10), ISAR-SMART (Drug-Eluting Stenting to Abrogate Restenosis in Small Arteries) (11), and SIRTAX (Sirolimus-Eluting Versus Paclitaxel-Eluting Stents for Coronary Revascularization) (12) trials, wherein 70% to 80% of patients who had binary angiographic restenosis (>50%) went on to have repeat revascularization. This contrasts with the 40% to 50% rate of revascularization observed in similar patients enrolled into prior pivotal drug-eluting stent (DES) trials (13,14) and suggests that the decision to revascularize (TLR/TVR) may have been more "operator dependent" rather than "clinically driven," as defined by the precedent trials. These previous studies required the demonstration of objective evidence of ischemia in patients with moderate (50% to 70%) stenoses or the adjudication of events by an independent, blinded clinical events committee (14). Such rigorous prospective requirements were not satisfied by the ISAR or SIRTAX trials. It is noteworthy that the ISAR and SIRTAX trials alone contributed 54 (63%) of the 85 excess revascularizations reported for PES- versus SES-treated patients, and the relative "outlier" status of these trials is evident in Figure 1A, which depicts 29 randomized controlled trials and registries involving 46,246 patients (including the 16 trials selected by Schömig et al. [5]). The patient-number weighted regression estimate of the slope for Taxus versus Cypher TLR/TVR is 0.96 (95% confidence interval [CI] for slope 0.85 to 1.08), and is confirmed by formal random effects modeling (Fig. 1B), which shows a pooled risk ratio of 1.12 (95% CI 0.98 to 1.29, p = 0.108). Thus, it appears likely that the presence of the 4 "outlier" trials in the 16-study meta-analysis was still able to distort the meta-analytic signal, relative to the apparent lack of significant difference between Cypher and Taxus repeat revascularization in the broader clinical experience.
View larger version (30K):
[in this window]
[in a new window]
[Download PPT slide]
|
Figure 1 Target Lesion and Vessel Revascularization
(A) Target lesion revascularization (TLR) in 29 trials and registries involving 46,246 patients. References are provided in the Online Appendix. (B) Random effects modeling of 29 trials and registries evaluating TLR/target vessel revascularization (TVR) after Cypher (sirolimus-eluting stent [SES]) or Taxus (paclitaxel-eluting stent [PES]) stent deployment. CI = confidence interval.
|
|
The apparent lack of difference in repeat revascularization between Cypher and Taxus stents is in contradistinction to the well-established differences observed for in-stent and in-segment late lumen loss (LLL) between these 2 DES. Although LLL in-stent and in-segment by quantitative coronary angiography has been correlated with clinical events including restenosis (15,16), the relatively low values of LLL observed across several recent trials of complex patient cohorts treated with the Taxus stent (17–19) remain higher than those observed with the Cypher stent (13). It appears that these relatively subtle differences in LLL may still be consistent with similar low rates of clinically driven reintervention for both devices (20).
Second, the Schömig et al. (5) meta-analysis raises concern regarding the apparent higher rate of stent thrombosis after Taxus versus Cypher stent deployment. The recent analysis of the pivotal blinded, randomized BMS controlled trials of both PES and SES (21) demonstrated no such difference in the incidence of definite or probable stent thrombosis using a uniform definition (Academic Research Consortium) (22) not employed by Schömig et al. (5). In addition, the REALITY (Prospective, Randomized, Multi-Center Comparison of the Cypher Sirolimus-Eluting and the Taxus Paclitaxel-Eluting Stent Systems) (17) and LONG DES (Percutaneous Treatment of Long Native Coronary Lesions with Drug-Eluting Stents of Sirolimus-Eluting Stent versus Paclitaxel-Eluting Stent) (23) trials alone contributed 16 (55%) of the 29 excess stent thromboses observed after PES. If the hazard for stent thrombosis after PES versus SES was truly 34% to 50% higher (as suggested by the Schömig et al. [5] meta-analysis), this magnitude of risk should be readily apparent across the broad range of clinical experience with these devices. By expanding the Schömig et al. (5) 16-trial analysis to include 26 randomized trials and registries involving 35,539 patients, the estimate of the regression slope (Taxus vs. Cypher) for stent thrombosis is 1.02 (95% CI 0.79 to 1.25) (Fig. 2A). This observation is confirmed by random effects modeling, which estimates the odds ratio to be 1.19 (95% CI 0.97 to 1.48, p > 0.10) (Fig. 2B). Again, these differences likely reflect the influence of "outlier" trials (REALITY and LONG DES) on the meta-analytic conclusions of Schömig et al. (5).
View larger version (32K):
[in this window]
[in a new window]
[Download PPT slide]
|
Figure 2 Stent Thrombosis
(A) Stent thrombosis (ST) in 26 trials and registries involving 35,539 patients treated with either Cypher or Taxus stents. References are provided in the Online Appendix. (B) Random effects modeling of 26 trials and registries evaluating stent thrombosis after Cypher (SES) or Taxus (PES) stents. Abbreviations as in Figure 1.
|
|
 |
The Real Issue(s)
|
|---|
It has been suggested that "a trial proving superiority of one DES over another would require a multi-center study with a clinical primary endpoint at an adequate power" (24). In the absence of such a trial, the default strategy has been to cobble together a series of smaller trials involving widely variable methodologies and end points and to rationalize the inadequacies of such an approach. This is particularly true for comparisons between the Cypher and Taxus stents, which underwent adequately powered, blinded, randomized comparisons only in pivotal trials with their respective BMS platforms and not with each other before approval and general availability (13,14). For logistic reasons (time and cost), trials of subsequent DES have been designed to demonstrate "noninferiority" and have chosen either of the "approved standards" (Cypher or Taxus) for comparison. Although expedient, the noninferiority trial design carries inherent risk for "end point drift" across successive device design iterations (25). As "next generation" DES platforms become commercially available (several are approved for use outside the U.S.), further attempts to make indirect statistical comparisons of the benchmark Cypher and Taxus stents to each other will be less relevant. Finally, the U.S. FDA has recently emphasized the importance of larger, adequately powered trials with a clinical primary end point and extended duration follow-up during the clinical evaluation of new DES platforms (26). The adoption of more uniform DES clinical trial strategies with standardized end point definitions and postmarket surveillance should reduce the need for future indirect meta-analytic comparisons.
|
Appendix
|
|---|
For accompanying references for Figures 1A and 2A, please see the online version of this article.
|
Acknowledgments
|
|---|
The author gratefully acknowledges the biostatistical analyses and support provided by Donald Baim, MD, Peter Lam, PhD, and Jeannette Banks, BS, of Boston Scientific Corporation in the performance and graphic display of the meta-analyses requested for this editorial.
|
Footnotes
|
|---|
* Editorials published in the Journal of the American College of Cardiology reflect the views of the authors and do not necessarily represent the views of JACC or the American College of Cardiology. 
1 Dr. Kereiakes serves on the scientific advisory boards of the Cordis, Abbott/Guidant, Boston Scientific, Biodegradable Vascular Solutions, and CoreValve Corporations. He has been both a consultant for and received research grant support from the Cordis, Abbott/Guidant, Boston Scientific, and Medtronic Corporations.
|
References
|
|---|
1. Flather MD, Farkouh ME, Pogue JM, Yusuf S. Strengths and limitations of meta-analysis: larger studies may be more reliable Control Clin Trials 1997;18:568-579.[CrossRef][Web of Science][Medline]2. Cleophas TJ, Zwinderman AH. Meta-analysis Circulation 2007;115:2870-2875.[Free Full Text] 3. Feinstein AR. Meta-analysis: statistical alchemy for the 21st century J Clin Epidemiol 1995;48:71-79.[CrossRef][Web of Science][Medline] 4. LeLorier J, Gregoire G, Benhaddad A, Lapierre J, Derderian F. Discrepancies between meta-analyses and subsequent large randomized, controlled trials N Engl J Med 1997;337:536-542.[Abstract/Free Full Text] 5. Schömig A, Dibra A, Windecker S, et al. A meta-analysis of 16 randomized trials of sirolimus-eluting stents versus paclitaxel-eluting stents in patients with coronary artery disease J Am Coll Cardiol 2007;50:1373-1380.[Abstract/Free Full Text] 6. Kastrati A, Dibra A, Eberle S, et al. Sirolimus-eluting stents vs paclitaxel-eluting stents in patients with coronary artery disease: meta-analysis of randomized trials JAMA 2005;294:819-825.[Abstract/Free Full Text] 7. Silber S. A new and rapid scoring system to access the scientific evidence from clinical trials J Interv Cardiol 2006;19:485-492.[CrossRef][Medline] 8. Pinto DS, Stone GW, Ellis SG, et al. Impact of routine angiographic follow-up on the clinical benefits of paclitaxel-eluting stents: results from the TAXUS-IV trial J Am Coll Cardiol 2006;48:32-36.[Abstract/Free Full Text] 9. Kastrati A, Mehilli J, von Beckerath N, et al. Sirolimus-eluting stent or paclitaxel-eluting stent vs balloon angioplasty for prevention of recurrences in patients with coronary in-stent restenosis: a randomized controlled trial JAMA 2005;293:165-171.[Abstract/Free Full Text] 10. Dibra A, Kastrati A, Mehilli J, et al. Paclitaxel-eluting or sirolimus-eluting stents to prevent restenosis in diabetic patients N Engl J Med 2005;353:663-670.[Abstract/Free Full Text] 11. Mehilli J, Dibra A, Kastrati A, Pache J, Dirschinger J, Schomig A. Randomized trial of paclitaxel-and sirolimus-eluting stents in small coronary vessels Eur Heart J 2006;27:260-266.[Abstract/Free Full Text] 12. Windecker S, Remondino A, Eberli FR, et al. Sirolimus-eluting and paclitaxel-eluting stents for coronary revascularization N Engl J Med 2005;353:653-662.[Abstract/Free Full Text] 13. Moses JW, Leon MB, Popma JJ, et al. SIRIUS Investigators Sirolimus-eluting stents versus standard stents in patients with stenosis in a native coronary artery N Engl J Med 2003;349:1315-1323.[Abstract/Free Full Text] 14. Stone GW, Ellis SG, Cox DA, et al. Taxus IV Investigators A polymer-based paclitaxel-eluting stent in patients with coronary artery disease N Engl J Med 2004;350:221-231.[Abstract/Free Full Text] 15. Mauri L, Orav EJ, Kuntz RE. Late loss in lumen diameter and binary restenosis for drug-eluting stent comparison Circulation 2005;111:3435-3442.[Abstract/Free Full Text] 16. Pocock S, Stone GW, Fahy M, et al. Relationship between late loss, diameter stenosis and target lesion revascularization after stent implantation: an examination of surrogate endpoints from a pooled analysis of eight large randomized DES trials(abstr) J Am Coll Cardiol 2006;47:188A. 17. Morice MC, Colombo A, Meier B, et al. Sirolimus- vs paclitaxel-eluting stents in de novo coronary artery lesions: the REALITY trial: a randomized controlled trial JAMA 2006;295:895-904.[Abstract/Free Full Text] 18. Stone GW, SPIRIT III Investigators Clinical, angiographic, and IVUS results from the pivotal U.S. randomized SPIRIT III Trial of the XIENCE V Everolimus Eluting Coronary Stent System 2007Paper presented at: ACC 56th Annual Scientific Session; March 24–27; New Orleans, LA. 19. Krucoff M, COSTAR II Investigators Multicenter randomized evaluation of a novel paclitaxel eluting stent with bioabsorbable polymer for the treatment of single and multivessel coronary disease: primary results of the CObalt Chromium STent with Antiproliferative for Restenosis II (COSTAR II) study 2007Paper presented at: Euro PCR 2007 Late Breaking Trial Presentation; March 22–25; Barcelona, Spain. 20. Cosgrave J, Melzi G, Corbett S, et al. Comparable clinical outcomes with paclitaxel- and sirolimus-eluting stents in unrestricted contemporary practice J Am Coll Cardiol 2007;49:2320-2328.[Abstract/Free Full Text] 21. Mauri L, Hsieh W, Massaro JM, Ho KK, D’Agostino R, Cutlip DE. Stent thrombosis in randomized clinical trials of drug-eluting stents N Engl J Med 2007;356:1020-1029.[Abstract/Free Full Text] 22. Cutlip DE, Windecker S, Mehran R, et al. Clinical end points in coronary stent trials: a case for standardized definitions Circulation 2007;115:2344-2351.[Abstract/Free Full Text] 23. Kim YH, Park SW, Lee SW, et al. Sirolimus-eluting stent versus paclitaxel-eluting stent for patients with long coronary artery disease Circulation 2006;114:2148-2153.[Abstract/Free Full Text] 24. Silber S. Cypher versus Taxus: are there differences? J Interv Cardiol 2005;18:441-446.[CrossRef][Medline] 25. Muni NI, Califf RM, Foy JR, Boam AB, Zuckerman BD, Kuntz RE. Coronary drug-eluting stent development: issues in trial design Am Heart J 2005;149:415-433.[CrossRef][Web of Science][Medline] 26. Krucoff MW, Boam A, Schultz DG. Drug-eluting stents "deliver heartburn" how do we spell relief going forward? Circulation 2007;115:2990-2994.[Free Full Text]
This article has been cited by other articles:
|
|
|
|
 
P. J. Vlaar and F. Zijlstra
Angiographic Restenosis and Clinical Recurrence After Sirolimus- and Paclitaxel-Eluting Stent Implantation
J. Am. Coll. Cardiol. Intv.,
August 1, 2009;
2(8):
776 - 778.
[Full Text]
[PDF]
|
|
|
|
|
|
|
Sirolimus-Eluting vs. Paclitaxel-Eluting Stents: A Meta-Analysis
Journal Watch Cardiology,
October 3, 2007;
2007(1003):
3 - 3.
[Full Text]
|
|
|
|
Top
The Real Issue(s)
Appendix