EDITORIAL COMMENT
Defining and Utilizing Surrogates in the Evaluation of Coronary StentsWhat Do We Really Want and Need to Know?*
Robert A. Harrington, MD, FACC , ,*,
Vic Hasselblad, PhD and
Robert M. Califf, MD, MACC,
Duke Clinical Research Institute, Duke University Medical Center, Durham, North Carolina
Department of Medicine, Duke University Medical Center, Durham, North Carolina
Duke Translational Medicine Institute, Duke University Medical Center, Durham, North Carolina.
* Reprint requests and correspondence: Dr. Robert A. Harrington, Duke Clinical Research Institute, 2400 Pratt Street, Durham, North Carolina 27705. (Email: robert.harrington{at}duke.edu).
It has now been 30 years since Andreas Gruentzig introduced the percutaneous treatment of obstructive coronary artery disease as a viable option for alleviating suffering from angina and reducing the risk of dying from myocardial infarction (1). In many ways, the interventional cardiology community has created a model for translational medicine by incorporating industry, academia, the Food and Drug Administration (FDA), and clinical practitioners into a constant effort to improve technology through innovation tested in randomized clinical trials. Despite the enormous appetite for trials in this community, the desire to make the process more efficient remains a laudable sentiment.
The goals in treating patients with ischemic heart disease are to prolong life, reduce symptoms, and avoid unpleasant events (rehospitalization, repeat procedures); from a societal perspective, all of this should be done in a culturally appropriate, cost-effective manner. Clinicians considering therapeutic options for their patients need to decide whether the medical or device interventions they plan meet at least one of these objectives without detracting from the others.
One challenge to developing a new therapeutic intervention is the multiple constituencies involved in the process. Although patients and clinicians are at the center of individual decision making, other interested parties include basic and clinician investigators, regulators, industry, and members of the financial community. The development of new medical products in America’s market-driven but heavily regulated society is made more complex by the fact that many of these groups play multiple roles at different times. Trying to understand a new medication or device as it goes through the development process requires that an observer recognize the many relationships and perspectives inherent in that process.
Whereas clinical practice is aided by guideline recommendations (2,3), the entry of new diagnostic and therapeutic medical devices into the marketplace is regulated in the U.S. by the FDA under Title 21 of the Code of Federal Regulations (4). For most cardiovascular devices, the FDA requires pre-market approval (PMA) "... based on a determination by FDA that the PMA contains sufficient valid scientific evidence to assure that the device is safe and effective for its intended uses" (5). Optimally, the approval of a new device and the subsequent guideline recommendations for its use follow a similar path: review of available evidence that demonstrates a convincing level of safety and efficacy, followed by a recommendation for approval and clinical use.
In this issue of the Journal, Pocock et al. (6) report on a set of analyses by which they tested the suitability of angiographic measures as surrogate end points in the evaluation of drug-eluting stents (DES) (6). Using protocol-mandated angiographic follow-up and patient-level data from 11 randomized trials that compared different DES (2 trials) or DES against bare-metal stents (9 trials), the investigators evaluated 4 angiographic measures used as surrogates for the clinical end point of target lesion revascularization (TLR). They concluded that all 4 angiographic measures, in-stent and in-segment late loss (LL), as well as both assessments of percent diameter stenosis (%DS), provided a reliable estimation of TLR rates in all the trials and met the accepted criteria for surrogate end points. They further reported that %DS may have an advantage over LL as a surrogate end point in that the relationship between %DS and TLR is consistent independent of vessel size; it can be determined with only a single measurement; and it is probably more easily understood by clinicians. With these findings, they then proposed that clinical trials evaluating new stent efficacy that use these surrogates as the primary outcome measure can have much smaller patient populations than is currently possible when TLR is the primary outcome. If there is an observed biological effect of the new device, then they suggest that larger studies, either randomized trials or observational registries, could provide information regarding patient safety, presumably through assessment of "hard" clinical events such as death, myocardial infarction, or any revascularization.
The concept of using physical signs, lab assays, or imaging measures as a substitute for clinical outcomes has a long history. In cardiovascular medicine, blood pressure, low-density lipoprotein, and hemoglobin A1C are all used as surrogates for clinical vascular events, including death, stroke, and myocardial infarction. Therapies that alter these measures in a favorable way have been assumed to be beneficial, largely based on the strength of epidemiologic evidence. Multiple authors have pointed out the fallacies that come with assuming that relationships between biomarkers or intermediate end points and the ultimate clinical end points are always predictable and reliable (7–10). Perhaps the most famous example in cardiovascular medicine is the results of the CAST (Cardiac Arrhythmia Suppression Trial) (11) study where treatment with antiarrhythmic therapy, and suppression of premature ventricular contractions (the biological hypothesis of the trial), caused a highly significant increased risk of death compared with that seen in the placebo group.
More recently, safety issues associated with drugs such as the cyclooxygenase-2 inhibitors and rosiglitazone raised serious concern, both from a policy as well as an individual patient perspective, about the wisdom of relying on intermediate measures when assessing and determining the effects of medical interventions (12–14). In these examples what emerges is the notion that the effects of therapies on populations of patients are typically more complex than the initial biological beliefs might have predicted. The ability to adequately quantify both the potential benefits and risks of any medical intervention therefore requires large clinical trials, appropriately designed with sufficient patients (power) to reliably detect both benefit and risk. Responsible public policy should demand such a quantifiable accounting, and the care of individual patients absolutely depends on such information being understandable and applicable to routine practice.
The emergence of a major public health issue surrounding late stent thrombosis after implantation of DES (15,16) points out the difficulty of depending on limited trials of insufficient size and duration to adequately characterize the true risks and benefits associated with implantable cardiac devices. Therapies that will potentially be used in large, diverse populations for extended periods of time by necessity require an adequate information base to properly quantify risks and benefits. To do less than this is an abdication of responsibility by all involved in the development of medical therapeutics.
Pocock et al. (6) should be congratulated for performing and presenting such a careful and elegant set of analyses to convincingly demonstrate that the angiographic biomarkers of LL and %DS do, in fact, reliably predict the clinical event of TLR. In particular, their analyses demonstrate that the Prentice criterion (17) has been met, namely that the surrogate marker for a true end point yields a valid test of the null hypothesis of no association between treatment and the true end point. This is rarely done in clinical practice, which is why Fleming et al. (8) have suggested that the Prentice criterion may be very difficult to verify and, in fact, may be too stringent (18). The authors show that both of these measures, whether considered in-stent or in-segment, meet the published and accepted criteria of surrogacy. They go a step further and, against conventional belief, make a compelling case for the ascendancy of %DS over LL as the preferred surrogate for TLR. The study by Pocock et al. (6) has a number of strengths: the inclusion of both DES versus bare-metal stents and DES versus DES studies, a wide range of angiographic lesions, different drugs and elution mechanisms included in the DES studies, the consistent definition of TLR employed, and the independent adjudication process for the clinical events in all of the trials.
Although these data are strong, as is the case for using these measures as a potential surrogate for TLR, the authors state that angiographic end points might best be used to limit the size of phase 2 trials where investigators can determine in modest sample sizes and with a reasonable degree of certainty that a new DES behaves in a biologically predictable manner. They acknowledge that a determination of safety of a new DES requires large studies. A detailed discussion of what type of larger study and the implications for regulatory approval and practice adoption were beyond the scope of their report.
However, the authors misinterpret and overstate the challenges associated with performing large clinical studies in patient populations representative of those who might ultimately be treated in routine clinical practice. The number of percutaneous coronary interventions performed annually on a global basis well exceeds one million. Most by necessity are performed in sophisticated medical settings that allow the conduct of clinical investigation. The problem is not, as the authors suggest, in the numbers of patients required to answer these questions but in the challenges that have emerged in the conduct of clinical investigation in both the U.S. and the European Union (i.e., in the willingness of investigators, academic medical centers, community clinicians, governments, regulators, and the medical products industry to address some fundamental issues surrounding clinical investigation).
The current approach to performing the type of research needed to properly evaluate medical therapies is broken, it is inefficient, and it is in danger of limiting our ability to rapidly evaluate and consider novel, emerging therapies that could have important benefits for human health.
The number of investigators participating in the research process is small and inadequate. Residency and fellowship training programs do not adequately educate their trainees about the conduct of clinical investigation. Because most research participation comes from community practices, this failure to prepare residents and fellows for a career that includes research involvement threatens the nation’s future health. Within the academic medical center, the failure to recognize the academic importance of research participation and the failure to support such involvement through mechanisms such as protected time, resources, and promotion compounds these problems. The National Institutes of Health funding of the Clinical and Translational Science Awards offers an opportunity to rectify some of the historical neglect of clinical research from academic centers (19,20).
Society desperately needs a more efficient clinical research system that addresses current deficiencies in training (for both investigators and study coordinators), in protecting human subjects (including institutional review boards), in contracting (21), in data collection, including integrating electronic health records with research databases, and in regulatory oversight requirements. What is desperately needed is a societal commitment to a better integration of practice with research. This includes a commitment from academics, government, and private groups, including the medical products and financial industries. Such a system would allow investigators to ask questions that are relevant to typical practice, for industry sponsors to truly test and understand their products in the context of how they might ultimately be used, and for regulators to be able to weigh risks against benefits that have been characterized and quantified with a high degree of certainty.
So, what do we need and what should we be demanding in the assessment of new medical products, including coronary devices such as DES? Pocock et al. (6) argue for modest-size trials using these carefully defined surrogates to evaluate the biological potential of new DES. If the follow-up %DS and LL are consistent with the desired/anticipated treatment effect, then larger studies can be performed to assess the true clinical outcomes. Pocock et al. (6) suggest this might be done as an evaluation of "safety" through large, simple, randomized clinical trials or by using observational patient registries. We agree with some of this suggestion but not all. Observational registries can be helpful in assessing how a therapy is adopted into routine practice, but an adequate and appropriate comparison of a new therapy against another therapy requires randomization. Even detailed statistical adjustments of the data from registries are insufficient to allow a proper assessment of one therapy compared with another.
Cutlip et al. (22) have proposed 2 complementary types of end point definitions to use in the evaluation of coronary stents: one set to evaluate device efficacy and the other to address patient-oriented outcomes. Using such a construct permits investigators to gain insight into the biological effect of the new device while gaining sufficient knowledge about what really matters—how patients fare with regard to freedom from all-cause death, myocardial infarction, and repeat revascularization procedures.
Understanding the biology of new devices is critical for moving innovative ideas forward into clinical practice. Pocock et al. (6) have provided some creative methods for achieving this aim. But the larger concern about how the effects of therapies are understood in clinical practice requires attention to the methods of randomized clinical trials. Trials that are appropriately designed will include the spectrum of patients presenting in routine practice and enough patients to adequately assess risks and benefits and to measure the things that really matter to patients—being able to live longer without symptoms and avoid unpleasant experiences such as rehospitalizations and repeat procedures. In order to adequately achieve these goals, the cardiovascular community needs to address the issue of a currently dysfunctional and broken clinical research process. That is what society really needs.
 |
Acknowledgments
|
|---|
The authors thank Penny Hodgson for her expertise in reviewing and editing this manuscript.
|
Footnotes
|
|---|
Dr. Harrington has received research grants from Schering-Plough, Bristol-Myers Squibb, Sanofi-Aventis, Bayer, Eli Lilly, Daichii-Sankyo, Cordis, Boston Scientific, Conor, Abbott, AstraZeneca, and The Medicines Company and has been a consultant for Schering-Plough, Bristol-Myers Squibb, and Sanofi-Aventis. All personal honoraria generated from industry activities (consulting, advisory boards, speaking, and so on) are donated to educational charities. Dr. Califf does not feel that he has conflicts of interest with regard to this editorial, but for those who wish to form their own judgments, his industry interactions are displayed at http://www.dcri.duke.edu/research/coi.jsp.
* Editorials published in the Journal of the American College of Cardiology reflect the views of the authors and do not necessarily represent the views of JACC or the American College of Cardiology. 
|
References
|
|---|
1. Gruentzig A. Transluminal dilatation of coronary artery stenosis Lancet 1978;1:263.[CrossRef][Web of Science][Medline]2. Gibbons RJ, Smith Jr. SC, Antman E. American College of Cardiology/American Heart Association clinical practice guidelines: part I: where do they come from? Circulation 2003;107:2979-2986.[Free Full Text] 3. Gibbons RJ, Smith Jr. SC, Antman E. American College of Cardiology/American Heart Association clinical practice guidelines: part II: evolutionary changes in a continuous quality improvement project Circulation 2003;107:3101-3107.[Free Full Text] 4. U.S. Food and Drug Administration Getting To Market With A Medical Devicewww.fda.gov/cdrh/devadvice/3122.html 2003Accessed August 13, 2007. 5. U.S. Food and Drug Administration Device Advicewww.fda.gov/cdrh/devadvice/pma 2003Accessed August 13, 2007. 6. Pocock SJ, Lansky AJ, Mehran R, et al. Angiographic surrogate end points in drug-eluting stent trials: a systematic evaluation based on individual patient data from 11 randomized, controlled trials J Am Coll Cardiol 2008;51:23-32.[Abstract/Free Full Text] 7. Temple RJ. Choice of treatment: outcomes and treatment goals Am Heart J 2003;146:565-567.[CrossRef][Web of Science][Medline] 8. Fleming TF, DeMets DL. Surrogate end points in clinical trials: are we being misled? Ann Intern Med 1996;125:605-613.[Abstract/Free Full Text] 9. Califf RM, DeMets DL. Lessons learned from recent cardiovascular clinical trials: part I Circulation 2002;106:746-751.[Free Full Text] 10. Califf RM, DeMets DL. Lessons learned from recent cardiovascular clinical trials: part II Circulation 2002;106:880-886.[Free Full Text] 11. The Cardiac Arrhythmia Suppression Trial (CAST) Investigators Effect of the antiarrhythmic agent moricizine on survival after myocardial infarction N Engl J Med 1992;321:406-412. 12. Solomon SD, McMurray JJV, Pfeffer MA, et al. Cardiovascular risk associated with celecoxib in a clinical trial for colorectal adenoma prevention N Engl J Med 2005;352:1071-1080.[Abstract/Free Full Text] 13. Mukherjee DM, Nissen SE, Topol EJ. Risk of cardiovascular events associated with selective COX-2 inhibitors JAMA 2001;286:954-959.[Abstract/Free Full Text] 14. Nissen SE, Wolski K. Effect of rosiglitazone on the risk of myocardial infarction and death from cardiovascular causes N Engl J Med 2007;356:2457-2471.[Abstract/Free Full Text] 15. Harrington RA, Ohman EM. The enigma of drug-eluting stents: hope, hype, humility, and advancing patient care JAMA 2007;297:2028-2030.[Free Full Text] 16. Maisell WH. Unanswered questions—drug-eluting stents and the risk of late thrombosis N Engl J Med 2007;356:981-984.[Free Full Text] 17. Prentice RL. Surrogate endpoints in clinical trials: definition and operational criteria Stat Med 1989;8:431-440.[Web of Science][Medline] 18. Fleming TR, Prentice RL, Pepe MS, Glidden D. Surrogate and auxiliary endpoints in clinical trials, with potential applications in cancer and AIDS research Stat Med 1994;13:955-968.[Web of Science][Medline] 19. NIH Roadmap for Medical Researchhttp://nihroadmap.nih.gov/overview.asp 1994Accessed August 12, 2007. 20. NIH National Center for Research Resources Clinical and Translational Science Awardshttp://www.ncrr.nih.gov/clinical_research_resources/clinical_and_translational_science_awards/ 1994Accessed August 12, 2007. 21. Schulman KA, Seils DM, Timbie JW, et al. A national survey of provisions in clinical-trial agreements between medical schools and industry sponsors N Engl J Med 2002;347:1335-1341.[Abstract/Free Full Text] 22. Cutlip DE, Windecker S, Mehran R, et al. Clinical end points in coronary stent trials: a case for standardized definitions Circulation 2007;115:2344-2351.[Abstract/Free Full Text]
Related Article
-
Angiographic Surrogate End Points in Drug-Eluting Stent Trials: A Systematic Evaluation Based on Individual Patient Data From 11 Randomized, Controlled Trials
- Stuart J. Pocock, Alexandra J. Lansky, Roxana Mehran, Jeffrey J. Popma, Martin P. Fahy, Yingbo Na, George Dangas, Jeffrey W. Moses, Tereza Pucelikova, David E. Kandzari, Stephen G. Ellis, Martin B. Leon, and Gregg W. Stone
J. Am. Coll. Cardiol. 2008 51: 23-32.
[Abstract]
[Full Text]
[PDF]
This article has been cited by other articles:
|
|
|
|
 
R. Chacko, M. Mulhearn, V. Novack, L. Novack, L. Mauri, S. A. Cohen, J. Moses, M. B. Leon, and D. E. Cutlip
Impact of Target Lesion and Nontarget Lesion Cardiac Events on 5-Year Clinical Outcomes After Sirolimus-Eluting or Bare-Metal Stenting
J. Am. Coll. Cardiol. Intv.,
June 1, 2009;
2(6):
498 - 503.
[Abstract]
[Full Text]
[PDF]
|
|
|
|
Top
References