Physicians seem increasingly skeptical of the results of randomized clinical trials. The reasons are multifactorial but might be due, in part, to concerns about generalizability and reproducibility of trial results. Contemporary cardiology trials routinely involve thousands of patients and hundreds of sites in multiple countries/continents, making them challenging to design, operationally complex to execute, and difficult to oversee (1). Recent clinical trials in cardiovascular medicine have been criticized by regulatory agencies, due to perceived high rates of subjects who are lost to follow-up or off study drug. In response, the Food and Drug Administration has released new guidance emphasizing the importance of patient retention and the need for innovation and improvement in site monitoring (2).

The very insightful study by Butler et al. (3) in this issue of the Journal adds several interesting observations related to the quality of clinical trials in the current era. The authors analyzed the EVEREST (Efficacy of Vasopressin Antagonism in Heart Failure: Outcome Study with Tolvaptan) trial that was performed in 4,133 patients hospitalized with decompensated heart failure at 436 sites. Overall, the trial found tolvaptan to have no effect on either all-cause mortality or cardiovascular death/hospital stay for heart failure (4). Despite fairly broad inclusion criteria (New York Heart Association functional class III/IV heart failure, ejection fraction ≤40%, randomized within 48 h of admission to the hospital with heart failure), 77 sites (18%) were unable to enroll a single patient, and 224 sites (51%) enrolled <10 patients. Among the 359 participating sites that did enroll patients, only 30 sites were able to enroll >30 subjects. Only 1.8% of North American sites and 1.3% of Western European sites were considered to be high-enrolling sites. In contrast, 20% of South American sites and 27% of Eastern European sites enrolled >30 subjects, illustrating the current challenges randomized trials have in enrolling patients from Western European and North American countries.

The authors went on to perform a post hoc analysis evaluating the relationship between the number of patients enrolled/site and clinical outcomes in the trial. They found that patients at high-enrolling sites (>30 subjects) had better clinical outcomes than those patients enrolled at intermediate- (11 to 30 subjects) or low-enrolling (≤10 subjects) sites. Furthermore, these high-enrolling sites had the highest proportion of patients who completed the protocol, whereas lower-enrolling sites had more subjects who withdrew consent or were lost to follow-up.

The differential in clinical outcomes between patients at high- versus intermediate- and low-enrolling sites is likely explained by fundamental differences in the patient populations. Patients from low-enrolling sites on average were older with lower ejection fraction, longer QRS duration, lower blood pressure, higher B-type natriuretic peptide, higher creatinine, more likely to have an implantable cardioverter-defibrillator previously placed, and more likely to be treated with inotropes upon admission. Yet, even after these factors were controlled in multivariable models, high-enrolling sites had better outcomes.

This study is not the first study to describe outcome differences in subgroups or even variability in treatment effect found in a randomized clinical trial. In particular, geographic variation in treatment effect has been seen in multiple randomized clinical trials and has been well-described ((5),(6),7). However, the present findings with regard to the association of site enrollment volume and outcome were independent of any effect of region.

Are the findings of heterogeneity in outcomes on the basis of site enrollment troublesome? It is quite possible that the improved outcomes seen at high-enrolling sites are predominately due to the enrollment of lower-acuity patients. For this reason, further studies are necessary from other clinical trials to determine whether similar relationships are present, because the interplay between high volume, high quality, and improved outcomes has been shown in many clinical settings (8). It remains possible that the differences shown between high- and low-enrolling sites might be due to the actual quality of sites participating in the clinical trial. Just as variability exists between providers, there is considerable variability in the quality of sites participating in clinical trials. Underreporting of safety and efficacy endpoints, inadequate concomitant therapy, and high patient discontinuation rates are just a few examples of recent site-specific issues seen in trials. These issues were the impetus for recent Food and Drug Administration guidance providing support for flexible monitoring plans that are able to adjust and provide further monitoring support and oversight to sites that seem to be lagging in quality. Yet, to date, there remains no standardized way in which to measure the quality of a research site in general.

It is not inconceivable that the differential outcomes seen between low- and high-enrolling sites could affect the overall results and interpretation of the trial. However, the randomization process, which used block randomization within each center, should have been robust enough that patients on and off study drug would be equally distributed between sites of potentially differing quality (9). This is supported statistically by the lack of a treatment interaction in the multivariable modeling. Of greater concern are the implications for randomized clinical trials moving forward.

Is it possible to minimize inter-site differences while at the same time improving site-based quality and outcomes? As a specialty, cardiology has taken great pride in its focus on evidenced-based medicine. Yet, the majority of recommendations found in our guidelines are not supported by randomized trials but rather by expert opinion, illustrating the tremendous opportunities for continued research (10). We have lagged behind specialties such as oncology that have formed large clinical trial networks that enroll patients into appropriate clinical trials.

The National Heart, Lung, and Blood Institute has worked to bring this type of focus to cardiology with initiatives such as the Heart Failure Clinical Research Network, and efforts should be made to expand this model to different disease states within cardiology ((11),12). As physicians, we must develop and advocate for improved clinical trial networks and encourage enrollment and appropriate follow-up. Registries and comparative effectiveness techniques offer the ability to study questions that are unable to be answered with clinical trials (13), although observational research alone has been shown on multiple occasions to be flawed (14). Thus, the development and promotion of a culture of scientific inquiry in which randomized trials are part of routine clinical practice would provide a tremendous opportunity to advance cardiovascular medicine and simultaneously improve care for all patients with cardiovascular disease.

⁎Editorials published in the Journal of the American College of Cardiology reflect the views of the authors and do not necessarily represent the views of JACC or the American College of Cardiology.