CORRESPONDENCE: LETTER TO THE EDITOR
Of Statistical Significance: "Trends" Toward Significance and Optimism Bias
Sripal Bangalore, MD, MHA and
Franz H. Messerli, MD*
* Hypertension Program, Division of Cardiology, Columbia University College of Physicians and Surgeons, St. Luke’s-Roosevelt Hospital Center, 1000 Tenth Avenue, Suite 3B-30, New York, New York 10025 (Email: fmesserl{at}chpnet.org).
In the study by McMurray et al. (1), the investigators discuss a post hoc analysis of the VALIANT (Valsartan in Acute Myocardial Infarction) trial and conclude: "These data, although not conclusive, also support the hypothesis that adding an ARB (angiotensin receptor blocker) to an ACE (angiotensin-converting enzyme) inhibitor may have a small additional anti-infarction effect." This conclusion was based on a "trend" toward decreased myocardial infarction (MI) and stroke compared with monotherapy. One has to question whether p values of 0.65 and 0.187 should be considered a "trend" toward significance. By the same analogy, there seems to be a "trend" toward greater incidence of angina in patients on combination therapy (based on survival curves). Moreover, based on the survival curves, at 6 months there seems to be a "trend" toward a greater number of strokes in the combination therapy compared to valsartan alone.
There should be some thought/discussion on what can reasonably be defined as a "trend." This question has surfaced in all post hoc analyses of randomized controlled trials where there is no preset criterion for significance. If we consider p < 0.05 as statistically significant, perhaps a p value between 0.05 to 0.10 may be considered as a trend toward significance. The conclusion that "These data also suggest, but do not prove, that adding an ARB to an ACE inhibitor may have a small additional anti-infarction effect" should thus be interpreted with caution. Given these somewhat slippery slopes, we call for a consensus as to what level of significance should be identified as a "trend" in a post hoc analysis and whether the observation of such a trend should be an acceptable basis for conclusions. Unless these terms are clearly defined a priori, one cannot quite avoid a euphemism such as "optimism bias" (2) to characterize the conclusions made by McMurray et al. (1).
 |
References
|
|---|
- McMurray J, Solomon S, Pieper K, et al. The effect of valsartan, captopril, or both on atherosclerotic events after acute myocardial infarction: an analysis of the Valsartan in Acute Myocardial Infarction Trial (VALIANT) J Am Coll Cardiol 2006;47:726-733.[Abstract/Free Full Text]
- Chalmers I, Matthews R. What are the implications of optimism bias in clinical research? Lancet 2006;367:449-450.[CrossRef][ISI][Medline]
Related Article
-
Reply
- John J.V. McMurray, Robert M. Califf, and Marc A. Pfeffer
J. Am. Coll. Cardiol. 2006 48: 1471.
[Full Text]
[PDF]
Top
References