EDITORIAL COMMENT
Do You See What I See?
Agreeing to Disagree*
Ann F. Bolger, MD, FACC, FAHA*
University of California, San Francisco, San Francisco, California.
* Reprint requests and correspondence: Dr. Ann F. Bolger, University of California, San Francisco, 1001 Potrero Avenue, San Francisco, California 94103-4813. (Email: abolger{at}medsfgh.ucsf.edu).
In this issue of the Journal, Hoffmann et al. (1) report on their comparison of conventional and contrast-enhanced echocardiography, biplane cine angiography, and cardiac magnetic resonance for the detection of regional wall motion abnormalities. Their effort to acquire three or four of these modalities in the same patients creates an opportunity to consider the relative utility of these tests in a group of patients across a spectrum of left ventricular function. The study gave careful attention to acquisition and interpretation protocols for each method. Compared to a panel-defined "standard of truth," the sensitivity, specificity, and accuracy of all four methods were high. Even the least reliable method, cine angiography, performed better than expected (2). The authors’ conclusion favors the utility of contrast-enhanced echocardiography in comparison to the other methods under these best-case conditions.
The excellent performance of contrast-enhanced echocardiography is not surprising; many previous comparisons have demonstrated the utility of better endocardial definition in stress studies and difficult imaging conditions (3–5). The patient-specific characteristics that made the extra time, risk, and expense of contrast worthwhile are not discernible from this study. Contrast can be expected to improve detection of wall motion abnormality in situations with poor endocardial definition, foreshortened views, and parallel alignment of beam and endocardium; we are not told how many of the subjects had difficult conventional echocardiographic images, nor the impact of the abnormalities’ severity, extent, or location on test performance. Two-thirds of the subjects had subnormal left ventricular ejection fraction and were likely to have some regional wall motion abnormality, predisposing against the accuracy of unenhanced echocardiography.
An unexpected lesson from this study arose with the interpretation of cardiac magnetic resonance imaging (MRI) images. The authors initially considered using cardiac MRI as their "gold standard," given its previously documented excellent performance (6,7). To their surprise, the interobserver agreement was actually much worse than expected. One of two offsite reviewers consistently interpreted more abnormal wall motion than did the other reviewers. The resulting disparities led the two reviewers to discount the third reviewer’s interpretation for comparison to other methods.
How could this happen? Don’t excellent images ensure consistent interpretation and, more importantly, accurate conclusions? Answering that question requires us to consider important and interrelated aspects of the individual’s perception and interpretation of visual data, as well as the factors that influence confidence in the final interpretation.
Although many wall motion abnormalities were consistently appreciated, in 15 of 100 cases readers did not agree. No gold standard for wall motion was available, and therefore an expert panel established a surrogate "standard of truth." This panel was encouraged to interpret the images in combination with clinical information, such as direct evidence of coronary artery disease, as well as risk factors such as hypertension, hyperlipidemia, and diabetes. What convinced this panel about the presence or absence of regional wall motion abnormality when other expert colleagues differed? We can anticipate that the panel, faced with a set of cases that were by definition controversial, would have been careful to use all data available to them to make their decision. The panel was supplied clinical information (unavailable to onsite and offsite readers) that allowed them to estimate a pretest probability of abnormality. Such an index of suspicion can affect data interpretation by shifting the reader’s threshold for a positive finding. Whether the clinical information from the patients in this study weighted wall motion interpretation appropriately can be neither proven nor assumed.
The final common pathway of all image interpretation passes squarely through the swamp of individual perception. As a trainer of echocardiographers, it is always of great interest to me to observe the individual eccentricities of sonographers and fellows in terms of what they image, what they perceive, and what they conclude. In this study, it is very interesting that the best concordance between the readings for each imaging modality was consistently between the onsite reader and the expert panel. The explanation for this concordance may lie in very basic and often neglected factors in perception. Individual adjustments of monitor brightness, contrast, and playback rates can improve reader performance, just as the lack of optimization may outweigh superior image quality. Did onsite readers have the advantage of using their own familiar image display conditions and settings? Did panel members take greater care to apply settings best suited to their own preferences and experience? In both situations, extra care may have been taken to make the subtle adjustments in the images that increase the reliability of the analysis.
How individuals perceive complex visual data and incorporate it into elaborate cognitive processes is the focus of a rapidly growing body of science arising from the artificial intelligence and computer science fields, in which human perception and response to visual data are key. Cardiac imagers will be familiar with the concept that certain features of an image tend to "pop out." Searching for them is very fast, and our emotional response to them is confident (8,9). This "preattentive processing" is a powerful innate function and drives our appreciation of "excellent" images. Even subtle factors such as lighting conditions, visual acuity, and color scales have important influence on this function (9,10). Beyond this initial perception, cognition and the specific tasks at hand come into play. Finally, our level of confidence in our findings can alter the weight of preattentive processing among the other parts of the complex interpretation algorithm.
Confidence in image findings—and tolerance for imperfect confidence—is an important area that differentiates clinical readers. Although some echocardiography trainees are "doubting Thomases" who take tremendous coaching to perceive wall motion abnormalities, others are too ready to recognize abnormality even in inconclusive views and suffer from "intent to see." These different readers eventually attain clinical competence and reproducibility by developing their own approach to the data that, in my experience, always incorporates image manipulation to meet their own preferences. We need to encourage active interactions with visual data, not only in selection of the modality most appropriate to the patient at hand (for example, contrast may offer very little to the echocardiographic study of patients with excellent acoustic windows) and to data acquisition (avoidance of foreshortening, optimization of spatial and temporal resolution, careful attention to gain and other machine settings), but also to data review using the reader’s own preferred display settings. Gain, contrast, and display rate settings are not "one size fits all" and must be individually optimized. Confidence-boosting visual clues to abnormal wall motion, such as hinge points and distribution concordant with coronary artery territories, need to be sought. The final, and often missing, element in the interpretation needs to be an indication of certainty, or confidence estimate, that conveys the strength of the reader’s conviction regarding the researchers’ findings (11). In the end, even experienced reviewers will occasionally disagree, as this and many other studies have demonstrated.
In the absence of a true gold standard, the relative performance of these more and less costly approaches to wall motion assessment cannot be taken from this study in an absolute sense. What does seem likely is that the benefits of contrast echocardiographic methods are particularly valuable in a subset of patients whose characteristics impair conventional echocardiographic imaging. As the authors point out, LV function assessments are not easily interchangeable between different modalities. Beyond this, the example of the overreading by the offsite MRI reader demonstrates the ever-present possibility that a skilled reader, of any modality, will not be adequately calibrated to his or her peers because of differences in perception, confidence thresholds, or cognitive algorithm. When a confident decision cannot be reached, communication of that uncertainty is critical to avoid clinical misdirection.
Our imaging methods continue to provide us with better spatial and temporal resolution and clarity with respect to important anatomic and functional features. Future quantitative image analysis methods closely tailored to the clinical question will bolster our subjective analysis of these information-rich image data. Even those quantitative data will need to hold up under the scrutiny of unassisted inspection, however; findings that cannot be directly visualized are less credible. In the end, you can’t call what you can’t see, and you must be able to believe your own eyes.
 |
Footnotes
|
|---|
* Editorials published in the Journal of the American College of Cardiology reflect the views of the authors and do not necessarily reflect the views of JACC or the American College of Cardiology. 
|
References
|
|---|
1. Hoffmann R, von Bardeleben S, Kasprzak JD, et al. Analysis of regional left ventricular function by cineventriculography, cardiac magnetic resonance imaging, unenhanced and contrast-enhanced echocardiographya multicenter comparison of methods. J Am Coll Cardiol 2006;47:121-128.[Abstract/Free Full Text]2. De Rouen TA, Murray JA, Owen W. Variability in the analysis of coronary angiograms Circulation 1977;55:324-328.[Abstract/Free Full Text] 3. Hundley WG, Kizilbash AM, Afridi I, et al. Administration of an intravenous perfluorocarbon contrast agent improves echocardiographic determination of left ventricular volumes and ejection fractioncomparison with cine magnetic resonance imaging. J Am Coll Cardiol 1998;32:1426-1432.[Abstract/Free Full Text] 4. Reilly J, Tunick P, Timmermans R, Stein B, Rosenzweig B, Kronzon I. Contrast echocardiography clarifies uninterpretable wall motion in intensive care unit patients J Am Coll Cardiol 2000;35:485-490.[Abstract/Free Full Text] 5. Thomson H, Basmadjian AJ, Rainbird AJ, et al. Contrast echocardiography improves the accuracy and reproducibility of left ventricular remodeling measurements J Am Coll Cardiol 2001;38:867-875.[Abstract/Free Full Text] 6. Bellenger NG, Davis LC, Francis JM, et al. Reduction in sample size for studies of remodelling in heart failure by the use of cardiovascular magnetic resonance J Cardiovasc Magn Reson 2000;2:271-278.[Web of Science][Medline] 7. Mogelvang J, Lindvig K, Sondergaard L, et al. Reproducibility of cardiac volume measurements including left ventricular mass determined by MRI Clin Physiol 1993;13:1011-1019. 8. Bruce V, Green PR, Georgeson MA. Visual Perception. Physiology, Psychology, and Ecology. 3rd edition. United Kingdom: Psychology Press; 1996. 9. Ware C. Information Visualization. Perception for Design. San Francisco, CA: Morgan Kaufmann (Academic Press); 2000. 10. Tory M, Möller T. Human factors in visualization research IEEE Trans Visualization Comput Graphics 2004;10:72-84.[CrossRef] 11. Cosyns B, El Haddad P, Lignian H, et al. Contrast harmonic imaging improves the evaluation of left ventricular function in ventilated patientscomparison with transesophageal echocardiography. Eur J Echocardiogr 2004;5:118-122.[Abstract/Free Full Text]
This article has been cited by other articles:
|
|
|
|
 
A. N. DeMaria, O. Ben-Yehuda, G. K. Feld, G. S. Ginsburg, B. H. Greenberg, W. Y.W. Lew, J. A.C. Lima, A. S. Maisel, J. Narula, D. J. Sahn, et al.
Highlights of the Year in JACC 2006
J. Am. Coll. Cardiol.,
January 30, 2007;
49(4):
509 - 527.
[Full Text]
[PDF]
|
|
|
|
Top
References