|
|
||||||||||
|
J Am Coll Cardiol, 2006; 47:121-128, doi:10.1016/j.jacc.2005.10.012
(Published online 9 December 2005). © 2006 by the American College of Cardiology Foundation |










* University Rheinisch-Westfälischen Technischen Hochschule Aachen, Aachen, Germany
Clinic Johannes Gutenberg University Mainz, Mainz, Germany
Medical University, Lodz, Poland
University Charite, Berlin, Germany
|| Academic Hospital Dijkzigt Rotterdam, the Netherlands
¶ Deutsches Herzzentrum, Munich, Germany
# Hopital du Haut Leveque, Pessac Cedex, France
** University Charite, Berlin, Germany

University Bonn, Bonn, Germany

Bracco-Altana-Pharma, Konstanz, Germany

Bracco Diagnostics, Princeton, New Jersey
|||| Cliniques Universitaires Saint-Luc, Brussels, Belgium
¶¶ John Radcliffe Hospital, Oxford, United Kingdom.
Manuscript received February 1, 2005; revised manuscript received May 21, 2005, accepted May 31, 2005.
* Reprint requests and correspondence: Dr. Rainer Hoffmann, Medical Clinic I, University RWTH Aachen, Pauwelsstraße 30, 52074 Aachen, Germany. (Email: rhoffmann{at}ukaachen.de).
| Abstract |
|---|
|
|
|---|
BACKGROUND: Detection of RWMA is integral to the evaluation of LV function.
METHODS: In 100 patients, cineventriculography and unenhanced and contrast-enhanced echocardiography were performed. Fifty-six of the patients underwent additional cMRI. RWMA were assessed referring to a 16-segment model for cMRI, unenhanced and contrast echocardiography. Cineventriculography was evaluated on a 7-segment model. Hypokinesia in one or more segments defined presence of RWMA. Interobserver agreement among three readers was determined within each imaging modality. Intermethod agreement between imaging modalities was analyzed. A standard of truth for the presence of RWMA was obtained by an independent expert panel decision (EPD) based on clinical data, electrocardiogram, coronary angiography, and blinded information from the imaging modalities.
RESULTS: Sixty-seven patients were found to have an RWMA by EPD. Interobserver agreement expressed as kappa coefficient was 0.41 (range 0.37 to 0.44) for unenhanced echocardiography, 0.43 (range 0.29 to 0.79) for cMRT, 0.56 (range 0.44 to 0.70) for cineventriculography, and 0.77 (range 0.71 to 0.88) for contrast echocardiography. Contrast enhancement compared to unenhanced echocardiography improved agreement of echocardiography related to cMRI (kappa 0.46 vs. 0.29) and related to cineventriculography (kappa 0.59 vs. 0.28). Accuracy to detect EPD-defined RWMA was highest for contrast echocardiography, followed by cMRI, unenhanced echocardiography, and cineventriculography.
CONCLUSIONS: Analysis of RWMA is characterized by considerable interobserver variability even using high-quality imaging modalities. Interobserver agreement on RWMA and accuracy to detect panel-defined RWMA is good using contrast echocardiography.
| ||||||||||
The objective of this multicenter study was: 1) to determine the IOA in the definition of RWMA in a comparative manner for each of the applied imaging modalities (unenhanced echocardiography, contrast echocardiography, cineventriculography, and cMRI); 2) to determine the agreement between different imaging modalities in the definition of RWMA; and 3) to assess the adequacy of determined RWMA related to a standard of truth on regional LV function, as defined by an expert panel based on clinical, electrocardiographic, angiographic, and imaging data. The design of this study allowed a direct comparison of the techniques on the same patients. Blinded on-site and off-site readings using experienced independent core laboratories were performed for each imaging technique according to prospectively defined standards.
| Methods |
|---|
|
|
|---|
Recommendations on the performance of image acquisition were prospectively defined for each imaging modality to secure uniform and interpretable image datasets from all participating institutions. Adherence to the predefined imaging protocols was monitored during the enrollment period of this multicenter trial.
Analysis of image datasets for RWMA was performed for each imaging technique by one on-site reader (OnR) as well as two independent off-site readers (OffR) not affiliated with the participating centers and who were unaware of the clinical data and the results of the other imaging techniques. Recommendations on the evaluation of regional LV function for each imaging technique were prospectively defined and provided as guidelines both to the OnR at the study sites and to the unaffiliated OffR using independent core laboratories (Appendix). For each analyzed segment, regional wall motion was defined as normokinetic, hypokinetic, akinetic, or dyskinetic. Whenever the regional function could not be defined owing to insufficient image quality, the function was assumed to be normal.
The study was conducted according to good clinical practice and in compliance with local regulatory requirements. The research protocol was approved by the applicable central and local institutional ethics committees. All patients gave written informed consent to participate in the study.
Patients. Eight European centers experienced in the applied imaging techniques enrolled 100 patients planned for routine coronary angiography. Patients with acute myocardial infarction were excluded. At each center, consideration was taken in patient enrollment for an even distribution within three predefined ejection fraction groups (>55%, 35% to 55%, and <35%) based on results from cineventriculography. Inclusion into the study required sinus rhythm and an interpretable cineventriculography.
Echocardiography. Two-dimensional echocardiography using tissue harmonic imaging for unenhanced and contrast-specific imaging for contrast-enhanced echocardiography was performed with a commercially available ultrasound scanner (Sonos 5500; Philips, Andover, Massachusetts). Written recommendations were provided for the uniform use of equipment presets, imaging conventions, imaging sequence, and annotations. Apical four-, two-, and three-chamber views were acquired without and with contrast enhancement. For unenhanced imaging, harmonic imaging (mechanical index [MI] 1.6, gain 50%, compression 70%) was used, whereas for contrast-specific imaging a low MI of 0.3 was preselected (gain 60%, compression 15%). Optimization of imaging conditions for endocardial border definition was performed by modulation of transmit power, gain, focus, and dynamic range as required.
For contrast-enhanced echocardiography, a 20-gauge intravenous catheter was introduced into the right antecubital vein. Sulphur hexafluoride microbubbles (SonoVue, Bracco Imaging, Milan, Italy) were administered with a starting infusion rate of 1 ml/min followed by subsequent rate adjustments and additional bolus injections if required to reach homogeneous LV cavity opacification without attenuation.
Analysis of unenhanced and contrast-enhanced echocardiograms by the OnR and OffR was performed in sequence. After finalization of unenhanced image evaluation, subsequent separate evaluation of contrast-enhanced images was performed. For each of the 16 LV segments defined by the American Society of Echocardiography, regional systolic LV function was determined (5).
Cineventriculography. Standard biplane cineventriculography was performed using a 30° right anterior oblique projection and a 60° left anterior oblique projection with injection of at least 30 ml contrast medium at a flow rate of 12 to 14 ml/s. Frame rate was set at 30 Hz. Regional systolic LV function was determined for each of seven segments (anterolateral, anterobasal, apical, posterobasal, posterolateral, diaphragmal, and septal) (4).
cMRI.
Electrocardiogram (ECG)-triggered cMRI investigations using a steady-state free-precession (SSFP) sequence at a field strength of 1.5 T during breathhold were performed for cardiac function assessment at five of the participating centers with on-site cMRI facilities using either Siemens Sonata (Siemens, Erlangen, Germany) or Philips Gyroscan (Philips, Eindhoven, the Netherlands) systems. Four-chamber, two-chamber, and three-chamber views were acquired with a temporal resolution of
50 ms using a special volume-adapted surface coil. Regional systolic function was determined for each of 16 segments considering wall thickening during systole as well as endocardial inward motion (6).
Definition of true regional LV function. To define a standard of truth for the presence of RWMA, a consensus decision was made for each patient between two independent panelists based on clinical data (known cardiomyopathy, history of myocardial infarction, and prior revascularization procedure), ECG, coronary angiography, and results of all image readings (from OnR and OffR). To define the standard of truth, the two panelists adhered to a predefined decision algorithm (Fig. 1). They had to consider primarily the clinical information and then the results of the given reads. Known cardiomyopathy, history of myocardial infarction in combination with ECG abnormalities, angiographically proven significant coronary artery disease, or previous coronary revascularization and RWMA in at least one imaging method by at least two readers indicated evidence for RWMA. No history of myocardial infarction in combination with normal ECG, no previous coronary revascularization, angiographic exclusion of coronary artery disease, and in all imaging modalities no RWMA by at least two readers was indicative for no RWMA. In case no decision could be made based on these data, the results of all imaging reads were considered. In cases with cMRI imaging, at least 10 points (one point per reader and imaging modality) on a consensus scale of 12 points had to be reached in order to obtain a result on the presence of a RWMA. This was obtained by consensus of at least 10 of the 12 readings (four imaging modalities and three readers per imaging modality). In cases without cMRI, at least 7 points on a consensus scale of 9 points (three imaging modalities and three readers per imaging modality) had to be reached. Furthermore, clinical data had to be compatible with this result.
|
Statistics. Statistical analysis was performed using the SPSS software package (SPSS, Chicago, Illinois). Continuous variables are presented as mean ± SD. The Cohen kappa coefficient was calculated first to evaluate IOA for each pair of observers (16). To evaluate IOA among three readers within each imaging modality, the mean kappa and its 95% confidence interval was calculated. A generalization of the Cohen reliability kappa for multiple readers and two categories outcome was estimated to support the results of mean kappa (17). Kappa was also obtained to evaluate intermethod agreement on detection of RWMA evaluated by one OffR of each imaging modality. The same analyses were performed for the agreement between OffR/OnR and the panel decision in terms of RWMA within each individual imaging modality. The kappa coefficient of agreement was graded as follows: 0 to 0.2 = poor to slight; 0.21 to 0.4 = fair; 0.41 to 0.6 = moderate; 0.61 to 0.8 = substantial; and 0.81 to 1.0 = nearly perfect. To evaluate the diagnostic performance of each imaging modality in terms of detection of RWMA, sensitivity, specificity, and accuracy were estimated using the panel decision as gold standard. Sensitivities, specificities and accuracies were compared using McNemar test for dependent samples. A value of p < 0.05 was considered statistically significant.
| Results |
|---|
|
|
|---|
|
|
|
|
|
|
|
| Discussion |
|---|
|
|
|---|
Regional LV function. Analysis of regional systolic function at rest as well as under stress conditions has considerable implications for patient management and prognosis. In the analysis of regional LV function, different parameters come into play, in particular, endocardial border inward motion and myocardial wall thickening. Interpretation thresholds for rating regional LV function as abnormal are subjective. Interobserver variability is a well known problem in the interpretation of cardiac imaging tests (8). Accuracy and reliability of a diagnostic test are, however, a prerequisite for adequate and consistent patient management and the minimization of operator dependence in the test interpretation is therefore of clinical importance.
Current study. This study allows for the first time a direct within-patient comparison of the different methods to evaluate the presence of RWMA using up-to-date technology for all four imaging modalities. For each method, similar analysis conditions were established using one OnR and two OffR at independent core laboratories. The first aspect of this study was the IOA between the different readers within each imaging modality. For cineventriculography the average kappa was 0.56 and thus better than previously indicated in an analysis between 11 observers which reported an average kappa of 0.34 (18). The better IOA in the current study may be due to the fact that all cineventriculograms were performed using current biplane technology and that technical adequacy of cineventriculography was a prerequisite for patient inclusion. For unenhanced echocardiography the average kappa over three readers was 0.41. In stress echocardiographic studies kappa values have been reported to be 0.55 using current unenhanced echocardiographic technology and only 0.37 if image quality is impaired (8,10).
The mean kappa on presence of RWMA for the three cMRI readers was only 0.43 in the current study, which was mainly related to a substantial overread by OffR2 resulting in high interobserver variability with OffR1 and OnR. The IOA between OnR and OffR1 was on a level expected for cMRI, with a kappa of 0.79. Cardiac magnetic resonance imaging is known for its high accuracy and reproducibility in the assessment of LV volumes and ejection fraction owing to the full-volumetric data acquisition and the usually excellent image quality (19,20). However, regarding regional function analysis recent publications have indicated IOA for RWMA to be far from perfect also for cMRI. Paetsch et al. (14) reported from a study on 150 patients undergoing dobutamine stress cMRI a mean kappa of 0.55 on test interpretation among three expert readers at different centers. In another study, which involved only readers from one center, the reported kappa values on interpretation of stress cMRI studies were 0.70 and 0.71 (15). For contrast-enhanced echocardiography, an overall high level of IOA on the definition of RWMA was found. Previous studies had already indicated that contrast enhancement improves analysis of global LV function and has a positive impact on regional function analysis (1113,21). In a comparative single-center study on 40 subjects between cMRI and echocardiography the agreement in the detection of wall motion abnormalities was 82% using unenhanced imaging and increased to 100% after administration of contrast (11).
The current study results indicate that IOA on regional wall motion assessment iswithin a range of 0.41 to 0.77 for averaged kappa valuesmethod and thus image-quality dependent. However, the findings also show that nearly perfect agreement was not achieved with any of the imaging modalities and that reader differences continue to exist even with high image quality owing to the different subjective reader thresholds in the evaluation of RWMA.
The second aspect evaluated in the study was the agreement between different methods in the assessment of RWMA. In the pairwise intermethod comparison, kappa values ranged between 0.28 and 0.60. Intermethod agreement was poor between unenhanced echocardiography and both cineventriculography and cMRI, whereas fair agreement was found for cineventriculography versus cMRI and for contrast-enhanced echocardiography versus all other modalities. The findings of the present study underline the importance to report the applied method used to evaluate a RWMA. This extends previous observations on global LV function analysis (22), that results on LV function assessments are not easily interchangeable between different imaging modalities.
To define a standard of truth on RWMA two experienced cardiologists had to find a consensus for each patient. This approach was selected in order to allow the assessment of accuracy in the definition of RWMA for each of the applied imaging techniques. Accuracy of cMRI was affected by a high false positive rate from a single reader, whereas diagnostic performance was extremely high for the remaining two readers.
For cMRI, limitations in frame rate and the potential migration of myocardial segments through imaging planes may cause difficulties in the reading of regional wall motion. Suboptimal baseline image quality is a potential cause for incorrect wall motion ratings using echocardiography. Administration of contrast seems to overcome the image quality-related limitations and results in more accurate and reliable regional wall motion assessment.
Study limitations. cMRI was performed at only five centers, allowing only 56 patients to be included. This reflects the limited number of centers able to perform all applied imaging modalities. However, there were no differences in patient characteristics and frequency of RWMA defined by echocardiography between all patients and the subgroup with cMRI.
The number of segments used to evaluate regional LV function by cineventriculography was only 7 as compared to 16 for the other imaging modalities. This is in part related to the biplane display of LV function instead of the triplane display with echocardiography and cMRI.
There is no objective gold standard for the definition of RWMA with which each imaging modality could be easily compared. We tried to circumvent this problem by defining a "standard of truth" based on a panel decision between two blinded expert cardiologists considering all available information in a well-defined decision algorithm. One might argue that cMRI should be used as the gold standard. The considerable variability between the three cMRI readers indicates that cMRI has limitations in this respect. However, in all of the 56 cMRI patients the panel decision was in agreement with at least two of the three cMRI readers.
All readers in this study were trained experts. The reported reader agreement and accuracy to detect RWMA is likely to reflect the best possible level whereas it may not reflect such a setting with less trained readers.
Conclusions. Analysis of RWMA is characterized by a considerable interobserver variability even using high-quality imaging modalities, owing to different reader thresholds in the evaluation of RWMA. Unenhanced echocardiography and cineventriculography have the lowest IOA in the evaluation of RWMA. These methods are also associated with a low intermethod agreement with other imaging modalities on RWMA. Contrast-enhanced echocardiography as compared to unenhanced echocardiography significantly improves IOA on the presence of RWMA and results in a high accuracy to detect consensus-defined regional function abnormalities.
| Appendix |
|---|
|
|
|---|
| Acknowledgments |
|---|
| Footnotes |
|---|
| References |
|---|
|
|
|---|
This article has been cited by other articles:
![]() |
G. Nucifora, N. A. Marsan, H.-M. J. Siebelink, J. M. van Werkhoven, J. D. Schuijf, M. J. Schalij, D. Poldermans, E. R. Holman, and J. J. Bax Safety of contrast-enhanced echocardiography within 24 h after acute myocardial infarction Eur J Echocardiogr, July 16, 2008; (2008) jen167v1. [Abstract] [Full Text] [PDF] |
||||
![]() |
T. H. Marwick and M. Schwaiger The Future of Cardiovascular Imaging in the Diagnosis and Management of Heart Failure, Part 1: Tasks and Tools Circ Cardiovasc Imaging, July 1, 2008; 1(1): 58 - 69. [Full Text] [PDF] |
||||
![]() |
R. Hoffmann, A. C. Borges, J. D. Kasprzak, S. von Bardeleben, C. Firschke, C. Greis, M. Engelhardt, H. Becher, and J. L. Vanoverschelde Analysis of myocardial perfusion or myocardial function for detection of regional myocardial abnormalities. An echocardiographic multicenter comparison study using myocardial contrast echocardiography and 2D echocardiography Eur J Echocardiogr, December 1, 2007; 8(6): 438 - 448. [Abstract] [Full Text] [PDF] |
||||
![]() |
R. Olszewski, J. Timperley, S. Cezary, M. Monaghan, P. Nihoyannopoulis, R. Senior, and H. Becher The clinical applications of contrast echocardiography Eur J Echocardiogr, June 1, 2007; 8(3): s13 - s23. [Full Text] [PDF] |
||||
![]() |
H. Jaochim Nesser, L. Sugeng, C. Corsi, L. Weinert, J. Niel, C. Ebner, R. Steringer-Mascherbauer, F. Schmidt, G. Schummers, R. M Lang, et al. Volumetric analysis of regional left ventricular function with real-time three-dimensional echocardiography: validation by magnetic resonance and clinical utility testing Heart, May 1, 2007; 93(5): 572 - 578. [Abstract] [Full Text] [PDF] |
||||
![]() |
A. N. DeMaria, O. Ben-Yehuda, G. K. Feld, G. S. Ginsburg, B. H. Greenberg, W. Y.W. Lew, J. A.C. Lima, A. S. Maisel, J. Narula, D. J. Sahn, et al. Highlights of the Year in JACC 2006 J. Am. Coll. Cardiol., January 30, 2007; 49(4): 509 - 527. [Full Text] [PDF] |
||||
![]() |
R. Hoffmann The demystification of magnetic resonance imaging? Eur. Heart J., June 2, 2006; 27(12): 1394 - 1395. [Full Text] [PDF] |
||||
![]() |
A. F. Bolger Do You See What I See?: Agreeing to Disagree J. Am. Coll. Cardiol., January 3, 2006; 47(1): 129 - 130. [Full Text] [PDF] |
||||
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| HOME | SUBSCRIPTIONS | CURRENT ISSUE | PAST ISSUES | CARDIOSOURCE | SEARCH | HELP | FEEDBACK |