Advances in the therapeutic options for cardiovascular disease, coupled with improvements in imaging technology, have led to an explosive growth in the performance of cardiovascular imaging. Yet this growth is challenging to interpret as it may represent appropriate use, underuse (i.e., the failure to provide services from which the patient would likely benefit), and/or overuse (i.e., the provision of services that may not be necessary or may expose the patient to greater potential harm than benefit).

The array of noninvasive cardiovascular diagnostic tools has expanded in recent years with innovations in contrast agents, molecular targeted radionuclides for positron emission tomography (PET) and single-photon emission computed tomography (SPECT) imaging, perfusion echocardiography, cardiac computed tomography (coronary angiography and calcium scoring), and cardiac magnetic resonance imaging (myocardial structure and viability). Current patterns of utilization are characterized not only by growth but also by significant regional variation (1). Faced with uncertainties about the true nature of current utilization rates and patterns, clinicians, payers, and patients are demanding criteria to evaluate the “appropriateness” of cardiovascular imaging (2).

Ideally, such criteria would arise from high-quality research evaluating the benefits and risks of performing imaging studies for various common clinical scenarios. Additionally, a complete evaluation of appropriateness might also include a comparison of the relative marginal cost and benefits of each imaging modality. Regrettably, there is currently insufficient evidence to make such evaluations across a broad spectrum of potential clinical indications and/or imaging modalities.

In the absence of ideal evidence for judging the appropriateness of interventions, the American College of Cardiology Foundation (ACCF) Appropriateness Criteria Working Group proposes a method for evaluating the appropriateness of cardiovascular imaging that examines the appropriateness of a single modality.

Despite the clear need, there is no consensus on how to determine the appropriateness of an imaging study. A group from the RAND Corporation, in collaboration with researchers from the University of California, Los Angeles (UCLA), initially described a method for determining the appropriateness of medical and surgical procedures, including cardiovascular procedures (3- 8). However, unlike procedures where there is a defined therapeutic benefit, imaging studies are performed with different goals in mind. The potential purposes of cardiovascular imaging include the detection or exclusion of disease, as well as risk stratification and the evaluation of therapeutic efficacy. Each of these goals is tied to specific clinical situations and often includes patient-level factors and test characteristics associated with the imaging modality. For example, certain imaging goals may favor specific test characteristics, such as highly sensitive tests for the exclusion of disease.

Additionally, imaging studies may have negative consequences, such as poor specificity with a high number of false positives leading to unwarranted further procedures or tests. Such risks and costs are generally not factored into the definition of procedural appropriateness, yet these factors have an obvious impact upon selecting an imaging modality and determining whether it is needed. Inaccurate test results represent true risks to the patient and costs to both patients and the health care system. Furthermore, cost considerations are often implicitly factored into decisions during clinical care once the added incremental benefit to clinical judgment of an imaging procedure has been determined. Therefore, to identify the true risks of imaging, both inherent risks and downstream effects, including costs, must be considered. As such, if the imaging study provides little incremental information for an indication over standard clinical judgment and care, then cost considerations should contribute to deeming the procedure inappropriate. In this manner, a determination of appropriateness should aim to overtly replicate the clinical care process of benefit and risk assessment for an imaging study.

Finally, it was believed that the perspective for the determination of appropriateness should be that of the patient. The evaluation should seek to determine how the information gained from the cardiovascular imaging study will influence subsequent care to improve patient outcomes including survival and health status (a patient’s symptoms, function, and quality of life). These issues demand modifications to the traditional definition used for procedural appropriateness.

In summary, a definition of an imaging test’s appropriateness must include test performance characteristics for a clinical indication, the potential negative consequences of imaging, an understanding of the implicit impact of cost on clinical decision making, and an explicit understanding of how the test results might lead to care that could improve the patient’s chances for better survival or improved health status. With these considerations in mind, a definition of appropriate cardiovascular imaging is presented in (Figure 1). Simply stated, the goal is to determine whether an experienced, evidence-based physician, faced with a specific clinical situation, would find performing the imaging study an acceptable step in providing good clinical care.

Aside from the shortcomings of the invasive procedure-based appropriateness definition, the overall RAND/UCLA Appropriateness Method (Figure 2) has many merits. The American College of Radiology (ACR) has applied the modified Delphi method used in the RAND/UCLA process in the development of appropriateness criteria for imaging. Members of the ACR panels review the appropriateness of multiple imaging modalities over numerous rounds to arrive at consensus recommendations of the relative benefits and risks of selecting one test over another for specific clinical indications (9- 13). However, the current approach used by the ACR still leaves the following questions unanswered.

The ACR criteria for cardiovascular indications focus on primarily obtaining a diagnosis based on symptoms of indeterminate origin. As stated earlier, cardiovascular imaging goals extend beyond the detection of disease to risk assessment and influence on patient management.

The ACR criteria focus on a limited number of topics that tend to be acute presentations with the goal of diagnosis. Management of patients with cardiovascular disease often involves evaluation of patients with previously known disease, such as chronic stable angina. The longitudinal obligation of the cardiovascular specialist to care for patients creates a need to monitor for important changes in patients’ disease status and/or risk for adverse outcomes over time.

Generally, ACR criteria address broad patient indications absent of the context of a patient history or physical examination. The role of pre-test probability or risk is not addressed. As such, imaging tests that provide a preliminary basic assessment of the patient may be unnecessary and may duplicate later tests that are required and are more appropriate given needed therapeutic interventions for a particular patient’s clinical situation.

Care of the cardiovascular patient includes the important clinical situations of identifying an adverse prognosis in asymptomatic patients and monitoring correctly diagnosed and treated patients over time. These were not explicitly performed with the ACR appropriateness criteria.

Although selection among multiple modalities is important, specific clinical scenarios and local conditions such as technical skill level and laboratory qualities often influence the initial test choice. Ranking the different modalities without this specific information is difficult. While comparisons across modalities should be done, the first step is to define appropriateness within each modality.

The ACR uses up to four rounds of rating, if necessary, to attain consensus. The ACCF use of the RAND/UCLA method, which is limited to two rounds of ratings, does not try to promote consensus and potentially reduce the real differences of clinical opinion that may underlie indications rated as uncertain.

The Working Group recommends use of the RAND/UCLA Appropriateness Method with a focus on answering the questions in the preceding text. To do so effectively, the Working Group determined it would be best to rate appropriateness one modality at a time (Figure 3). This document addresses how one might then choose among different imaging modalities, deemed appropriate for an individual indication, so as to ensure efficient imaging. The steps of the RAND/UCLA Appropriateness Method, as they are proposed to be applied to imaging, are outlined in the following text.

The first step requires reviewing the literature and developing the clinical indications to be rated. As mentioned, for cardiovascular imaging, imaging studies may be performed for diagnosis, risk stratification and prognosis, therapeutic management decisions, or simply to exclude disease. Therefore, the indications to be rated for each modality should capture the general presenting symptoms, the clinical reason for imaging, the patient population, and other specific factors. Additionally, the list of clinical indications must be exclusive of other clinical indicators and represent common practice indications. The clinical indication list also should attempt to include clinical scenarios for which there is practice variation. Health plans and employers may also be able to provide additional data on high-volume clinical indications to be considered in developing a comprehensive list.

Ideally, the clinical indications would be developed by a few general cardiovascular clinicians and by specialists in the imaging modality being evaluated. Indications may be in part derived from existing guidelines for cardiovascular imaging modalities, where available. However, common scenarios not included in the guidelines should be incorporated in an effort to assess appropriateness.

Next, a standardized literature review should be initiated for the clinical indications, as currently performed in the development of the American College of Cardiology/American Heart Association (ACC/AHA) clinical practice guidelines (14). As suggested by the RAND methodology, “evidence tables” should be formed when significant evidence is available for a specific indication or set of indications. Because the majority of imaging studies have incorporated observational cohort designs, particular emphasis should be placed on identifying common sources of bias, such as referral bias, selected populations, and blinding (15). Many of these are incorporated into the Standards for Accurate Reporting of Diagnostic studies (STARD) initiative checklist (16). The ACC/AHA guidelines also can be useful sources of literature reviews related to most cardiovascular imaging modalities (17- 18).

The second step for determining the appropriateness of a cardiovascular imaging modality is review of the clinical indications by an expert panel. It is recommended that the expert panel consist of 9 to 15 people, ideally including physicians from different specialties, experts in different imaging modalities, general cardiovascular and other medical practitioners who often order such imaging studies, payers, and health services researchers. This type of diverse panel composition helps to ensure the production of equitable and reproducible ratings (19- 21).

Each panelist should then complete a first round of ratings to provide indications in isolation and without discussion with other participants on the panel. The panelists assign the ratings to each indication based on the following scoring system:

The panelists are to rate the appropriateness of an indication based on the available evidence for each specific indication. Panelists also should use reproducibility and patient-specific information in determining the appropriateness for each indication (Figure 4). When available, the ACC/AHA clinical practice guideline recommendations also should be included for the panel to review. Although this information is provided as a guide, it does not dictate a particular rating of any indication. In addition, where there is a lack of evidence or guideline, clinical experience forms the basis for the appropriateness rating.

Once first round ratings are complete, a panel meeting should be convened to discuss the indication list. At the panel meeting the ratings are presented as a distribution of scores and a median score. Additionally, all panel members receive identification of their personal score in relation to the distribution. The purpose of the panel meeting is not to reach consensus but to provide panel members an opportunity to share perspectives. The ratings with significant widespread distribution identify areas for clarification and serve as the basis for discussion among members. Importantly, sources of disagreement based on misunderstandings of the indications are addressed by refining the wording of indications where required. A second round of rating by individual panel members should occur either at the end of the face-to-face meeting or during the weeks following the meeting.

The final step is tabulating the appropriateness ratings from the second round. Each indication has been scored by the panel and the median score is used to determine the final appropriateness score.

After the second rating round, a procedure to measure the level of agreement among panelists (e.g., the BIOMED Concerted Action on Appropriateness definition) is applied to the final ratings. Under the BIOMED method, which is useful for panels with 11 to 13 members, agreement is defined as three or fewer panelists rating outside the three-point region containing the median. Disagreement is defined as at least 4 panelists rating in each extreme (1 to 3 and 7 to 9). For those clinical indications for which the panel cannot agree, the imaging study is marked as uncertain for that indication regardless of the median score.

Interpretation and context of the final appropriateness scores are important. Clinicians and payers will be faced with a list of clinical indications for a single imaging modality that are deemed appropriate with scores of 7 to 9, uncertain with scores of 4 to 6, and inappropriate with scores of 1 to 3. An uncertain classification generally indicates there is not sufficient evidence, experience with the imaging study, or detailed patient characteristics for the indication to definitively categorize an imaging procedure as appropriate. It does not indicate that the imaging test should not be performed in that particular situation, or that there is no evidence of benefit, but rather that more information and/or research could benefit updating the criteria.

Payers should be aware that the appropriateness determinations provide general criteria for imaging modalities, with the understanding that clinicians may be faced with unique issues regarding individual patients. A consistent pattern of inappropriate testing by a provider should prompt further review. Conversely, providers may also decide not to perform an imaging study that is deemed “appropriate,” and this may simply relate to different levels of clinical certainty for specific indications or other factors such as patient preference. Finally, payers should note that the technical panel and clinical community do not consider uncertain indications as those that should not be performed or reimbursed, as many may be the standard of care in specific regions of the country. Rather, the uncertain indications are those where the opinions of the panel vary. Indications with high clinical volume that are rated as uncertain in the appropriateness review may suggest areas for increased focus and research. The appropriateness evaluations will also provide an opportunity to identify indications for which numerous modalities are appropriate. These again will be areas where high-quality clinical trials can improve patient care and potentially reduce cost.

Appropriateness criteria for cardiovascular imaging differ from guidelines and performance measures in important ways. Guidelines attempt to provide a comprehensive review of the available evidence and best practices for the management of a clinical condition such as heart failure. Performance indicators capture aspects of care recommended in the guideline that have been indisputably proven to improve patient outcomes and for which data can be collected that are interpretable, actionable, and feasible (22). In contrast, appropriateness criteria for imaging explicitly evaluate the relative benefits and risks of an imaging study for a specific indication to determine whether it is “reasonable” to consider performing the study.

Both validation and evaluation of the proposed appropriateness ratings by modality are essential steps. The first evaluation should be to determine whether the indications chosen for a specific modality can be applied. Do the indications cover the majority of reasons for which the imaging test is performed? Can the indications be easily abstracted from current care data? Are they reproducible? These types of questions can be addressed by collaboration with medical directors, payers, and large clinical practices. Additionally, the entire appropriateness ratings for a modality should be evaluated as described by the initial RAND methodology with either a retrospective review of previous imaging or with a prospective review of indications for an imaging study to determine the rate of appropriate imaging, true positive and true negative rates and, ideally, the consequences for patient outcomes.

The Working Group recognizes that the method for evaluating cardiac imaging will continue to adapt and evolve with experience and increased national focus. However, even with these anticipated changes, the growing demand for a framework with which to evaluate current patterns of care warrants proceeding with an initial attempt to define the issues involved and to provide an initial method for evaluating imaging use. In addition to the results from the field testing of appropriateness ratings for a modality, new evidence in imaging will likely require updates to the appropriateness ratings.

Once the appropriateness evaluation is complete across multiple imaging modalities, clinicians and payers will naturally attempt to compare modalities not only for predictive accuracy but also for criteria such as cost-effectiveness or “efficiency” across similar clinical indications. These comparisons will be especially challenging as issues surrounding each imaging appropriateness review, the imaging test’s specific characteristics, and exact patient factors for each indication will likely vary substantially for each imaging modality.

Nevertheless, efforts to formally evaluate efficient imaging for specific indications addressed by a single imaging modality or by multiple modalities are planned. Unfortunately, an assumed starting point for an evaluation of efficient imaging is absent from current evidence. Explicit standard acceptable values for true positive and true negative results from imaging studies for specific indications have not been developed. As such, an alternative strategy for determining the efficient imaging utilization is required. Although some implicit weighing of costs occurs during the evaluation of a specific imaging modality, more explicit weight can be given comparing different modalities. A comparison is possible where several imaging modalities have been previously deemed appropriate. Cost comparison at this point may allow the relative weighing of costs per imaging modality with comparative benefits per clinical indication, thus providing a measure of efficiency.

As with appropriateness, the initial step in creating recommendations of imaging efficiency requires constructing a list of relevant clinical indications. This would include determining those common clinical indications for which there are not sufficient data to recommend a single imaging strategy. All imaging modalities that were deemed to be appropriate for those indications should be evaluated. Clinicians and payers should participate in identifying among those clinical indications for which there is high clinical volume and need for efficient imaging recommendations.

The panel that evaluates efficient imaging recommendations should represent a diverse range of expertise. The group should include clinicians with medical expertise in the area of the indication, clinicians with imaging expertise for all the considered modalities, health service researchers, payers, and invasive cardiologists or surgeons if the indication may lead to a procedure. This group would then score the various imaging modalities for the specific indication through a modified Delphi process as described previously. Great care must be undertaken in the selection of panel members to avoid an undue bias in favor or against any and all imaging procedures.

The method for analysis of efficient imaging will require significant future work. Some potential metrics for analysis of efficient imaging include test characteristics such as the relative cost of the different modalities, the relative strength of test performance characteristics of each imaging modality with regards to the specific indication, and the comparative availability of evidence for the individual modalities. Unintended consequences for each modality, both additional ancillary information that may provide the final diagnosis and risks from the procedure, should also be considered in efficiency determination. Moreover, variance in local availability and quality of various techniques needs to be considered. Finally, consideration of patient preferences and comfort must be included.

Essential to this type of evaluation are studies comparing strategies using different imaging modalities that carefully capture all of these aspects including diagnostic yield, cost, reproducibility, and downstream patient outcomes. Currently, the majority of evidence for cardiovascular imaging describes the diagnostic yield compared to a reference standard, or the prognostic significance of findings from the imaging study.

Few studies exist that randomize patients with specific presenting symptoms to one diagnostic strategy versus another (23). In fact, although other study designs are logistically easier, randomized trials evaluating two different diagnostic strategies may be the best pragmatic way to determine the efficiency of different imaging modalities (24).

Thus, the efficient imaging recommendation that provides a comprehensive evaluation of different cardiovascular imaging modalities for a specific indication, incorporating evidence on different test performance, cost, and downstream outcomes, remains an important goal. Although the current available literature may not be sufficient, it is hoped that continued research in cardiovascular imaging will allow evaluation of efficient imaging in the future.

Cardiovascular imaging has enjoyed fantastic technical advances over the last 25 years. These advances, coupled with significant improvements in the therapeutic options for patients with heart disease, provide an unparalleled opportunity to decrease the burden of cardiovascular disease. However, a great threat to achieving this goal is the inappropriate application of these imaging modalities resulting in substantial, unexplained regional variability and increased attendant costs.

This document provides a framework to determine appropriate clinical indications for an imaging modality. Additionally, the future direction for rating the efficiency of imaging for specific indications across single and multiple appropriate modalities is discussed. These goals for cardiovascular imaging are both economically and clinically critical. Long-term success of such efforts will require high-quality research for common clinical indications to extend the evidence base for future panels. Such research will reduce the number of uncertain indications and confirm the validity of both appropriate and inappropriate ratings. It is with this type of work and ongoing evaluation of cardiovascular imaging that patients and society will truly reap the benefit from advances in cardiovascular imaging. As such, the development of appropriateness criteria and efficiency considerations for cardiovascular imaging can serve as an important guide for delivery of high-quality clinical care with imaging and provide a structure to evaluate utilization, including underuse and overuse of current modalities.