Decision Analysis of Antibiotic and Diagnostic Strategies in Ventilator-associated Pneumonia

Decision Analysis of Antibiotic and Diagnostic Strategies in Ventilator-associated Pneumonia David E. Ost, Charles S. Hall, Gnanaraj Joseph, Christine Ginocchio, Susan Condon, Emily Kao, Michele LaRusso, Richard Itzla, and Alan M. Fein Center for Pulmonary and Critical Care Medicine, Department of Laboratory Medicine, Pharmacy, and Finance, North Shore-Long Island Jewish Health System, Manhasset; New York University School of Medicine, New York; and SUNY at Stony Brook, Stony Brook, New York The optimal strategy for ventilator-associated pneumonia remains controversial. To clarify the tradeoffs involved, we performed a decision analysis. Strategies evaluated included antibiotic therapy with and without diagnostic testing. Tests that were explored included endotracheal aspirates, bronchoscopy with protected brush or bronchoalveolar lavage, and nonbronchoscopic mini-bronchoalveolar lavage (mini-bal). Outcomes included dollar cost, antibiotic use, survival, cost-effectiveness, antibiotic use per survivor, and the outcome perspective of financial cost antibiotic use per survivor. Initial coverage with three antibiotics was better than expectant management or one or two antibiotic approaches, leading to both improved survival (54% vs. 66%) and decreased cost ($55,447 vs. $41,483 per survivor). Testing with mini-bal did not improve survival but did decrease costs ($41,483 vs. $39,967) and antibiotic use (63 vs. 39 antibiotic days per survivor). From the perspective of minimizing cost, minimizing antibiotic use, and maximizing survival, the best strategy was three antibiotics with mini-bal. Keywords: ventilator-associated pneumonia; decision analysis; evidence-based medicine; bronchoscopy Ventilator-associated pneumonia (VAP) complicates the course of patients on mechanical ventilation in approximately 8 28% (1 4) of cases and is associated with a mortality rate of 24 50% (2, 5 11). It has been shown that appropriate initial antibiotic therapy is among the most important predictors of successful clinical outcome (12 19). Diagnostic testing strategies, although frequently leading to changes in therapy, have not been definitively shown to directly impact survival (17, 18, 20 22). This is because the information gained from such tests often comes too late to alter outcome. Thus, initial broad-spectrum antibiotic therapy in high-risk populations has been advocated. However, increasing antibiotic resistance patterns among intensive care unit pathogens, fostered by empiric-broad spectrum antibiotic regimens, represents a significant competing concern. Indeed, antibiotic use before the development of VAP is associated with an increased risk for potentially resistant gram-negative infections and Methicillin-resistant Staphylococcus aureus (MRSA) (4, 23 26). In addition, infection with these resistant pathogens is associated with poorer outcome. Balancing these two competing goals, namely the need for rapid treatment with antibiotics that cover a broad range of potential pathogens versus limiting unnecessary antibiotic use, remains one of the clinical challenges in dealing with VAP. The (Received in original form February 11, 2003; accepted in final form June 26, 2003) Correspondence and requests for reprints should be addressed to David E. Ost, M.D., Center for Pulmonary and Critical Care Medicine, North Shore University Hospital, 300 Community Drive, Manhasset, NY 11030. E-mail: dost@nshs.edu This article has an online supplement, which is accessible from this issue s table of contents online at www.atsjournals.org Am J Respir Crit Care Med Vol 168. pp 1060 1067, 2003 Originally Published in Press as DOI: 10.1164/rccm.200302-199OC on July 3, 2003 Internet address: www.atsjournals.org objective of this study was to define more precisely the trade-offs involved, as well as the impact various diagnostic and antibiotic selection strategies might have on these two competing goals. To do this, we performed a decision analysis of late-onset VAP. METHODS Overview and Definitions We constructed a hypothetical cohort of immunocompetent patients in the intensive care unit, intubated for 7 days, with evidence of lateonset VAP based on Centers of Disease Control criteria of fever, purulent secretions, leukocytosis, and radiographic infiltrates, with an estimated mortality rate of 20% for use in a decision analysis model. The initial decision was whether to do a diagnostic test immediately. The diagnostic strategies evaluated were as follows: none (empiric treatment only), quantitative nonprotected endotracheal cultures, bronchoscopy, and nonbronchoscopic mini-bronchoalveolar lavage (mini-bal). If a diagnostic test was done, it was assumed to take place as soon as the VAP syndrome was recognized so that there was no significant delay in initial antibiotic therapy (27). The second decision was how many initial antibiotics to give. From zero to three antibiotics could be chosen. Based on a literature review and American Thoracic Society guidelines, two antibiotics were used for the baseline analysis (13, 14, 22, 23, 27 30). Zero initial antibiotics represented expectant management, with subsequent treatment based on culture results that would arrive in 72 hours. One premise was that the specific antibiotics chosen would be based on American Thoracic Society guidelines, as adapted to local antibiograms, formularies, and intensive care unit pathways (31). Antibiotics would be given immediately after the patient met Centers of Disease Control criteria for VAP and would be continued until diagnostic test results returned. If pathogens were identified, antibiotics would be adjusted to cover any identified pathogens, and unnecessary antibiotics would be discontinued. If all cultures were negative and the patient had ongoing severe sepsis or was unstable, the physician would continue antibiotics. If the patient was stable, antibiotics would be discontinued (22, 32, 33). The model was designed to assess the cost-effectiveness of different diagnostic and treatment strategies for VAP. Two separate aspects of cost were considered: financial cost and amount of antibiotics used. Effectiveness was measured in terms of hospital survival. The Decision Analysis Model A decision analysis model that examined 16 strategies in the management of VAP was constructed with the use of DATA PRO (TreeAge Software, Williamstown, MA). Diagnostic strategies were combined systematically with each of the four treatment strategies so that all combinations were evaluated. The strategies were zero to three initial antibiotics with no diagnostic testing, zero to three initial antibiotics with endotracheal tube (ETT) aspirate quantitative cultures, zero to three initial antibiotics with bronchoscopic cultures, or zero to three initial antibiotics with nonbronchoscopic mini-bal quantitative cultures. The program determined the expected survival, antibiotic days received, and the cost of diagnostic tests and treatment. Antibiotic days received was defined as the number of antibiotics administered multiplied by the number of days given. Probability Variables: Methodology and Criteria A literature review was conducted to estimate model variables and the range of plausible values. Using a free form search in Medline, we used

Ost, Hall, Joseph, et al.: Decision Analysis of Ventilator-associated Pneumonia 1061 the term ventilator-associated pneumonia, requesting English-only articles pertaining to adults from 1986 to 2002. The relevance filter on the search engine was set at the halfway point between emphasizing relevance and quantity to ensure a proper number of articles. This search returned 555 citations. We further narrowed our search looking for only the following publication types: clinical trials, congresses, controlled clinical trials, guidelines, journal articles, meta analysis, multicenter studies, randomized controlled trials, and/or literature reviews. This narrowed the list to 304 citations. We then reviewed the title and abstracts of these 304 citations assessing if they pertained to our area of interest. We found 111 articles that met the criteria, and these were fully reviewed. Bibliographies and review articles were also used to identify additional sources. Selection criteria for studies used to determine decision variables included immunocompetent human adults subjects, diagnosis of VAP based on protected specimen culture (either bronchoscopic or nonbronchoscopic protected specimen) or histopathology, and peer-reviewed journals only (no abstracts). Probability variables that were included in the decision model are shown in Tables 1 and 2. A more detailed description of the probability variables and their definitions is available in the online supplement in the Methods. Cost and Charges Cost-effectiveness analysis was from the institutional perspective. This was chosen because many of the costs incurred by the different strategies would have to be paid for by the hospital without significant changes in Medicare reimbursement based on the current diagnostic-related groups system. To determine costs, data and input were used from administration, materials operation, finance, nursing, respiratory therapy, pharmacy, and microbiology. Costs were based on hospital charges taken from the hospital charge disk master for fiscal year 2002 and adjusted for departmental cost/charge ratios using a method similar to prior costeffectiveness studies in the area of pneumonia (34 36). Antibiotic use was considered another form of cost and was used in a separate costeffectiveness analysis. These variables are summarized in Table 2. Sensitivity Analysis A plausible range of high and low values for each variable was used to conduct sensitivity analyses that examined the influence of error in the values of variables on the results of the analysis. Sensitivity analysis was conducted for each of the variables in the model, including the effect on the outcomes of survival, cost of VAP evaluation and treatment, cost/ benefit, and antibiotic use. Sensitivity analysis ranges were based on published data in the literature as well as calculations based on analysis of extremes (Tables 1 and 2). For institutional costs, sensitivity analysis was conducted using a range of plus or minus 25% of the baselinecalculated cost, as well as published literature values when available (Table 2). Monte Carlo simulations were also performed so that all variables in the model were simultaneously varied over their plausible range, with values drawn from probabilistically weighted custom distributions. The weighting in these custom distributions was based on the sample size of the reference studies used to determine the plausible range. In all sensitivity analyses, a three-step process was used. First, oneway sensitivity analysis was done on all variables, and tornado diagrams were analyzed. Second, a two- and three-way sensitivity analysis was conducted, starting with the most sensitive variables. Combinations of less sensitive variables were also explored when those variables interacted, for example, efficacy of late adequate treatment and sensitivity of bronchoscopy. Finally, analysis of extremes was done using best case and worst case scenarios that favored each of the strategies. RESULTS The decision variables being studied were the number of antibiotics in the initial regimen and whether a diagnostic strategy should be used. Overall, to achieve the simultaneous goals of minimizing antibiotic use and financial cost while maximizing survival, a trade-off had to be made between antibiotic use on the one hand and cost minimization and survival on the other. A strategy of more initial antibiotics was better in terms of survival, financial cost, and financial cost per survivor. Antibiotic use, as measured by antibiotic days per survivor, increased with this strategy. Although diagnostic testing did not significantly impact survival, it helped by minimizing antibiotic days per survivor and financial costs. An analysis for each of the outcomes of (1) survival, (2) financial cost, (3) financial cost per survivor, (4) antibiotic use, (5) antibiotic use per survivor, and (6) the combined perspective of financial cost antibiotic use cost per survivor is presented later in this article. Additional data and sensitivity analysis are available in the online supplement. Survival Outcomes For the outcome of survival, the optimum strategy involved use of three antibiotics empirically with bronchoscopic testing (Table 3). The magnitude of the difference between strategies using different numbers of antibiotics was significant (the absolute difference between one and three antibiotics strategies was 6%). TABLE 1. SELECTED PROBABILITIES IN THE DECISION-ANALYSIS MODEL Variable Base Probability Range Reference Value Range References Baseline mortality 17.5% 5 60% 8 59% 7, 9, 21, 44 50 Probability initial antibiotics cover pathogen 60% 30 100% 31 77% 13, 14, 16, 23, 42 Probability that 12 14, 16, 22, 23, 27, 30, 51 One antibiotic covers pathogen 40% 30 50% Two antibiotics cover pathogen 60% 35 80% Three antibiotics cover pathogen 80% 60 90% Probability of positive bronchoscopic cultures 85% 30 100% 33 100% (BAL or PSB) 32, 52 55 Probability of positive nonbronchoscopic cultures 80% 30 100% 63 100% (mini-bal or BBS) 32, 52, 56, 57 Probability of positive tracheal secretion cultures 70% 50 100% 38 100% 32, 52, 56 Attributable VAP mortality if antibiotics correct 15% 0 40% 3.6 33.0% 12 14, 16, 51 Efficacy of adequate initial antibiotic therapy 0.55 0.344 0.668 0.344 0.668 12 14, 16, 51 Efficacy of adequate late antibiotic therapy 0.05 0.01 0.10 0 16, 51 Increase in hospital length of stay due to VAP Average of 12.82 days, with correct antibiotics 11.8 days 5 23 range of 5 23 days 9, 19, 21, 44, 46, 48, 50 Increase in ICU length of stay due to VAP Average of 10.9 days, with correct antibiotics 9.9 days 3.3 18.0 range of 4 18 days 9, 19, 21, 44, 46, 48, 50 Increase in length of stay with inadequate antibiotics as compared with adequate antibiotics 3 days 0 5 Range of 2.8 3.0 days 12, 19, 51 Definition of abbreviations: BAL bronchoalveolar lavage; BBS blind bronchial brush; ICU intensive care unit; PSB protected specimen brush; VAP ventilatorassociated pneumonia. The base estimate is considered the most probable value for each variable, while the plausible low and high estimates are shown in the range column.

1062 AMERICAN JOURNAL OF RESPIRATORY AND CRITICAL CARE MEDICINE VOL 168 2003 TABLE 2. COST AND ANTIBIOTIC USE VARIABLES Variable Base Cost Range References and Ranges Bronchoscopy with protected specimen Cx $215 $160 270 Based on institutional cost analysis 25%* Mini-BAL $44 $30 55 Based on institutional cost analysis 25%* ETT catheter Gram stain and Cx $11 $8 14 Based on institutional cost analysis 25%* ICU day $2,009 $750 2,500 Based on institutional cost analysis 25%* (34 36, 58) Cost per Abx/day $62 $25 110 Based on institutional costs* (34 36, 58) Cost of culturing one to three organisms $17, $29, $41 $13 51 Based on institutional costs* Length of empiric therapy 14 10 21 (31, 32, 36, 42, 52, 54) Number of antibiotics in initial regimen 2 0 3 Historical control subjects (31, 32, 52) Probability of physician stopping antibiotics if cultures are negative 0.5 0 1 (22, 28, 41, 51) Definition of abbreviations: Abx antibiotic; Cx culture; ETT endotracheal tube; ICU intensive care unit; mini-bal blind mini-bronchoalveolar lavage. * The base estimate is considered the most probable value for each variable, whereas the plausible low and high estimates are shown in the range column. Strategy options to maximize survival were also sensitive to changes in the sensitivity of ETT aspirates, bronchoscopy, and mini-bal (Table 4). However, although optimal strategy was sensitive to the diagnostic tests sensitivity, the magnitude of the difference between diagnostic strategies was small (maximum absolute difference of less than 1%) when comparing strategies that used the same initial number of antibiotics. Financial Cost Outcomes For the outcome of minimizing financial cost, the best strategy was mini-bal, followed by ETT aspirate, bronchoscopy, and finally empiric therapy with no diagnostic testing. The optimal strategy was sensitive to the number of initial antibiotics given, the average cost per antibiotic day, the length of antibiotic treatment, the sensitivity of diagnostic testing, and the cost of mini- BAL (Table 5). When no (expectant management) or one initial antibiotic was used, the best strategy in terms of financial cost minimization was empiric treatment without diagnostic testing. For two or more antibiotics, the best strategy was a diagnostic test with mini-bal or ETT aspirate. The magnitude of the difference between mini-bal and ETT aspirate was relatively small ($12 41), with a slight advantage going to either strategy depending on willingness to withhold antibiotics, the number of initial antibiotics, and the sensitivity of the tests (Figures 1A, 2, and 3). Financial Cost per Survivor For the outcome of financial cost-effectiveness, the best strategy in the baseline scenario was mini-bal or ETT aspirate because effectiveness was essentially the same in all four strategies and these had the lowest cost. When the number of antibiotics in the initial regimen was considered as a factor, the most cost- TABLE 3. COST-EFFECTIVENESS OF VENTILATOR-ASSOCIATED PNEUMONIA STRATEGIES BASED ON NUMBER OF ANTIBIOTICS IN INITIAL REGIMEN No. of Initial Antibiotics Strategy* ($ ) Survival C/E ICER 0 Empiric 29,733 0.536 55,447 NA Empiric ETT asp 30,327 0.544 55,707 72847 Empiric mini-bal 30,430 0.545 55,792 101479 Empiric FOB 30,633 0.546 56,117 433261 1 Empiric 28,190 0.600 47,002 NA Empiric ETT asp 28,292 0.605 46,789 20734 Empiric mini-bal 28,344 0.605 46,828 86184 Empiric FOB 28,523 0.606 47,102 634288 2 Empiric ETT asp 27,462 0.635 43,260 NA Empiric mini-bal 27,464 0.635 43,236 4854 Empiric FOB 27,618 0.635 43,465 819710 Empiric 27,853 0.632 44,103 (Dominated) 3 Empiric mini-bal 26,583 0.665 39,967 NA Empiric ETT asp 26,632 0.665 40,052 (Dominated) Empiric FOB 26,712 0.666 40,155 1375978 Empiric 27,516 0.663 41,483 (Dominated) Definition of abbreviations: C/E cost-effectiveness; Empiric ETT asp empiric antibiotics plus quantitative culture of unprotected ETT aspirate; Empiric FOB empiric antibiotics plus bronchoscopy; Empiric mini-bal empiric antibiotics plus quantitative cultures of protected specimen blind mini-bronchoalveolar lavage; ICER incremental cost-effectiveness ratio; NA not applicable strategy is cheapest in category for the number of antibiotics specified. * Strategies are sorted by number of antibiotics in the initial regimen and then by increasing cost. The incremental cost-effectiveness ratio is a comparison with less costly strategies that use the same number of initial antibiotics. The comparison is made with the strategy directly above on the table that uses the same number of antibiotics. The top strategy is less costly but may not be as effective. If the cheaper (top) strategy is less effective, then the ICER represents the cost per additional life saved by using the more expensive but more effective strategy. If the top strategy is equal or greater in efficacy than the bottom strategy, then the bottom strategy is considered dominated by the top strategy (because the top is both cheaper and at least as effective). Cost

Ost, Hall, Joseph, et al.: Decision Analysis of Ventilator-associated Pneumonia 1063 TABLE 4. SENSITIVITY ANALYSIS FOR OUTCOME OF SURVIVAL Baseline Plausible Survival Threshold Variable Value Range Range Value Sensitive Background mortality 17.5% 5 60% 30.8 73.2% NT No Number of antibiotics in initial regimen 2 0 3 54.6 66.5% NT NA Baseline VAP mortality not treated 35% 23.5 42% 59.7 69.8% NT No Probability of adequate initial antibiotic coverage 60% 30 90% 59.1 68.0% NT No Efficacy of adequate antibiotic coverage 0.55 0.344 0.668 60.0 65.6% NT No Efficacy of late adequate antibiotics 0.05 0.01 0.10 63.3 63.9% NT No Sensitivity bronch 85% 30 100% 63.5 63.6% 81% Yes Sensitivity of routine ETT asp 70% 50 100% 63.5 63.6% 84% Yes Sensitivity of mini-bal 80% 30 100% 63.5 63.6% 84% Yes Definition of abbreviations: bronch bronchoscopy; ETT asp endotracheal tube aspirate; mini-bal blind mini-bronchoalveolar lavage; NA not applicable; NT no threshold, not sensitive; VAP ventilator-associated pneumonia. Cost includes equipment acquisition, personnel time, and processing costs. effective strategy was a three-antibiotic strategy with mini-bal (Table 3 and Figure 2). The absolute difference between diagnostic testing strategies (mini-bal, ETT aspirate, or bronchoscopy) was relatively small, but the difference between these and the empiric strategy without testing was significant. Subsequent sensitivity analysis was done using three antibiotics combined with mini-bal as the comparator. The optimum cost-effective strategy of three antibiotics with mini-bal was sensitive to the variables of bronchoscopy sensitivity (threshold value for bronchoscopy sensitivity, 0.99), sensitivity of ETT aspirate (threshold value for ETT aspirate sensitivity, 0.77), and mini-bal sensitivity (threshold value, 0.73). Antibiotic Use Costs For the outcome of minimizing antibiotic use as measured by antibiotic days, the optimum strategy was sensitive to the number of antibiotics used in the initial regimen and the sensitivity of bronchoscopy, mini-bal, and ETT aspiration. If zero or one antibiotic was used in the initial regimen, an empiric treatment strategy without testing was optimal. If two or more antibiotics were used, the optimum strategy shifted to strategies employing diagnostic testing. Whichever diagnostic technique had the highest sensitivity was the best strategy when two or more antibiotics were used. However, over the range of expected values, the magnitude of this difference was small (maximum difference of 1.2 antibiotic days per patient). In contrast, the magnitude of the difference between empiric strategies without diagnostic testing versus strategies with diagnostic testing was significant (range of 3.5 12 antibiotic days per patient) (Figure 1B). Antibiotic Use per Survivor When considering antibiotic use and survival simultaneously, we used cost-effectiveness techniques so that we could measure the number of antibiotic days per survivor. In this approach, antibiotic use was viewed as a cost (in terms of promoting antibiotic resistance) and survival as the benefit. Using a bronchoscopy strategy was most cost-effective in the baseline case when two antibiotics were used in the initial regimen (Figure 2). Antibiotic use per survivor was sensitive to the number of antibiotics in the initial regimen and the sensitivity of bronchoscopy, mini- BAL, and ETT aspiration. When comparing all strategies, the strategy of three antibiotics combined with diagnostic testing was best, provided that there was a willingness to use up to 28 antibiotic days per patient (Figure 3). Financial Cost Antibiotic Use Survival Benefit Finally, to clarify the simultaneous tradeoffs involved in terms of financial cost, antibiotic resistance issues, and survival, we constructed a three-dimensional graphic analysis of cost, antibiotic use, and survival. This is shown in Figure 2. We used methods of analysis similar to traditional two-variable cost-effectiveness for this approach, with cost and antibiotic use versus effectiveness being the primary considerations. For this analysis, cost-effectiveness was viewed in two perspectives. In the first perspective, cost was viewed TABLE 5. SENSITIVITY ANALYSIS FOR OUTCOME OF COST Variable Baseline Value Plausible Range Range of Cost/Patient Threshold Value Sensitive Increase LOS if VAP treated with correct antibiotics 11.8 5 22 $13,801 49,963 NT No Cost per ICU day $2,009 $750 2,500 $11,095 33,845 NT No Additional LOS if antibiotics inadequate initially 3 0 5 $25,051 29,069 NT No Number of antibiotics in initial regimen 2 0 3 $26,583 30,514 1.2 and 2.0 Yes Average cost of antibiotics per day $62 $25 110 $26,679 28,455 65.90 Yes Empiric length of treatment 14 10 21 $27,120 28,042 14.7 Yes Sensitivity of ETT asp 70% 50 100% $27,378 27,464 NT No Sensitivity of mini-bal 80 30 100% $27,408 27,462 0.81 Yes Cost mini-bal $44 $30 55 $27,450 27,462 $42 Yes Definition of abbreviations: ETT asp endotracheal tube aspirate; ICU intensive care unit; LOS length of stay; mini-bal blind mini-bronchoalveolar lavage; NT no threshold, not sensitive; VAP ventilator-associated pneumonia.

1064 AMERICAN JOURNAL OF RESPIRATORY AND CRITICAL CARE MEDICINE VOL 168 2003 Figure 3. Bubble plot of survival, diagnostic strategy, initial number of antibiotics, and total antibiotic use. The size of the dots is proportional to the initial number of antibiotics given at presentation, ranging from zero to three. All strategies employ initial antibiotics plus a diagnostic strategy. Abx antibiotics; 0 initial Abx no initial antibiotics with subsequent treatment based on cultures. Diagnostic strategies are empiric antibiotics without diagnostic testing (filled blue circles), endotracheal aspirates (green rings), bronchoscopy (violet rings), and nonbronchoscopic mini-bal (red rings). Figure 1. Sensitivity analyses on number of antibiotics in initial regimen: Impact on (A) cost and (B) antibiotic use. In both cases, the impact varies with the diagnostic strategy used. Diagnostic strategies evaluated are empiric antibiotics without diagnostic testing (solid blue line), endotracheal aspirates (dashed-dotted green line), bronchoscopy (dotted violet line), and nonbronchoscopic mini-bronchoalveolar lavage (mini-bal; dashed-dotted red line). as financial cost. In the second, cost was in terms of antibiotic use as a surrogate for promotion of antibiotic resistance. In both perspectives, effectiveness was expressed in terms of survival. For the perspective of financial cost versus effectiveness, optimal strategies will be those that are down and to the right (cheaper and more effective). For the second perspective of limiting anti- biotic resistance while maintaining effectiveness, optimal strategies will be those with smaller dots (less antibiotic use) and to the right (more effective). Using this combined perspective, the optimal strategy for VAP depends on the willingness to pay, expressed in willingness to use antibiotics as well as dollars. Although traditional financial cost-effectiveness suggests that a three-antibiotic strategy combined with mini-bal is most cost-effective, this graphic clarifies the tradeoffs made to achieve this in that an average of 26 antibiotic days per patient will be required to do this, as compared with the strategy of doing nothing. The incremental cost-effectiveness with respect to antibiotic use for the three-antibiotic with mini-bal strategy is 200.8 antibiotic days per additional survivor compared with the no-antibiotic strategy. From the perspective of minimizing antibiotic use, this is less than the incremental cost-effectiveness for all of the other strategies except for bronchoscopy with three antibiotics. Thus, the three-antibiotic mini- BAL strategy meets the definition for extended dominance and Figure 2. Bubble plot of financial cost antibiotic use survival as a function of ventilator-associated pneumonia strategy. The size of the dots is proportional to antibiotic use, which appears as a label next to each dot. All strategies employ initial antibiotics plus a diagnostic strategy. Abx antibiotics; 0 initial Abx no initial antibiotics with subsequent treatment based on cultures. Diagnostic strategies are empiric antibiotics without diagnostic testing (filled blue circles), endotracheal aspirates (green rings), bronchoscopy (violet rings), and nonbronchoscopic mini-bal (red rings).

Ost, Hall, Joseph, et al.: Decision Analysis of Ventilator-associated Pneumonia 1065 is superior to all one- and two-antibiotic strategies, irrespective of diagnostic technique, in terms of both minimizing antibiotic use and financial cost per additional survivor. This requires a willingness to pay of approximately 201 antibiotic days per additional survivor and $40,000 per survivor. DISCUSSION Our analysis indicates that the best strategy overall in late-onset VAP is broad-spectrum three-antibiotic initial coverage combined with diagnostic testing. As prior studies have suggested, diagnostic testing alone has relatively little impact on the individual patient s survival (17, 18, 20, 28) but is cost-effective in that it limits cost by decreasing unnecessary antibiotic use. Although some experts have advocated this previously, there is still considerable controversy in this area, and diagnostic testing for VAP is infrequently used. Second, testing alone is not enough. It must be combined with broad-spectrum coverage initially to optimize cost-effectiveness. The model structures the problem so that critical variables are more readily identified, allowing future research to target those areas that are most likely to have the greatest impact on patient outcomes. In this model, we can see that of the variables amenable to physician intervention, it is improving antibiotic selection that will have the greatest impact. This can be approached from a variety of perspectives. One perspective would be to enforce rigid guidelines that follow American Thoracic Society criteria. However, this might result in a wide spectrum of possible outcomes, given regional variation in incidence and resistance (37). Therefore, following published guidelines is a questionable outcome measure. Another approach would be to evaluate and develop more rapid and sensitive diagnostic tests. This decision model is useful in that it provides a framework for evaluating the potential utility of such diagnostic tests and can help predict whether a new diagnostic test would be costeffective. The model also makes transparent the tradeoffs being made in terms of antibiotic use, cost, and survival so that decision makers can develop a rational policy. Although broad-spectrum initial coverage combined with mini-bal is the most financially cost-effective, there is a trade-off in terms of antibiotic use. When viewed from the perspective of limiting the development of resistance, this financially cost-effective outcome costs over 200 antibiotic days per additional survivor. This emphasizes the issue of sustainable probability of coverage. If antibiotics are used very broadly, the short-term survival will be good, but over the long term, resistance may develop more rapidly. Excessive restriction of antibiotic use will not be cost-effective and will lead to excess mortality in the present rather than in the future. The model makes clear that the goal is the maximally efficient use of antibiotics over time. We can use the model as a framework to assess how additional antibiotics affect outcome, essentially asking what is the return on our investment of additional antibiotics. This is the incremental costeffectiveness concept, but with antibiotic use rather than dollars as the unit of cost. Our analysis suggests that increasing the number of initial antibiotics from one to three is efficient. A similar theoretical approach could be applied to other clinical problems but would probably result in a much different assessment of incremental cost-effectiveness (e.g., community-acquired pneumonia). This study supplements the information and analysis of other investigators who have used decision theory methods to study VAP (38, 39). A prior decision analysis favored withholding antibiotics for patients with clinically diagnosed VAP (38). However, their approach did not take into account the probability of adequate antibiotic coverage and the evidence that suggests that inadequate initial antibiotic coverage increases mortality and cost. Thus, prior models could not adequately portray the strategy of broad initial coverage with rapid tapering based on diagnostic testing and could not provide insight into one of the most pivotal questions, namely how many antibiotics to use initially and how probability of adequate coverage affects outcome. Prior models did include consideration of adverse drug reactions and the probability of future episodes of drug resistant VAP. However, when we integrated data on adverse drug reactions, the impact was always negligible in terms of cost and survival for all scenarios. It was therefore eliminated from the tree at the outset of the study. This is consistent with the findings from prior investigators (38). Although future episodes of potentially drug-resistant VAP were integrated into prior models, our review of the literature did not reveal any strong evidence that would allow accurately portraying the impact of treatment for VAP today on future incidence and outcome of VAP, especially considering how often antibiotics are used in the intensive care unit. In comparison to the work of prior investigators, our model also allows measurement of other outcomes, such as antibiotic use and cost, to be considered along with survival. Ideally, a single multiattribute utility function integrating these outcomes could be developed. One limitation of our study was that we could not define this single multiattribute utility function. This was not feasible because we could not convert antibiotic use today to the future probability of antibiotic resistance globally or put a dollar cost on the impact of antibiotic use on future resistance patterns given the available data. Thus, although a multiattribute utility function combining the three outcome variables of antibiotic use, cost, and survival (based on the premise of preferential independence and utility independence) is in theory possible, the paucity of published data makes this impractical. Our study has several limitations. Some investigators have advocated that antibiotics be stopped when protected specimen culture results are negative and patients do not demonstrate evidence of severe sepsis, but physicians are often reluctant to do so (22, 28, 29, 32, 33, 40, 41). If antibiotics can be stopped earlier, when quantitative cultures are negative and the clinical context is favorable, then the benefit of diagnostic testing strategies will be greater. Sensitivity analysis of antibiotic stopping rules was therefore used to model the wide spectrum of clinical practice. Even if a full course of therapy was always given in the setting of negative culture results (conservative stopping rule), the benefits of diagnostic testing outweighed the costs, provided that two or more antibiotics were used initially (additional sensitivity analysis can be found in the online supplement). Our study also did not address the potential value of direct examination of bronchoalveolar lavage for neutrophils and intracellular organisms (22). We chose not to do this initially because it is not that widely available, and therefore, the point estimates of sensitivity as compared with other diagnostic tests are not well described. In particular, we did not know how often there was discordance between smear and culture (i.e., gram-positives grow on culture but not in smear or vice versa), how often there was partial concordance (i.e., gram-positive in both but only culture grows a gram-negative), what reference standard would take precedence (presumably culture), and how much this information would change the probability of adequate coverage. In addition, we could not estimate the actual cost of doing the procedure from our time motion studies because of our lack of institutional expertise in this area. Certainly knowing that a gram-positive organism is present would be highly useful if vancomycin was not routinely given. The utility of knowing that a

1066 AMERICAN JOURNAL OF RESPIRATORY AND CRITICAL CARE MEDICINE VOL 168 2003 gram-negative organism was present might be less. The time delay between testing and subsequent antibiotic adjustments might be improved significantly, and there might be significant antibiotic savings. Also, if antibiotics could be withheld very quickly (after one dose) when results were negative, the impact would be even larger. Another limitation of this model is the uncertainty in the relationship between the number of antibiotics used and probability of effective coverage. Estimates of the efficacy of various antibiotic regimens (13, 14, 16, 23, 42) depend heavily on accurate knowledge of underlying incidence and resistance patterns as well as the formulary of available antibiotics. However, although this relationship has been described based on empiric data, there are no mathematical models in the literature relating number of antibiotics used to probability of adequate coverage. Given the variation in etiology across treatment sites, this could vary widely based on local patterns (37). Sensitivity analysis can give insight into how this information impacts on outcome. A sensitivity analysis on the probability of adequate coverage for one, two, and three antibiotic regimens demonstrates that if the third antibiotic provides an absolute incremental benefit of approximately 5% above the two-antibiotic strategy (e.g., 80% vs. 75% coverage for three versus two antibiotic regimens), then three antibiotics will remain the most cost-effective alternative provided that diagnostic testing is used (see online supplement for a more detailed sensitivity analysis). Despite these limitations, important conclusions can still be drawn regarding optimal strategies for late-onset VAP. The majority of alternative strategies can be eliminated based on a careful analysis of our willingness to use antibiotics and basic cost-effectiveness principles of extended dominance (Figure 2), as most of the inferior strategies cost more and are less effective while using more antibiotics per additional survivor compared with the mini-bal with three-antibiotic strategy. This reduces the complex strategy question to the more manageable question of broad antibiotic coverage with either mini-bal or bronchoscopy. Antibiotics before diagnostic testing, especially recent antibiotics, can significantly decrease sensitivity (43). If sensitivity falls below the 70% range, diagnostic testing becomes less costeffective. Therefore, the diagnostic strategy chosen must be feasible for rapid execution around the clock because several studies have demonstrated that delays in adequate antibiotic therapy lead to adverse outcomes. Given that bronchoscopy is not readily available around the clock, mini-bal is probably the most feasible strategy and will be cost-effective. Conflict of Interest Statement : D.E.O. has no declared conflict of interest; C.S.H. has no declared conflict of interest; G.J. has no declared conflict of interest; C.G. received a research grant from Merck for $12,000 in 2002 2003 and is on Merck s Speakers Bureau, receiving $4,000 in 2003, and received $10,000 from Roche Pharmaceuticals for serving as an advisor and speaker; S.C. has no declared conflict of interest; E.K. has no declared conflict of interest; M.L. has no declared conflict of interest; R.I. has no declared conflict of interest; A.M.F. has no declared conflict of interest. Acknowledgment : The authors thank Stephan Kamholz for his editorial review and Elizabeth Murray for her assistance with the preparation of this article. References 1. Haley RW, Culver DH, White JW, Morgan WM, Emori TG. The nationwide nosocomial infection rate: a new need for vital statistics. Am J Epidemiol 1985;121:159 167. 2. Pennington JE. Nosocomial respiratory infections. In: Mandell GL, Douglas RG, Bennet JE, editors. Principles and practice of infectious diseases. St. Louis: Churchill Livingstone; 1990. p. 2199 2205. 3. Monitoring hospital-acquired infections to promote patient safety United States, 1990 1999. MMWR Morb Mortal Wkly Rep 2000;49: 149 153. 4. Celis R, Torres A, Gatell JM, Almela M, Rodriguez-Roisin R, Agusti- Vidal A. Nosocomial pneumonia: a multivariate analysis of risk and prognosis. Chest 1988;93:318 324. 5. National Nosocomial Infections Surveillance (NNIS) System report, data summary from January 1990 May 1999, issued June 1999. Am J Infect Control 1999;27:520 532. 6. Fagon JY, Chastre J, Domart Y, Trouillet JL, Pierre J, Darne C, Gibert C. Nosocomial pneumonia in patients receiving continuous mechanical ventilation: prospective analysis of 52 episodes with use of a protected specimen brush and quantitative culture techniques. Am Rev Respir Dis 1989;139:877 884. 7. Torres A, Aznar R, Gatell JM, Jimenez P, Gonzalez J, Ferrer A, Celis R, Rodriguez-Roisin R. Incidence, risk, and prognosis factors of nosocomial pneumonia in mechanically ventilated patients. Am Rev Respir Dis 1990;142:523 528. 8. Craven DE, Kunches LM, Kilinsky V, Lichtenberg DA, Make BJ, McCabe WR. Risk factors for pneumonia and fatality in patients receiving continuous mechanical ventilation. Am Rev Respir Dis 1986; 133:792 796. 9. Markowicz P, Wolff M, Djedaini K, Cohen Y, Chastre J, Delclaux C, Merrer J, Herman B, Veber B, Fontaine A, et al. Multicenter prospective study of ventilator-associated pneumonia during acute respiratory distress syndrome: incidence, prognosis, and risk factors: ARDS Study Group. Am J Respir Crit Care Med 2000;161:1942 1948. 10. Rello J, Ausina V, Ricart M, Castella J, Prats G. Impact of previous antimicrobial therapy on the etiology and outcome of ventilator-associated pneumonia. Chest 1993;104:1230 1235. 11. Rello J, Rue M, Jubert P, Muses G, Sonora R, Valles J, Niederman MS. Survival in patients with nosocomial pneumonia: impact of the severity of illness and the etiologic agent. Crit Care Med 1997;25:1862 1867. 12. Kollef MH, Sherman G, Ward S, Fraser VJ. Inadequate antimicrobial treatment of infections: a risk factor for hospital mortality among critically ill patients. Chest 1999;115:462 474. 13. Luna CM, Vujacich P, Niederman MS, Vay C, Gherardi C, Matera J, Jolly EC. Impact of BAL data on the therapy and outcome of ventilatorassociated pneumonia. Chest 1997;111:676 685. 14. Alvarez-Lerma F. Modification of empiric antibiotic treatment in patients with pneumonia acquired in the intensive care unit: ICU-Acquired Pneumonia Study Group. Intensive Care Med 1996;22:387 394. 15. Kollef MH, Bock KR, Richards RD, Hearns ML. The safety and diagnostic accuracy of minibronchoalveolar lavage in patients with suspected ventilator-associated pneumonia. Ann Intern Med 1995;122:743 748. 16. Rello J, Gallego M, Mariscal D, Sonora R, Valles J. The value of routine microbial investigation in ventilator-associated pneumonia. Am J Respir Crit Care Med 1997;156:196 200. 17. Sanchez-Nieto JM, Torres A, Garcia-Cordoba F, El-Ebiary M, Carrillo A, Ruiz J, Nunez ML, Niederman M. Impact of invasive and noninvasive quantitative culture sampling on outcome of ventilator-associated pneumonia: a pilot study. Am J Respir Crit Care Med 1998;157:371 376. 18. Ruiz M, Torres A, Ewig S, Marcos MA, Alcon A, Lledo R, Asenjo MA, Maldonaldo A. Noninvasive versus invasive microbial investigation in ventilator-associated pneumonia: evaluation of outcome. Am J Respir Crit Care Med 2000;162:119 125. 19. Dupont H, Mentec H, Sollet JP, Bleichner G. Impact of appropriateness of initial antibiotic therapy on the outcome of ventilator-associated pneumonia. Intensive Care Med 2001;27:355 362. 20. Sole Violan J, Fernandez JA, Benitez AB, Cardenosa Cendrero JA, Rodriguez de Castro F. Impact of quantitative invasive diagnostic techniques in the management and outcome of mechanically ventilated patients with suspected pneumonia. Crit Care Med 2000;28:2737 2741. 21. Heyland DK, Cook DJ, Griffith L, Keenan SP, Brun-Buisson C. The attributable morbidity and mortality of ventilator-associated pneumonia in the critically ill patient: the Canadian Critical Trials Group. Am J Respir Crit Care Med 1999;159:1249 1256. 22. Fagon JY, Chastre J, Wolff M, Gervais C, Parer-Aubas S, Stephan F, Similowski T, Mercat A, Diehl JL, Sollet JP, et al. Invasive and noninvasive strategies for management of suspected ventilator-associated pneumonia: a randomized trial. Ann Intern Med 2000;132:621 630. 23. Trouillet JL, Chastre J, Vuagnat A, Joly-Guillou ML, Combaux D, Dombret MC, Gibert C. Ventilator-associated pneumonia caused by potentially drug-resistant bacteria. Am J Respir Crit Care Med 1998; 157:531 539. 24. Hanberger H, Garcia-Rodriguez JA, Gobernado M, Goossens H, Nilsson LE, Struelens MJ. Antibiotic susceptibility among aerobic gramnegative bacilli in intensive care units in 5 European countries: French and Portuguese ICU Study Groups. JAMA 1999;281:67 71.

Ost, Hall, Joseph, et al.: Decision Analysis of Ventilator-associated Pneumonia 1067 25. Itokazu GS, Quinn JP, Bell-Dixon C, Kahan FM, Weinstein RA. Antimicrobial resistance rates among aerobic gram-negative bacilli recovered from patients in intensive care units: evaluation of a national postmarketing surveillance program. Clin Infect Dis 1996;23:779 784. 26. Bergogne-Berezin E. Treatment and prevention of nosocomial pneumonia. Chest 1995;108:26S 34S. 27. Iregui M, Ward S, Sherman G, Fraser VJ, Kollef MH. Clinical importance of delays in the initiation of appropriate antibiotic treatment for ventilator-associated pneumonia. Chest 2002;122:262 268. 28. Heyland DK, Cook DJ, Marshall J, Heule M, Guslits B, Lang J, Jaeschke R. The clinical utility of invasive diagnostic techniques in the setting of ventilator-associated pneumonia: Canadian Critical Care Trials Group. Chest 1999;115:1076 1084. 29. Croce MA, Fabian TC, Shaw B, Stewart RM, Pritchard FE, Minard G, Kudsk KA, Baselski VS. Analysis of charges associated with diagnosis of nosocomial pneumonia: can routine bronchoscopy be justified? J Trauma 1994;37:721 727. 30. Fowler RA, Flavin KE, Barr J, Weinacker AB, Parsonnet J, Gould MK. Variability in antibiotic prescribing patterns and outcomes in patients with clinically suspected ventilator-associated pneumonia. Chest 2003; 123:835 844. 31. Hospital-acquired pneumonia in adults: diagnosis, assessment of severity, initial antimicrobial therapy, and preventive strategies: a consensus statement, American Thoracic Society, November 1995. Am J Respir Crit Care Med 1996;153:1711 1725. 32. Chastre J, Fagon JY. Ventilator-associated pneumonia. Am J Respir Crit Care Med 2002;165:867 903. 33. Singh N, Rogers P, Atwood CW, Wagener MM, Yu VL. Short-course empiric antibiotic therapy for patients with pulmonary infiltrates in the intensive care unit: a proposed solution for indiscriminate antibiotic prescription. Am J Respir Crit Care Med 2000;162:505 511. 34. Paladino JA, Fell RE. Pharmacoeconomic analysis of cefmenoxime dual individualization in the treatment of nosocomial pneumonia. Ann Pharmacother 1994;28:384 389. 35. Paladino JA. Cost-effectiveness comparison of cefepime and ceftazidime using decision analysis. Pharmacoeconomics 1994;5:505 512. 36. McNabb JJ, Nightingale CH, Quintiliani R, Nicolau DP. Cost-effectiveness of ceftazidime by continuous infusion versus intermittent infusion for nosocomial pneumonia. Pharmacotherapy 2001;21:549 555. 37. Rello J, Sa-Borges M, Correa H, Leal SR, Baraibar J. Variations in etiology of ventilator-associated pneumonia across four treatment sites: implications for antimicrobial prescribing practices. Am J Respir Crit Care Med 1999;160:608 613. 38. Sterling TR, Ho EJ, Brehm WT, Kirkpatrick MB. Diagnosis and treatment of ventilator-associated pneumonia impact on survival: a decision analysis. Chest 1996;110:1025 1034. 39. Baker AM, Bowton DL, Haponik EF. Decision making in nosocomial pneumonia: an analytic approach to the interpretation of quantitative bronchoscopic cultures. Chest 1995;107:85 95. 40. Torres A, el-ebiary M. Invasive diagnostic techniques for pneumonia: protected specimen brush, bronchoalveolar lavage, and lung biopsy methods. Infect Dis Clin North Am 1998;12:701 722. 41. Bonten MJ, Bergmans DC, Stobberingh EE, van der Geest S, De Leeuw PW, van Tiel FH, Gaillard CA. Implementation of bronchoscopic techniques in the diagnosis of ventilator-associated pneumonia to reduce antibiotic use. Am J Respir Crit Care Med 1997;156:1820 1824. 42. Ibrahim EH, Ward S, Sherman G, Schaiff R, Fraser VJ, Kollef MH. Experience with a clinical guideline for the treatment of ventilatorassociated pneumonia. Crit Care Med 2001;29:1109 1115. 43. Michaud S, Suzuki S, Harbarth S. Effect of design-related bias in studies of diagnostic tests for ventilator-associated pneumonia. Am J Respir Crit Care Med 2002;166:1320 1325. 44. Fagon JY, Chastre J, Hance AJ, Montravers P, Novara A, Gibert C. Nosocomial pneumonia in ventilated patients: a cohort study evaluating attributable mortality and hospital stay. Am J Med 1993;94:281 288. 45. Leroy O, Guilley J, Georges H, Choisy P, Guery B, Alfandari S, Beaucaire G. Effect of hospital-acquired ventilator-associated pneumonia on mortality of severe community-acquired pneumonia. J Crit Care 1999;14:12 19. 46. Rello J, Quintana E, Ausina V, Castella J, Luquin M, Net A, Prats G. Incidence, etiology, and outcome of nosocomial pneumonia in mechanically ventilated patients. Chest 1991;100:439 444. 47. Kollef MH, Silver P, Murphy DM, Trovillion E. The effect of late-onset ventilator-associated pneumonia in determining patient mortality. Chest 1995;108:1655 1662. 48. Baker AM, Meredith JW, Haponik EF. Pneumonia in intubated trauma patients: microbiology and outcomes. Am J Respir Crit Care Med 1996; 153:343 349. 49. Kollef MH. Ventilator-associated pneumonia: a multivariate analysis. JAMA 1993;270:1965 1970. 50. Ibrahim EH, Tracy L, Hill C, Fraser VJ, Kollef MH. The occurrence of ventilator-associated pneumonia in a community hospital: risk factors and clinical outcomes. Chest 2001;120:555 561. 51. Kollef MH, Ward S. The influence of mini-bal cultures on patient outcomes: implications for the antibiotic management of ventilatorassociated pneumonia. Chest 1998;113:412 420. 52. Grossman RF, Fein A. Evidence-based assessment of diagnostic tests for ventilator-associated pneumonia: executive summary. Chest 2000;117: 177S 181S. 53. Torres A, El-Ebiary M. Bronchoscopic BAL in the diagnosis of ventilator-associated pneumonia. Chest 2000;117:198S 202S. 54. Baughman RP. Protected-specimen brush technique in the diagnosis of ventilator-associated pneumonia. Chest 2000;117:203S 206S. 55. Niederman MS. Bronchoscopy in nonresolving nosocomial pneumonia. Chest 2000;117:212S 218S. 56. Cook D, Mandell L. Endotracheal aspiration in the diagnosis of ventilatorassociated pneumonia. Chest 2000;117:195S 197S. 57. Campbell GD Jr. Blinded invasive diagnostic procedures in ventilatorassociated pneumonia. Chest 2000;117:207S 211S. 58. Detsky AS, Naglie G, Krahn MD, Redelmeier DA, Naimark D. Primer on medical decision analysis: II: building a tree. Med Decis Making 1997;17:126 135.