Method Preferences and Test Accuracy of Antimicrobial Susceptibility Testing

Method Preferences and Test Accuracy of Antimicrobial Susceptibility Testing Updates From the College of American Pathologists Microbiology Surveys Program (2000) Ronald N. Jones, MD; for the College of American Pathologists Microbiology Resource Committee Objective. To summarize the antimicrobial susceptibility testing results from the College of American Pathologists (CAP) Microbiology Surveys Program for 2000. Specifically, the frequency of tests used and the quantitative and qualitative (susceptibility category) accuracy were assessed. Design. The CAP Microbiology Surveys challenged subscribers in 2000 with 3 well-characterized organisms for antimicrobial susceptibility testing in pure culture. Each laboratory was to use the test method and reporting procedures routinely applied to patient samples. The strains were National Committee for Clinical Laboratory Standards (NCCLS) quality control organisms with precisely defined antimicrobial susceptibility patterns and reproducibility. Results reported by participants (2685 2979/sample) were graded for categorical accuracy and quantitative performance by comparing reported minimal inhibitory concentrations ( g/ml) or zone diameters (mm) against quality control ranges published by the NCCLS. The appropriateness of reported drugs was determined in the context of the type and anatomic location of the infection. Results. The tests most often used varied by the species of the organism and growth characteristics of the isolated strains. Nonfastidious, rapid-growing Surveys unknowns (Escherichia coli ATCC 25922, Pseudomonas aeruginosa ATCC 27853) were most often tested with commercial systems (, 42.0% 42.4%; Vitek, 41.5% 43.0%) or with the standardized disk diffusion method (12.8% 13.9%). In contrast, fastidious species, such as Streptococcus pneumoniae (ATCC 49619), were predominantly tested by Etest (40.3%), followed by disk diffusion (27.6%) and (23.2%). Categorical accuracy was essentially equal between dilution (98.9%) and diffusion (99.0%) methods. Among the minimal inhibitory concentration methods used to test penicillin against S pneumoniae, Etest method quantitative accuracy (96.3%) was greater than that of (92.4%). Quantitative accuracy was greatest for dilution minimal inhibitory concentration methods, with more than 90% of results within NCCLS quality control ranges for nearly all reported antimicrobials. Reevaluations of quality control ranges may be needed for 4 to 7 agents, depending on method. Reporting errors were also detected in 2 areas: (1) reporting results for drugs not active at the site of infection and (2) reporting results for drugs tested with suboptimal methods without published NCCLS interpretive criteria. Conclusions. Antimicrobial susceptibility testing methods used in US laboratories were dominated by commercial products with relatively high accuracy (qualitative and quantitative). As available methods have become better suited to both fastidious and rapid-growing species, reporting errors have assumed a higher level of concern to the CAP Surveys in an effort to minimize prescription errors. (Arch Pathol Lab Med. 2001;125:1285 1289) The College of American Pathologists (CAP) Surveys Program for clinical microbiology represents one of the largest comprehensive external quality assurance programs in the world. A broad variety of Surveys are available; they include 2 to 3 clinical challenge mailings of specimens covering the topics of bacteriology, mycology, mycobacteriology, parasitology, virology, and associated cell culture diagnostics, as well as focused programs on molecular diagnostics and epidemiology of infectious diseases. An important component of the CAP Bacteriology Accepted for publication June 1, 2001. From Tufts University School of Medicine, Boston, Mass, and The JONES Group/JMI Laboratories, North Liberty, Iowa. Reprints: Ronald N. Jones, MD, 345 Beaver Kreek Centre, Suite A, North Liberty, IA 52317 (e-mail: ronald-jones@jonesgr.com). Survey has been the antimicrobial susceptibility testing challenges (3 4 organisms/y). Previous reports from this program have documented consistent high-quality performance overall, 1 11 but periodic methods or commercial product deficiencies have led to technical or methods modifications by the National Committee for Clinical Laboratory Standards (NCCLS) 12 14 and/or product changes by various commercial systems. The results of the 2000 CAP Surveys are summarized in this article, including the tabulation of the most utilized antimicrobial susceptibility testing methods or systems. MATERIALS AND METHODS In 2000, the CAP Microbiology Surveys (D-series) had 3857 subscribing laboratories that were sent 3 unknown challenge organisms for routine susceptibility testing. The strains included Arch Pathol Lab Med Vol 125, October 2001 Antimicrobial Susceptibility Testing Jones 1285

Table 1. Use Statistics From the College of American Pathologists Surveys Program for 2000 Listing Antimicrobial Susceptibility Test Methods and Commercial Systems* No. of Occurrences (%) for Each Sample D-03 D-09 Method or System (Streptococcus pneumoniae) (Escherichia coli) Disk diffusion 740 (27.6) 351 (12.8) 413 (13.9) Dilution methods Agar Broth (macro) Broth (micro) Commercial API Micromedia PASCO Sceptor Sensititre Vitek Etest Total No. of responses * indicates not tabulated. Includes 5 Vitek-II users. 1945 (72.4) 622 (23.2) 16 (0.6) 1083 (40.3) 2685 2388 (87.2) 6 (0.2) 11 (0.4) 1( 0.1) 3 (0.1) 1151 (42.0) 16 (0.6) 4 (0.2) 10 (0.4) 1179 (43.0) 2( 0.1) 2739 D-17 (Pseudomonas aeruginosa) 2566 (86.1) 6 (0.2) 12 (0.4) 10 (0.3) 1( 0.1) 1261 (42.4) 16 (0.5) 4 (0.1) 13 (0.4) 1228 (41.5) 2979 well-characterized quality control (QC) strains recommended for regular, routine use by the NCCLS. 12 14 These strains were Escherichia coli (ATCC 25922; sample D-09, 2000), Pseudomonas aeruginosa (ATCC 27853; sample D-17, 2000), and Streptococcus pneumoniae (ATCC 49619; sample D-03, 2000). Each strain was to be processed by identification and susceptibility methods routinely used in the participating laboratory, with the reporting of only those antimicrobial agents considered appropriate for the clinical settings stated on the CAP Survey. The clinical infection settings for each pathogen were meningitis caused by S pneumoniae, blood stream infection caused by E coli, and a P aeruginosa lower respiratory tract infection diagnosed by a simulated bronchoalveolar lavage. The reporting of antimicrobials that achieved clinically adequate concentrations only in the urine (cinoxacin, nitrofurantoin, norfloxacin, trimethoprim, etc) was considered unacceptable performance for these 3 samples. Acceptable graded categorical results were those achieved by greater than 90% of both referees and participants. RESULTS AND COMME Table 1 lists the methods and commercial systems utilized by the CAP Surveys participants in processing the year 2000, Survey D organisms. A total of 2685 to 2979 (69.9% 77.6% of all Surveys participants) sets of test results were submitted by the subscribers. Very consistent methods results were noted for challenges D-09 and D-17, for which nearly 12.8% to 13.9% of laboratories still used the standardized disk diffusion method. 12 Among the dilution (manual or automated) tests or systems reported, the 2 dominant systems were Vitek (41.5% 43.0%) and (42.0% 42.4%). Use of any other single method or system by participants was less than 1.0%. In contrast (Table 1), the Etest (AB BIODISK, Solna, Sweden) was the principal method (40.3%) used for testing the fastidious organism, S pneumoniae. The other most commonly used methods to test pneumococci were the disk diffusion test (27.6%), (23.2%), and PAS- CO (0.6%). Other methods were infrequently used, usually because of organism-specific method/system disclaimers, and were not included to tabulate rates. Table 2 lists the susceptibility testing categorical accuracy (susceptible, intermediate, resistant) for specimens D- 09 and D-17. High-level accuracy was demonstrated for all listed agents for disk diffusion (90.0% %; average 98.96%) and minimal inhibitory concentration (MIC) (90.2% %; average 98.92%) test users. These rates compare favorably to prior Surveys reports, before the 90% participant and referee consensus grading policy was instituted. The standard for grading of the earlier Surveys challenges was greater than 80% consensus of results from selected referees who used NCCLS methods, 12,13 not commercial or automated systems. The evaluation of the S pneumoniae isolate (clinical setting of meningitis) produced highly accurate categorical values from the 3 most frequently used MIC methods (Table 3). The quantitative accuracy of Etest and methods was also very acceptable when compared with published NCCLS QC ranges (Table 4). Among the appropriate antimicrobial agents listed in Table 3 (cefotaxime, ceftriaxone, chloramphenicol, penicillin, and vancomycin), the test accuracy was 98.0%, 98.2%, and 97.1% for, Etest, and PASCO, respectively. For penicillin tests, the test accuracy was slightly less, and the accuracy rank order for these products was Etest (94.1%), followed by (91.3%) and PASCO (90.6%). This degree of interpretive (categories) and quantitative accuracy for the evaluated dilution test systems was remarkably high and unprecedented in the CAP Surveys using a S pneumoniae challenge strain. 1,5,6,9 For the pneumococcal Surveys sample, several laboratories reported disk diffusion results for antimicrobials for which no current interpretive criteria are available in NCCLS tables and/or the disk diffusion method was considered suboptimal. 12 These results were graded as unacceptable, regardless of their agreement with consensus qualitative, categorical susceptibility values. Similarly, many dilution test users reported antimicrobial agents that did not achieve adequate therapeutic concentrations in cerebrospinal fluid. This routine practice of reporting clinically inappropriate antimicrobial agents results could cause some physicians to select useless therapeutic agents. Selection of such drugs could lead to adverse or fatal clinical outcomes in this clinically emergent setting (cerebrospinal fluid and meningitis). These results were not penalized, but subsequent participant reports of test results of this type will result in unsatisfactory grades. 1286 Arch Pathol Lab Med Vol 125, October 2001 Antimicrobial Susceptibility Testing Jones

Table 2. Categorical Accuracy 14,16 of Antimicrobial Susceptibility Testing Methods for 2 College of American Pathologists Survey Challenge Strains in 2000 (D-09, Escherichia coli; D-17, Pseudomonas aeruginosa) Antimicrobial Amikacin Amoxicillin/clavulanate Ampicillin Ampicillin/Sulbactam Aztreonam Carbenicillin Cefazolin Cefepime Cefoperazone Cefotaxime Cefotetan Cefoxitin Ceftazidime Ceftizoxime Ceftriaxone Cefuroxime Cephalothin Ciprofloxacin Gentamicin Imipenem Levofloxacin Meropenem Mezlocillin Ofloxacin Piperacillin Piperacillin/tazobactam Tetracycline Ticarcillin Ticarcillin/clavulanate Tobramycin Trimethoprim/sulfamethoxazole Disk Diffusion D-09 D-17 98.4 97.9 93.9 98.6 97.2 90.9 90.0 97.1 Minimal Inhibitory Concentration Methods D-09 D-17 98.2 98.2 98.6 98.0 97.2 90.2 97.0 99.5 97.0 99.3 98.0 92.1 98.1 97.5 96.8 The NCCLS Subcommittee on Antimicrobial Susceptibility Testing will address infection site specific guidelines (meningitis, urine, other sites) for future publication. 14 The position statement of the CAP Microbiology Resource Committee and more detailed survey critiques for each of these specimens cited can be found at the CAP Web site (www.cap.org). Use of NCCLS QC strains as Surveys specimens affords the opportunity to judge participant performance on qualitative and quantitative scales. Defined QC ranges of MICs or millimeter zones 14 can be compared with the participant-reported quantitative results. Table 5 lists the proportions of participant results found within published NCCLS QC ranges 14 for a subset of problematic antimicrobials. Based on these contemporary real-world statistics from a large volume of CAP Survey participants, Table 3. Categorical Accuracy of the 3 Most Commonly Reported Dilution Test Systems/Methods for the 10 Most Frequently Tested Antimicrobial Agents (8018 responses; Streptococcus pneumoniae ATCC 49619 used in College of American Pathologists Specimen D-03, 2000) Antimicrobial Agent (No. of Tests)* Appropriate agents Cefotaxime (1023) Ceftriaxone (1257) Chloramphenicol (517) Penicillin (1807) Vancomycin (1024) Inappropriate agents Clindamycin (431) Erythromycin (699) Levofloxacin (187) Tetracycline (472) Trimethoprim/sulfamethoxazole (527) % Categorical Accuracy (No. of Tests) (4223) Etest (3570) PASCO (285) 91.3 99.5 * Number of tests for the 3 listed systems/methods. The categorical accuracy 16 was acceptable; however, reporting of these agents for a case of meningitis would be considered inappropriate. Arch Pathol Lab Med Vol 125, October 2001 Antimicrobial Susceptibility Testing Jones 1287 94.1 93.8 96.2 97.4 97.5 90.6 87.0 92.9

Table 4. Quantitative Accuracy* of Commonly Used Dilution Test Systems/Methods When Testing 10 Antimicrobials Against Streptococcus pneumoniae ATCC 49619 (College of American Pathologists Specimen D-03, 2000) Antimicrobial Agent Appropriate agents Cefotaxime Ceftriaxone Chloramphenicol Penicillin Vancomycin Inappropriate agents Erythromycin Clindamycin Levofloxacin Tetracycline Trimethoprim/sulfamethoxazole 96.4 (502) (552) 96.7 (421) 92.4 (622) 93.9 (587) (477) (376) 45.4 (412) 94.5 (439) % Accuracy (No. of Tests) Etest 88.9 (442) 92.3 (552) 92.4 (66) 96.3 (1083) 90.6 (360) 99.5 (195) 98.9 (174) 60.8 (51) Total (All Methods) 93.3 (1052) 95.3 (1319) 96.3 (563) (1875) 92.9 (1074) 97.9 (766) 96.6 (468) 99.0 (199) 53.4 (521) 90.7 (578) * Accuracy determined by the number of reported minimal inhibitory concentration (MIC) values within published MIC quality control ranges 16 or manufacturer s package insert (AB BIODISK, Solna, Sweden). Only those organism/system or method combinations with 50 reported results for the tested drug are listed. Drugs considered inappropriate for reporting in a clinical case of meningitis. several antimicrobials may require reevaluation of their QC ranges. Examples of the most aberrant results for the disk diffusion test were as follows (percentage of results within control limits; target 90%): Ecoli, ticarcillin/clavulanate (54.0%), cefazolin (61.7%), and amoxicillin/clavulanate (70.0%); P aeruginosa, ofloxacin (72.7%) and ticarcillin (71.6%). The antimicrobials needing further study of MIC QC ranges were more limited (percentage of results within control guidelines): P aeruginosa, amikacin (80.3%) and cefotaxime (82.5%; data not shown). These results from the antimicrobial susceptibility testing component of the 2000 CAP Microbiology Surveys demonstrate acceptable test accuracy and reproducibility overall for the most commonly used testing systems or methods. Few diagnostic problems were identified, but some possible QC guideline flaws need rapid resolution. To minimize potentially fatal prescription errors (1999 Report of the Institute of Medicine, Washington, DC), 15 clinical laboratories performing susceptibility tests must carefully select agents for reporting in emergent clinical infections (meningitis, bacteremia). 16 Where antimicrobial susceptibility testing results are reported for epidemiologic purposes only, in addition to potential therapeutic agents for meningitis, these results should be clearly delineated as not having relevance to the treatment of the index case. No laboratory should have complicity in the negligence of placing a patient in harm s way by reporting test results leading to inappropriate therapeutic choices by clinicians. The CAP Microbiology Resource Committee appreciates the constructive suggestions received from Surveys participants. Participant input will serve to strengthen the program and elevate the quality of our laboratory discipline. Members of the College of American Pathologists Microbiology Resource Committee included W. C. Winn, Jr, chair (University of Vermont College of Medicine, Burlington, Vt); J. C. Steele, Jr, vice-chair (Medical College of Georgia, Augusta, Ga); A. D. Alrahwan (University of Texas, San Antonio, Tex); K. Beavis (Cook County Hospital, Chicago, Ill); J. D. Christie (East Carolina University, Greenville, NC); J. L. Harris (Pathology Associates, Tallahassee, Fla); E. Koneman (Breckenridge, Colo); D. N. Persing (Corixa Corp, Seattle, Wash); M. A. Pfaller (University of Iowa, Iowa City); W. O. Rogers (NMRC Malaria Program, Silver Springs, Md); J. Versalovic (Massachusetts General Hospital, Boston); M. M. Yungbluth (St Francis Hospital, Evanston, Ill); D. L. Church, Consultant (Calgary Laboratory Services, Calgary, Alberta); and L. S. Garcia, consultant (Santa Monica, Calif). Table 5. Quantitative Performance of Disk Diffusion Test Users in the College of American Pathologists Surveys (D-09 and D-17, 2000). Only Those Agents With Suboptimal Quantitative Performance Are Listed or 80% of Participant Results Within National Committee for Clinical Laboratory Standards Quality Control Guidelines 16 Antimicrobial Agent (No. of Responses) Amoxicillin/clavulanate (70) Ampicillin/sulbactam (88) Cefazolin (219) Cefepime (35) Cefoperazone (30) Cefotaxime (54) Ceftriaxone (164) Ofloxacin (22) Ticarcillin (53) Ticarcillin/clavulanate (50) Trimethoprim/sulfamethoxazole (133) D-09 (Escherichia coli ATCC 25922) NCCLS Range, mm 19 25 20 24 23 29 29 35 29 35 25 29 24 32 Participant Results Mean Zone, mm 19.8 20.5 22.9 32.5 29.7 25.1 24.9 % in Range D-17 (Pseudomonas aeruginosa ATCC 27853) NCCLS Range, mm Participant Results Mean Zone, mm % in Range 1288 Arch Pathol Lab Med Vol 125, October 2001 Antimicrobial Susceptibility Testing Jones 70.0 76.1 61.7 74.4 70.8 54.0 79.6 23 29 18 22 17 21 22 28 25.8 19.5 17.4 22.5 79.9 79.7 72.7 71.6

The coauthors thank the following support staff at the College of American Pathologists for significant contributions to the 2000 Surveys Program and analyses of participant data: D. Bird and S. Schneider. Additional appreciation is extended to K. L. Meyer for manuscript suggestions and preparation. References 1. Doern GV, Brueggemann AB, Pfaller MA, Jones RN. Assessment of laboratory performance with Streptococcus pneumoniae antimicrobial susceptibility testing in the United States: a report from the College of American Pathologists Microbiology Proficiency Survey Program. Arch Pathol Lab Med. 1999;123:285 289. 2. Jones RN. Antimicrobial susceptibility testing (AST): a review of changing trends, quality control guidelines, test accuracy, and recommendation for the testing of -lactam drugs. Diagn Microbiol Infect Dis. 1983;1:1 24. 3. Jones RN. Recent trends in the College of American Pathologists proficiency results for antimicrobial susceptibility testing: preparing for CLIA 88. Clin Microbiol Newsl. 1992;14:33 37. 4. Jones RN, Edson DC. Interlaboratory performance of disk agar diffusion and dilution antimicrobial susceptibility tests, 1979 1981: a summary of the microbiology portion of the College of American Pathologists (CAP) Surveys. Am J Clin Pathol. 1982;78(suppl):651 658. 5. Jones RN, Edson DC. The ability of participant laboratories to detect penicillin-resistant pneumococci: a report from the microbiology portion of the College of American Pathologists (CAP) Surveys. Am J Clin Pathol. 1982;78(suppl): 659 663. 6. Jones RN, Edson DC. Special topics in antimicrobial susceptibility testing: test accuracy against methicillin-resistant Staphylococcus aureus, pneumococci, and the sensitivity of -lactamase methods. Am J Clin Pathol. 1983;80(suppl): 609 614. 7. Jones RN, Edson DC. Antibiotic susceptibility testing accuracy: a review of the College of American Pathologists (CAP) Microbiology Survey, 1972 1983. Arch Pathol Lab Med. 1985;109:595 601. 8. Jones RN, Edson DC, for the Microbiology Resource Committee of the College of American Pathologists. The identification and antimicrobial susceptibility testing of Neisseria gonorrhoeae, 1980 1987: results from the College of American Pathologists Microbiology Surveys Program. Arch Pathol Lab Med. 1988; 112:485 488. 9. Jones RN, Edson DC. Antimicrobial susceptibility testing (AST) trends and accuracy in the United States: a review of the College of American Pathologists Microbiology Surveys, 1972 1989. Arch Pathol Lab Med. 1991;115:429 436. 10. Jones RN, Edson DC, Gilmore BF. Contemporary quality control practices for antimicrobial susceptibility tests: a report from the microbiology portion of the College of American Pathologists (CAP) Surveys Program. Am J Clin Pathol. 1983;80(suppl):622 625. 11. Jones RN, Edson DC, Marymont JV. Evaluations of antimicrobial susceptibility test proficiency by the College of American Pathologists Survey Program: a clarification of quality control recommendations. Am J Clin Pathol. 1982;78: 168 172. 12. National Committee for Clinical Laboratory Standards. Performance Standards for Antimicrobial Disk Susceptibility Tests. 7th ed. Approved standard M2- A7. Wayne, Pa: National Committee for Clinical Laboratory Standards; 2000. 13. National Committee for Clinical Laboratory Standards. Methods for Dilution Antimicrobial Susceptibility Tests for Bacteria That Grow Aerobically. 5th ed. Approved standard M7-A5. Wayne, Pa: National Committee for Clinical Laboratory Standards; 2000. 14. National Committee for Clinical Laboratory Standards. Performance Standards for Antimicrobial Susceptibility Testing. 10th informational supplement M100-S10. Wayne, Pa: National Committee for Clinical Laboratory Standards; 2000. 15. Barlas S. The push to report medical errors. Pharm Ther. 2001;26:21. 16. Gilbert DN, Moellering RC, Sande MA. The Sanford Guide to Antimicrobial Therapy, 2000. 13th ed. Dallas, Tex: Antimicrobial Therapy Inc; 2000. Arch Pathol Lab Med Vol 125, October 2001 Antimicrobial Susceptibility Testing Jones 1289