Short Report Frequent major errors in antimicrobial susceptibility testing of bacterial strains distributed under the Deutsches Krebsforschungszentrum Quality Assurance Program R Boot Former Section of Laboratory Animal Microbiology, Diagnostic Laboratory for Infectious Diseases, National Institute of Public Health and the Environment, PO Box 1, 3720 BA Bilthoven, The Netherlands Email: r.boot@hotmail.com Abstract The Quality Assurance Program (QAP) of the Deutsches Krebsforschungszentrum (DKFZ) was a proficiency testing system developed to service the laboratory animal discipline. The QAP comprised the distribution of bacterial strains from various species of animals for identification to species level and antibiotic susceptibility testing (AST). Identification capabilities were below acceptable standards. This study evaluated AST results using the DKFZ compilations of test results for all bacterial strains showing the number of participants reporting the strain as resistant (R), sensitive (S) or intermediate susceptible (I) to each antibiotic substance used. Due to lack of information about methods used, it was assumed that what the majority of the participants reported (R or S) was the correct test result and that an opposite result was a major error (ME). MEs occurred in 1375 of 14,258 (9.7%) of test results and ME% ranged from 0% to 23.2% per bacterial group agent group combination. Considerable variation in MEs was found within groups of bacteria and within groups of agents. In addition to poor performance in proper species classification, the quality of AST in laboratory animal diagnostic laboratories seems far below standards considered acceptable in human diagnostic microbiology. Keywords: Bacteria, antimicrobial susceptibility testing, quality, diagnostic laboratories, proficiency testing Laboratory Animals 2012; 46: 253 257. DOI: 10.1258/la.2012.011085 The Quality Assurance Program (QAP) of the Deutsches Krebsforschungszentrum (DKFZ, Dr Werner Nicklas) was a proficiency testing system developed to service the laboratory animal field. The QAP comprised the quarterly distribution of two bacterial strains originating from various species of animal for identification to the species level and antibiotic susceptibility testing (AST). In 2007 we compared the identification results reported by QAP participants over the years 1996 2004 with those obtained by the Dutch Bacterial Diagnostics reference laboratory, the only participant that used genetically-based methods for identification. We found that the capabilities of laboratory animal diagnostic laboratories to correctly identify bacterial species was considerably below what is considered acceptable for human diagnostic laboratories. 1 The outcome of the AST over the same period is now reported. Identification results were available for 71 bacterial strains and a fungus. In the present study the outcome of AST of five of the 72 strains was omitted. In QAP distribution no. 24, the same outcome was reported for Staphylococcus cricetus and CDC EF4a, suggesting an error in record keeping; in QAP distribution no. 26 results of European laboratories could not be separated from those from the USA; and the one strain of QAP distribution no. 42 was omitted, being a fungus. As a consequence AST results on 66 bacterial strains remained. The bacteria comprised of Gram-positive rods (n ¼ 6) and cocci (n ¼ 21) and Gram-negative rods (n ¼ 38) and a coccus (n ¼ 1). Where possible the bacteria were grouped in bacterial families containing at least three strains. According to their Gram stain characteristics and morphology, the remaining bacteria were assigned to the groups other Gram-negative rods, other Gram-positive cocci and other Gram-positive rods. In this way 11 groups were formed. The number of strains per group ranged from one (the Gram-negative Neisseria) to 21 (Enterobacteriaceae). The AST results for each strain were compiled by Dr Nicklas in a list showing the number of participants reporting the strain as resistant (R), sensitive (S) or intermediate susceptible (I) to each antibiotic substance used. All data were included in an Excel file which was used for Laboratory Animals 2012; 46: 253 257
254 Laboratory Animals Volume 46 July 2012 Table 1 Percentage major errors (MEs) in testing susceptibility of 66 bacterial strains for antimicrobial agents Am b-l V F-I Linc Mac Pol Quin Tet Var All ME average Minimum Maximum Neisseriaceae R 43 56 5 18 9 14 7 14 21 25 212 ME n ¼ 5 8 0 2 0 2 0 0 1 3 21 % 11.6 14.3 0 11.1 0 14.3 0 0 4.76 12 10 0 14.3 Alcaligenaceae R 229 306 16 114 41 71 47 67 116 128 1135 ME n ¼ 14 43 0 26 0 11 2 4 5 10 115 % 6.11 14.1 0 22.8 0 15.5 4.26 5.97 4.31 7.81 10.1 0 22.8 Enterobacteriaceae R 802 1347 90 359 160 293 183 266 379 451 4330 ME n ¼ 69 79 2 37 6 11 16 12 28 33 293 % 8.6 5.86 2.22 10.3 3.75 3.75 8.74 4.51 7.39 7.32 6.7 2.22 10.3 Moraxellaceae R 123 178 11 52 22 41 23 35 49 66 600 ME n ¼ 2 19 0 12 1 5 0 3 2 3 47 % 1.63 10.7 0 23.1 4.55 12.2 0 8.57 4.08 4.55 7.8 0 23.1 Other Gram-negative rods R 368 527 34 152 60 113 81 113 156 176 1780 ME n ¼ 42 61 2 34 3 23 4 6 23 17 215 % 11.4 11.6 5.88 22.4 5 20.4 4.94 5.31 14.7 9.66 12.1 4.94 22.4 Enterococcaceae R 133 198 24 59 34 55 28 37 66 81 715 ME n ¼ 11 38 3 12 2 1 2 7 10 14 100 % 8.27 19.2 12.5 20.3 5.88 1.82 7.14 18.9 15.2 17.3 14 1.82 20.3 Staphylococcaceae R 278 401 39 120 70 105 51 78 129 184 1455 ME n ¼ 12 31 1 26 3 2 11 3 4 5 98 % 4.32 7.73 2.56 21.7 4.29 1.9 21.6 3.85 3.1 2.72 6.7 1.9 21.7 Streptococcaceae R 303 485 44 144 75 122 67 90 142 181 1653 ME n ¼ 50 43 1 32 8 8 12 12 15 17 198 % 16.5 8.87 2.27 22.2 10.7 6.56 17.9 13.3 10.6 9.39 12 2.27 22.2 Other Gram-positive cocci R 189 291 29 89 46 61 40 61 86 113 1005 ME n ¼ 37 55 2 12 8 8 2 8 13 22 167 % 19.6 18.9 6.9 13.5 17.4 13.1 5 13.1 15.1 19.5 16.6 5 0 Corynebacteriaceae R 158 241 27 69 38 69 37 43 77 99 858 ME n ¼ 13 13 0 16 1 1 6 1 2 6 59 % 8.23 5.39 0 23.2 2.63 1.45 16.2 2.33 2.6 6.06 6.9 0 23.2
Boot. Antimicrobial susceptibility testing QAP-strains 255 Other Gram-positive rods R 75 120 9 36 21 29 15 25 41 44 415 ME n ¼ 15 16 2 5 2 6 1 6 3 6 62 % 20 13.3 22.2 13.9 9.52 20.7 6.67 24 7.32 13.6 15 6.67 24 All R 2701 4150 328 1212 576 973 579 829 1262 1548 14158 ME n ¼ 270 406 13 214 34 78 56 62 106 136 1375 Average % 10 9.78 3.96 17.7 5.9 8.02 9.67 7.48 8.4 8.79 15 3.96 17.7 Minimum 1.63 5.39 0 10.3 0 1.45 0 0 2.6 2.72 Maximum 20 19.2 22.2 23.2 17.4 20.7 21.6 24 15.2 19.5 Am: aminoglycosides (n ¼ 6); b-l: beta-lactam antibiotics (n ¼ 11); V: vancomycin (n ¼ 1); F-I: inhibitors of folic acid synthesis (n ¼ 3); Linc: lincoamides (n ¼ 2); Mac: macrolides (n ¼ 3); Pol: polymyxines (n ¼ 2); Quin: quinolones (n ¼ 2); Tet: tetracyclines (n ¼ 3); Var: various (n ¼ 5) Bacteria grouped in families comprising 3 strains or according to Gram stain characteristics. Antimicrobials grouped according to mode of activity calculations. The 66 strains included in this study were numbered one to 66. Antimicrobial substances During the study period 1996 2004, the participating laboratories reported the use of 43 antimicrobial agents. Piperacillin, naladixic acid, ofloxacin, metronidazole and spectinomycin were only rarely used and results obtained with these five agents were omitted from calculations. It was possible so far to group the remaining 38 substances according to their mode of action in nine groups comprising aminoglycosides (n ¼ 6: amikacin, gentamycin, kanamycin, neomycin, streptomycin and tobramycin), betalactam antibiotics (n ¼ 11: amoxicillin, amoxicilin/clavulanic acid, ampicillin, bacitracin, carbenicillin, imipenem, oxacillin, penicillin, ticarcillin, cefotaxim and cephalotin), glycopeptides (n ¼ 1: vancomycin), inhibitors of folic acid synthesis (n ¼ 3: co-trimoxazole, sulfonamide and trimethoprim-sulfamethoxazole); lincosamides (n ¼ 2: clindamycin and lincomycin), macrolides (n ¼ 3: erytromycin, spiramycin and tylosin), polymixines (n ¼ 2: colistin and polymixin B), quinolones (n ¼ 2 ciprofloxacin and enrofloxacin) and tetracyclines (n ¼ 3: doxycycline, oxytetracycline and tetracycline). The remaining five agents were included in the 10th rest group comprising chloramphenicol, fucidine, nitrofurantoin, novobiocin and rifampicin. The number of antimicrobials per group ranged from 1 (vancomycin) to 11 (beta-lactam antibiotics). Laboratories did not provide information regarding the methods used and no information was available regarding which strains were studied by specific laboratories. It must be assumed that most laboratories would have used so-called disc methods, and some may have had minimum inhibitory concentrations (MICs). However, this lack of information meant that only overall results could be described and that it was impossible to evaluate the data for possible statistically significant differences, for instance between participants, methods, groups of bacteria and antibiotic substances. Each test of the susceptibility of a bacterial strain for an antimicrobial substance yields an individual test outcome which obviously may be scored as either R, S or I. It has been assumed that what the majority of the participants reported is the correct test result, as there is no alternative. If the majority reports a bacterium R or S for a given substance, I results are acceptable. Where bacterium antimicrobial combinations yielded a majority of I results or an equal number of S and R outcomes, a majority was created by addition of the number of I to either the number of R or S (so R þ I. SorSþI. R). The opposite outcome (S and R, respectively) reported by the minority was then considered a major error (ME). The total number of test results reported from the use of the 38 agents and 66 bacteria was 14,258. For each bacterium antimicrobial combination it may be expected that the lower the number of AST test results the more likely it will be to have no MEs. Conversely, the likelihood of finding disagreement in test results between
256 Laboratory Animals Volume 46 July 2012 laboratories increases with the number of test results (data not shown; correlation not calculated). The ME percentages were classified and, although this may seem arbitrarily, 10% MEs means that one in 10 test results deviates from what was reported by the majority; with 20% and 33.3% MEs one in five and one in three test results, respectively, deviates from the correct test result. I counted 1375 MEs among 14,258 test results (9.7%). Table 1 shows the ME rates for each of the 11 groups of bacteria and the 10 groups of antimicrobial substances. The ME% ranged from 0 to 23.2. Percentages exceeding 20 were found in seven of 11 groups of bacteria (range of averages 6.7 16.6) and with six of 10 groups of antibacterial substances (range of averages 4 15). Figure 1 shows the average ME% calculated for all successively distributed 66 bacterial strains. Of course grouping of bacteria and grouping of antibacterial substances masks variation in AST results within groups. As an example, Table 2 specifies findings obtained for the 11 bacterial groups with the six individual aminoglycosides. Of the 66 bacterium antimicrobial combinations possible, 51 of 66 (77%) yielded MEs. Of the 121 bacterium antimicrobial combinations possible with the 11 betalactam antibiotics, 97 of 121 (80%) showed MEs. Differences in the outcome of AST of bacteria have been reported. 2,3 Discrepancies in findings between laboratories have motivated improvement through the publication of standards and guidelines for AST and interpretation of results. A series of recommendations has been issued by the Clinical and Laboratory Standards Institute (CLSI), 4 as a successor to the (US) National Committee for Clinical Laboratory Standards (NCCLS). Other recommendations, however, exist. 5 The CLSI guidelines provide guidance on the most relevant drugs to test and report on specific organisms, quality control ranges to assure accurate and reproducible Figure 1 Percentage of major errors in antimicrobial susceptibility testing of 66 bacterial strains successively distributed under the Deutsches Krebsforschungszentrum Quality Assurance Program (1996 2004) results, and interpretative criteria or breakpoints to interpret MICs and disc diffusion zones. The most widely used CLSI recommendations are those for disc diffusion tests (M2) 6 and for the determination of MICs (M7) 7 of various drugs. The CLSI issues cover a wide variety of bacterial species including infrequently isolated and fastidious bacteria. 8 Guideline M37 (version 3) 9 issued in 2008 deals with antimicrobial substances used in the veterinary field. It is unclear whether the laboratory animal diagnostic laboratories that participated in the QAP followed such recommendations. The data show that AST of bacteria by laboratory animal diagnostic laboratories may frequently yield conflicting results. If most laboratories report a bacterium S to a given antimicrobial agent, an opposite test result (R) should be considered an ME. The percentage of MEs appeared to be about 10, and in various bacteria antimicrobial combinations tested ME% were above 20. Based on ME percentages (Figure 1) the capability of participating laboratories in AST did not improve over time. According to CLSI recommendation M23A 10 the ME rate should not exceed 3.5%. AST has several pitfalls and an analysis of recently published articles revealed a number of frequently occurring Table 2 Percentage major errors (MEs) in testing susceptibility of 66 bacterial strains for six aminoglycosides n 5 Amikacin Gentamycin Kanamycin Neomycin Streptomycin Tobramycin Neisseriaceae 1 R n 4 12 7 8 9 3 ME % 25 0 0 37.5 11 0 Alcaligenaceae 5 R n 19 62 36 44 49 19 ME % 0 6.5 8.5 7 8 0 Enterobacteriaceae 21 R n 84 226 124 141 145 82 ME % 8.5 7.5 8 8 11 10 Moraxellaceae 3 R n 13 34 17 22 24 13 ME % 0 3 0 0 0 8 Other Gram-negative rods 9 R n 33 107 54 66 72 36 ME % 12 14 4 13.5 14 5.5 Enterococcaceae 3 R n 10 38 27 21 28 9 ME % 0 13 7.5 14.5 3.5 0 Staphylococcaceae 7 R n 29 74 53 44 51 27 ME % 3.5 4 7.5 4.5 4 0 Streptococcaceae 7 R n 23 87 53 56 63 21 ME % 13 21 13 18 11 24 Other Gram-positive cocci 4 R n 17 48 30 37 38 19 ME % 17.5 19 30 16 18.5 16 Corynebacteriaceae 4 R n 12 46 23 28 36 13 ME % 8.5 11 4.5 3.5 14 0 Other Gram-positive rods 2 R n 5 19 13 17 14 7 ME % 0 16 38.5 17.5 28.5 0
Boot. Antimicrobial susceptibility testing QAP-strains 257 shortcomings in AST of animal bacterial strains, which may have an impact either directly on the quality of the results obtained or on the conclusions drawn. 11 We reported earlier that laboratory animal diagnostic laboratories showed poor performance in proper species classification of bacterial strains. 1 The present evaluation indicates that the quality of AST by laboratory animal diagnostic laboratories participating in the DKFZ-QAP over the period 1996 2004 was also far below standards considered acceptable in human diagnostic microbiology. In the last 10 15 years, several automated instruments have been developed for identification and AST of bacterial strains considered relevant in human and veterinary infections. 12,13 Their advantages comprise ease of use, reduction of work load and standardization of reading test results. A major disadvantage of all systems is that they require the manual preparation of inocula and inoculation of test panels. Inoculum size has been found to be a primary determinant of the accuracy of AST by the Vitek system. 14 It is unclear to which extent automated instruments are presently used by laboratory animal diagnostic laboratories. Unfortunately, DKFZ-QAP was terminated in 2004. Today the only proficiency testing system to service the laboratory animal discipline is the Performance Evaluation Program (PEP) for Diagnostic Laboratories established in 2006 by ICLAS. 15 The PEP does not however include AST, so no more recent data are available than those summarized here. Until new data are published, one should be concerned by the limited capabilities of laboratory animal diagnostic laboratories in AST of bacteria. REFERENCES 1 Boot R, Reubsaet FAG. Identification of bacterial strains by laboratories participating in the DKFZ Quality Assurance Program (QAP). Lab Anim 2007;41:481 91 2 Tenover FC, Mohammed MJ, Stelling J, et al. Ability of laboratories to detect antimicrobial resistance: proficiency testing and quality control results form the World Health Organization s external quality assurance system for antimicrobial susceptibility testing. J Cin Microbiol 2001;39:241 50 3 Varaldo PE. Antimicrobial resistance and susceptibility testing: an evergreen topic. J Antimicrobial Chemother 2002;50:1 4 4 Clinical and Laboratory Standards Institute. See www.clsi.org (last checked 20 May 2012) 5 Gray JT, Shrycock TR. Antibiotic susceptibility testing of bacteria isolated from animals. Clin Microbiol Newsl 2005;17:131 5 6 CLSI. Performance Standards for Antimicrobial Disk Susceptibility Tests. Approved Standard M2-A9. Wayne, PA, 2006 7 CLSI. Methods for Dilution Antimicrobial Susceptibility Tests for Bacteria that Grow Aerobically. Approved Standard M7-A7. Wayne, PA, 2006 8 Jorgensen JH, Hindler JF. New consensus guidelines for the clinical and laboratory standards institute for antimicrobial susceptibility testing of infrequently isolated or fastidious bacteria. Clin Infect Dis 2007;44:280 6 9 CLSI. Development of In Vitro Susceptibility Testing Criteria and Quality Control Parameters for Veterinary Antimicrobial Agents. Approved Guideline M37, 3rd edn. Wayne, PA, 2008 10 NCCLS. National Committee for Clinical Laboratory Standards. Development of In Vitro Susceptibility Criteria and Quality Control Parameters. Approved Guideline M23-A. Villanova, PA, 1994 11 Schwartz S, Silley P, Simjee S, Woodford N, et al. Assesssing the antimicrobial susceptibility of bacteria obtained from animals. Vet Microbiol 2010;141:1 4 12 Barth Reller L, Weinstein N, Jorgensen JH, et al. Antimicrobial susceptibility testing: a review of general principles and contemporary practices. Clin Infect Dis 2009;49:1749 55 13 Tenover FC. Potential impact of rapid diagnostic testing on improving antimicrobial use. Ann NY Acad Sci 2010;1213:70 80 14 Doern GV, Brueggemann AB, Perla R, et al. Multicentre laboratory evaluation of the biomerieux Vitek antimicrobial susceptibility testing system with 11 antimicrobial agents versus members of the family Enterobacteriaceae and Pseudomonas aeruginosa. J Clin Microbiol 1997;35:2115 19 15 International Council for Laboratory Animal Science. See www.iclas. org/network.htm (last checked 20 May 2012) (Accepted 21 April 2012)