Aalborg Universie Inerpreaive reading of he anibiogram Andreassen, Hans Seen; Falborg, Alina Zalounina; Paul, Mical; Sanden, Line Rugholm; Leibovici, Leonard Published in: Arificial Inelligence in Medicine DOI (link o publicaion from Publisher): 10.1016/j.armed.2015.08.004 Publicaion dae: 2015 Documen Version Publisher's PDF, also known as Version of record Link o publicaion from Aalborg Universiy Ciaion for published version (APA): Andreassen, S., Zalounina, A., Paul, M., Sanden, L., & Leibovici, L. (2015). Inerpreaive reading of he anibiogram: a semi-naïve Bayesian approach. Arificial Inelligence in Medicine, 65(3), 209-217. DOI: 10.1016/j.armed.2015.08.004 General righs Copyrigh and moral righs for he publicaions made accessible in he public poral are reained by he auhors and/or oher copyrigh owners and i is a condiion of accessing publicaions ha users recognise and abide by he legal requiremens associaed wih hese righs.? Users may download and prin one copy of any publicaion from he public poral for he purpose of privae sudy or research.? You may no furher disribue he maerial or use i for any profi-making aciviy or commercial gain? You may freely disribue he URL idenifying he publicaion in he public poral? Take down policy If you believe ha his documen breaches copyrigh please conac us a vbn@aub.aau.dk providing deails, and we will remove access o he work immediaely and invesigae your claim. Downloaded from vbn.aau.dk on: April 29, 2017
Arificial Inelligence in Medicine 65 (2015) 209 217 Conens liss available a ScienceDirec Arificial Inelligence in Medicine j o ur na l ho mepage: www.elsevier.com/locae/aiim Inerpreaive reading of he anibiogram a semi-naïve Bayesian approach Seen Andreassen a, Alina Zalounina a, Mical Paul b, Line Sanden a,, Leonard Leibovici c a Cener for Model-Based Medical Decision Suppor, Aalborg Universiy, Fredrik Bajers Vej 7D2, DK-9220 Aalborg, Denmark b Uni of Infecious Diseases, Rambam Healh Care Campus, HaAliya HaShniya S 8, Haifa, Israel c Sackler Faculy of Medicine Tel-Aviv Universiy, Rama Aviv 69978, Israel a r i c l e i n f o Aricle hisory: Received 10 December 2014 Received in revised form 8 July 2015 Acceped 5 Augus 2015 Keywords: Bayes heorem Bacerial infecions Animicrobial herapy Anibiogram Cross-resisance Animicrobial Sewardship a b s r a c Background: An anibiogram (ABG) gives he resuls of in viro suscepibiliy ess performed on a pahogen isolaed from a culure of a sample aken from blood or oher issues. The insiuional cross-abg consiss of he condiional probabiliy of suscepibiliy for pairs of animicrobials. This paper explores how inerpreaive reading of he isolae ABG can be used o replace and improve he prior probabiliies sored in he insiuional ABG. Probabiliies were calculaed by boh a naïve and semi-naïve Bayesian approaches, boh using he ABG for he given isolae and insiuional ABGs and cross-abgs. Mehods and Maerial: We assessed an isolae daabase from an Israeli universiy hospial wih ABGs from 3347 clinically significan blood isolaes, where on average 19 animicrobials were esed for suscepibiliy, ou of 31 animicrobials in regular use for paien reamen. For each of 14 pahogens or groups of pahogens in he daabase he average (prior) probabiliy of suscepibiliy (also called he insiuional ABG) and he insiuional cross-abg were calculaed. For each isolae, he normalized Brier disance was used as a measure of he disance beween suscepibiliy es resuls from he isolae ABG and respecively prior probabiliies and poseriori probabiliies of suscepibiliy. We used a 5-fold cross-validaion o evaluae he performance of differen approaches o predic poserior suscepibiliies. Resuls: The normalized Brier disance beween he prior probabiliies and he suscepibiliy es resuls for all isolaes in he daabase was reduced from 37.7% o 28.2% by he naïve Bayes mehod. The smalles normalized Brier disance of 25.3% was obained wih he semi-naïve min2max2 mehod, which uses he wo smalles significan odds raios and he wo larges significan odds raios expressing respecively cross-resisance and cross-suscepibiliy, calculaed from he cross-abg. Conclusion: A pracical mehod for predicing probabiliy for animicrobial suscepibiliy could be developed based on a semi-naïve Bayesian approach using saisical daa on cross-suscepibiliies and cross-resisances. The reducion in Brier disance from 37.7% o 25.3%, indicaes a significan advanage o he proposed min2max2 mehod (p < 10 99 ). 2015 The Auhors. Published by Elsevier B.V. This is an open access aricle under he CC BY-NC-ND license (hp://creaivecommons.org/licenses/by-nc-nd/4.0/). 1. Inroducion When a paien presens a a hospial wih a bacerial infecion, animicrobials will be adminisered o he paien. Before he animicrobials are adminisered, samples will be aken from he Corresponding auhor a: Cener for Model-Based Medical Decision Suppor, Deparmen of Healh Science and Technology, Aalborg Universiy, Fredrik Bajers Vej 7D2, DK-9220 Aalborg, Denmark. Tel.: +45 29604034. E-mail addresses: sa@hs.aau.dk (S. Andreassen), az@hs.aau.dk (A. Zalounina), m paul@rambam.healh.gov.il (M. Paul), sanden@hs.aau.dk (L. Sanden), leibovic@pos.au.ac.il (L. Leibovici). paien, usually boh a blood sample and a local sample from he sie of infecion, for example a urine sample if he paien is suspeced of a urinary rac infecion. Wihin a day or wo baceria are successfully isolaed from hese samples in approximaely 30% of he paiens [1]. Once isolaed, he baceria are esed for heir in viro suscepibiliy o a range of animicrobials. The es resuls are called an anibiogram (ABG), which specifies he suscepibiliy of he pahogen o each esed animicrobial. These ABGs ofen make i relevan o change he iniial empirical reamen given o he paien ino a definiive reamen, where i is known from he suscepibiliy resuls ha he isolaed baceria are suscepible o he animicrobial(s) given in he definiive reamen. hp://dx.doi.org/10.1016/j.armed.2015.08.004 0933-3657/ 2015 The Auhors. Published by Elsevier B.V. This is an open access aricle under he CC BY-NC-ND license (hp://creaivecommons.org/licenses/by-nc-nd/4. 0/).
210 S. Andreassen e al. / Arificial Inelligence in Medicine 65 (2015) 209 217 A any given ime a large number of animicrobials are in use in a hospial. Ou of hese only a limied se of animicrobials is esed due o pracical and economic consrains. Therefore i occasionally happens ha none of he esed animicrobials are clinically accepable. This may for example be due o allergies, o significan oxiciy, o limied gasroinesinal absorpion of he drug in severely sepic paiens, o preferences for bacericidal raher han baceriosaic animicrobials or due o preferences imposed by animicrobial sewardship programs, where for example quinolones are considered a less desirable choice relaive o cephalosporins (Danish Healh and Medicines Auhoriy, 2013) [2]. In such cases i is desirable o have addiional informaion abou he suscepibiliy of he isolaed pahogens o animicrobials for which no suscepibiliy resuls are available. Some informaion can be derived from he exper rules of he European Commiee on Animicrobial Suscepibiliy Tesing (EUCAST) [3]. EUCAST exper rules come in hree forms (edied in he ineres of simpliciy): 1. Inrinsic resisance: E.g. Rule 2.6: Pseudomonas aeruginosa is resisan o ampicillin. 2. Excepional resisance phenoypes: E.g. Rule 6.1: Saphylococcus aureus is (almos always) suscepible o vancomycin, linezolid, quinuprisin/dalfoprisin, dapomycin and igecycline. 3. Inerpreive rules: E.g. Rule 8.6: If Enerococcus spp. is resisan o ampicillin, repor as resisan o ureidopenicillins and carbapenems. The firs wo forms exend he ABG wih knowledge of respecively resisance and suscepibiliy for animicrobials, which have no been esed. The hird form exends he suscepibiliy resuls from he esed animicrobials o some of hose no esed. These rules represen an improvemen in he reporing of suscepibiliy resuls, bu cover only a very limied number of pahogen/animicrobial combinaions. I would herefore be desirable o have a compuaionally feasible mehod ha reains he advanages of he EUCAST exper rules, bu which may also provide some help when none of hese apply. This will be achieved by compiling insiuional ABGs from he isolae daabases mainained by mos clinical microbiology laboraories. For example, in he insiuional ABG compiled from he isolae daabase used in his sudy, we can read ha he prior probabiliy of Escherichia coli (E. coli) isolaes being suscepible o cefuroxime is 83%. This informaion indicaes ha prescripion of cefuroxime agains an E. coli infecion may well be useful, even if he E. coli isolae s suscepibiliy o cefuroxime has no been esed. Insiuional cross-abgs, conaining saisics on he condiional probabiliies of suscepibiliy for pairs of animicrobials can likewise be compiled. For example, in he cases where E. coli was suscepible o ofloxacin, we can read from he insiuional cross- ABG ha he condiional probabiliy of suscepibiliy o cefuroxime given suscepibiliy o ofloxacin is 96%. This can be used o prescribe cefuroxime wih good cerainy (96%) ha i will cover he E. coli infecion. This paricular case, where he quinolone, ofloxacin, may be replaced by he cephalosporin, cefuroxime, would be an example of anibioic sewardship, in line wih for example he Danish guidelines on prescribing of anibioics [2], where he use of quinolones is more resriced han he use of cephalosporins. We will refer o he condiional probabiliies as insiuional crosssuscepibiliies, or insiuional cross-resisances, if condiional on resisance. Joinly, he insiuional cross-suscepibiliies and he cross-resisances will be referred o as he insiuional cross- ABGs. The purpose of his paper is o develop a mehod of inerpreaive reading of suscepibiliy es resuls where poserior probabiliies of suscepibiliies of an isolae o unesed animicrobials are calculaed from he insiuional ABG (he priors) and he insiuional cross-abg, given he esed isolae s ABG. 2. Mehods and Maerial 2.1. The isolae daabase An isolae daabase of bacerial pahogens isolaed from blood culures aken from paiens suspeced of bacerial infecions will be used o illusrae he mehods for esimaion of poserior probabiliies of suscepibiliy. The daabase was compiled beween 2002 and 2004 a Rabin Medical Cener in Israel and includes 3347 clinically significan pahogens. The isolae daabase conains he pahogen ideniy and an ABG for each isolae. The suscepibiliy resuls are repored in he socalled S-I-R sysem. If S (sensiive) is repored as he resul of he suscepibiliy es, hen he animicrobial can eradicae he pahogen in viro and wih some excepions his will also lead o clinical success, i.e. in vivo eradicaion of he pahogen. R (resisan) is expeced o resul in clinical failure and I (inermediae) may lead o eiher. For he purposes of his paper inermediae es resuls (I) will be considered as resisan (R). 2.2. Compilaion of insiuional ABGs and insiuional cross-abgs For each of he 14 pahogen groups in he isolae daabase an insiuional ABG was compiled, conaining he (prior) probabiliy of an isolae from a given group being suscepible o a given animicrobial. From he isolae daabase he cross-suscepibiliy and cross-resisance ables can also be compiled, as previously described by Zalounina e al. (2007) [4]. These ables conain condiional probabiliies of suscepibiliy for pairs of animicrobials and hey are called cross-suscepibiliies when condiional on suscepibiliy and cross-resisances, when condiional on resisance. Table IV gives an example of he compilaion of cross-suscepibiliies and cross-resisances. The cross-suscepibiliies and cross-resisances are joined ino he insiuional cross-abg ables where hey are expressed as odds raios for increased or decreased suscepibiliy (Appendix Eq. (8)). Fisher s exac es is used o deermine he significance of he odds raios. When mos isolaes are esed wih he same animicrobials he observaions are dependen, wih missing values. Therefore Fisher s exac es may underesimae he significance of some odds raios. 2.3. Calculaion and validaion of poserior probabiliies of suscepibiliy The poserior probabiliies will be calculaed by several versions of naïve and semi-naïve Bayesian mehods. In he naïve Bayes mehod all significan odds raios in he cross-abg will be used. In he semi-naïve Bayesian mehods only some of he significan odds raios will be used. Each mehod for calculaion of poserior probabiliies is validaed by 5-fold cross-validaion, where he isolae daabase is divided in a learning se and a validaion se. The insiuional ABGs and insiuional cross-abgs compiled from he learning se are used o calculae poserior probabiliies in he validaion se. The qualiy of he calculaed probabiliies will be assessed by deleing one suscepibiliy es resul a a ime for a given isolae in he validaion se and hen using he mehod o calculae he poserior probabiliy of suscepibiliy given he remaining suscepibiliy es resuls for ha paricular isolae. This will be repeaed for each of he es resuls for he given isolae and subsequenly for all isolaes in he 5-fold validaion se. The accuracy of he poserior probabiliies will be assessed by calculaing he disance (Appendix Eq. (11)) beween each es resul in he ABG and is calculaed poserior probabiliy. The normalized Brier disances are hen calculaed by adding all disances and normalizing (Appendix Eq. (12)).
S. Andreassen e al. / Arificial Inelligence in Medicine 65 (2015) 209 217 211 Table 1 A segmen of he isolae daabase. Isolae Pahogen OFL MER AMK 1 Acineobacer spp. R R S 2 E. coli R S R 3 Proeus spp. S NA S S = suscepible, R = resisan, NA = he suscepibiliy is no assessed The normalized Brier disance can be considered as he sandard deviaion in percen beween calculaed and esed suscepibiliies. The normalized Brier disances beween suscepibiliy es resuls and respecively prior probabiliies and poseriori probabiliies of suscepibiliy were compared using he paired samples -es (2-ailed). 3. Resuls 3.1. The isolae daabase The isolae ABGs conained a oal of 63590 suscepibiliy es resuls. A segmen of he isolae daabase is illusraed in Table 1. As an example, suscepibiliy resuls are here shown for hree animicrobials: ofloxacin (OFL), meropenem (MER) and amikacin (AMK), bu he daabase conains on average es resuls for 19 differen animicrobials per pahogen. The ABGs in he isolae daabase were used o compile he insiuional ABG of each pahogen. To avoid esimaes based on very few resuls, he pahogens were placed in 14 groups wih roughly similar suscepibiliies (Table 2). Learning suscepibiliy of a pahogen using daa from similar pahogens was previously described by Andreassen e al. (2009) [5]. The choice of animicrobials used o es suscepibiliies depended on he pahogen. Table 3 illusraes a oal of five differen ypical ses of animicrobials esed on: a) Gram negaive baceria b) Saphylococci c) Srepococci d) Enerococci and e) Oher Gram posiive baceria Table 3 shows a clear paern in he ses of animicrobials seleced for suscepibiliy es agains a pahogen. I reflecs a sraegy of: a) Only esing animicrobials known o have effec on he given pahogen. This is for example why he narrow specrum lacam, mehicillin, is no esed agains Gram negaive baceria. Table 2 Isolaes organized ino pahogen groups. Number of isolaes (%) Pahogen group Tes group 716 (21.4) E. coli Gram pos. 375 (11.2) Klebsiella spp. 266 (7.9) Acineobacer spp. 237 (7.1) Pseudomonas spp. 230 (6.9) Proeus spp. 173 (5.2) Oher Gram neg. 114 (3.4) Enerobacer spp. 454 (13.6) Saph coag posiive Saph spp. 221 (6.6) Saph coag negaive 110 (3.3) Oher srep Srep spp. 92 (2.7) Srep pneumoniae 50 (1.5) Srep viridans 283 (8.5) Enerococcus spp. Enerococcus spp. 26 (0.8) Oher Gram pos. Oher Gram pos. 3347 (100) All b) Using assumpions on cross-resisance and cross-suscepibiliy o minimize he number of ess. For example he choice of esing suscepibiliy o mehicillin agains saphylococci is based on knowledge of 100% cross-resisance o oher -lacam anibioics. 3.2. Insiuional ABGs and insiuional cross-abgs Table 4, column 2 gives an example of how he enry for he animicrobial cefuroxime in he insiuional ABG for E. coli is compiled. The column is labeled Prior conains he number of E. coli isolaes esed agains cefuroxime, N CEF = 705; N cef = 582 is he number of isolaes esed suscepible o cefuroxime; and P(cef) = 0.83 is he prior probabiliy of E. coli being suscepible o cefuroxime. In Table 4, column 3 labeled Cross-suscep. N CEF ofl = 522 is he number of isolaes esed agains cefuroxime ha concurrenly showed suscepibiliy o ofloxacin; N cef ofl = 501 is he number of isolaes esed suscepible o cefuroxime and concurrenly suscepible o ofloxacin, and P(cef ofl) = 0.96 is hen he condiional probabiliy of suscepibiliy o cefuroxime, given ha he isolae is suscepible o ofloxacin. The column labeled Cross-resisance shows he corresponding couns and probabiliies condiional on resisance o ofloxacin. The change from he prior probabiliy of 83% o he poserior probabiliy of 96%, condiional on suscepibiliy o ofloxacin, corresponds o an odds raio OR(cef ofl) = 5.0 (Eq. (9) in he Appendix) and he change o 44% condiional on resisance corresponds o OR( cef ofl) = 0.16. These wo odds raios are enered ino he cross- ABG able for E. coli. In Table 5, as an example we show he insiuional ABG for E. coli in he column labelled Prior. The columns labeled ORsus and ORres give he odds raios in he insiuional E. coli cross-abg for cefuroxime, condiional on suscepibiliy o he animicrobials lised in he rows. For example, he odds raios menioned above of 5.0 and 0.16 can be found in he Ofloxacin row. All odds raios for he ypically esed animicrobials in he E. coli example in Table 5 are significan (wih p < 0.1), excep he carbapenems and colisin. Carbapenems and colisin have prior suscepibiliies close o 100%, which implies ha resisance and hereby cross-resisance is very rarely or never observed (since he pahogen is almos always suscepible). I is also difficul o observe significan cross-suscepibiliy o hese animicrobials and also irrelevan since we already know ha he pahogen is suscepible. The odds raio condiional on suscepibiliy o cephalohin is infiniy, which implies ha suscepibiliy o cephalohin, which is a firs generaion cephalosporin guaranees suscepibiliy o cefuroxime, a second generaion cephalosporin. The odds raios, condiional o resisance o cefazidime and cefriaxone, boh hird generaion cephalosporins, and o cefepime, a fourh generaion cephalosporin, is zero or very close o zero. This means ha resisance o one of hese hree cephalosporins implies resisance o cefuroxime as well. The columns labeled ORsus and ORres in Table 5 shows only a small par of he cross-abg for E. coli. The complee insiuional ABG for E. coli conains Table 5, bu also ORsus and ORres columns for all oher animicrobials besides cefuroxime. Such daa are compiled for each of he 14 pahogen groups. The EUCAST inerpreaive rules can be enered direcly ino ables of insiuional cross-abgs, as defined in he mehods secion. For he 14 groups of pahogens and 31 animicrobials in he isolae daabase he cross-abg ables have 14*31*31*2 = 26908 enries. The inerpreaive EUCAST rules only allow 70 or 0.26% of hese enries o be filled. In comparison he saisical compilaion from he isolae daabase of enries inro he insiuional cross-abg ables allow 4682 or 17.4% of he enries o be filled.
212 S. Andreassen e al. / Arificial Inelligence in Medicine 65 (2015) 209 217 Table 3 ABG ses used for pahogen es groups. Anibioic Class Anibioic Gram neg. Saph spp. Srep spp. Enerococcus spp. Oher Gram pos. Penicillin Cephalosporin Carbapenem Penicillin x x x x Ampicillin x x x x Ampicillin/Sulbacam x Amoxicillin/Clavulanae x Piperacillin x Piperacillin/Tazobacam x Mehicillin x Cephalohin x x a x Cefuroxime x x a x Cefazidime x x a x Cefriaxone x x a x Cefepime x x x Erapenem b Imipenem a a a a Meropenem x x x x Monobacam Azreonam x Glycopepide Vancomycin x x x x Macrolide Eryhromycin x x x x Teracycline Aminoglycoside Quinolone Minocycline x a Teracycline x x x x x Amikacin x a x a x Genamicin x x x a x Tobramycin x Ciprofloxacin x a a a a Ofloxacin x x a a a Oher Chloramphenicol x x x x Clindamycin x x x x Colisin x Fusidic acid x x x x Rifampicin x x x x Sulfa-Trim x x x a x a: This es was phased ou during he sudy period. b: This es was phased in during he sudy period. The insiuional ABG and he odds raios in he insiuional cross-abg can be used o calculae he poserior probabiliy of suscepibiliy o an unesed animicrobial, given a suscepibiliy es resul for anoher animicrobial. If we consider a siuaion where suscepibiliy o cefuroxime of an E. coli isolae has no been esed, and ha he E. coli isolae is esed suscepible o ofloxacin, hen he poserior probabiliy of suscepibiliy cefuroxime can be calculaed. This is done by convering he prior probabiliy of suscepibiliy o cefuroxime ino odds (OD) for suscepibiliy: OD(cef) = 0.83/(1 0.83) = 4.88 (1) in accordance wih Eq. (6) in he Appendix. The poserior odds for cefuroxime, given suscepibiliy o ofloxacin, is hen calculaed as: OD(cef ofl) = OR(cef ofl) OD(cef) = 5.0 4.88 = 24.41 (2) in accordance wih Eq. (9) in he Appendix. Finally, he odds are convered back o a probabiliy: P(cef ofl) = OD(cef ofl)/(1 + OD(cef ofl)) = 24.41/(1 + 24.41) = 0.96 (3) which can be recognized as he already known condiional probabiliy from Table 4. 3.3. Mehods for calculaion of poserior probabiliies of suscepibiliy from he ABG In he ABG presened in Table 6, he suscepibiliy o cefuroxime was esed and he isolae was found o be suscepible o cefuroxime. If we again imagine ha he suscepibiliy o cefuroxime had no been esed, hen we can ask if we could have prediced he in viro suscepibiliy o cefuroxime from he oher 20 es resuls. 3.3.1. Calculaion from Bayes heorem The sraighforward soluion is o compile he 20 dimensional probabiliy marix for cefuroxime, condiional on he oher 20 resuls in he ABG. Bayes heorem can hen be used direcly o calculae he poserior probabiliy of suscepibiliy o cefuroxime. This is far from a viable soluion, because i would be impossible o populae he 2 20 elemens in his marix even wih an isolae daabase compiled over many years in a large hospial. Table 4 Suscepibiliy of E. coli o cefuroxime given knowledge of suscepibiliy o ofloxacin. Pahogen: E. coli Prior (cefuroxime) Cefuroxime condiional on ofloxacin Cross-suscepibiliy Cross-resisance Number of isolaes N CEF N cef N CEF ofl N cef ofl N CEF ofl N cef ofl 705 582 522 501 181 79 Suscepibiliy P(cef) 582/705 = 0.83 P(cef ofl) 501/522 = 0.96 P(cef ofl) 79/181 = 0.44
S. Andreassen e al. / Arificial Inelligence in Medicine 65 (2015) 209 217 213 Table 5 Insiuional ABG for E. coli, odds raios for E. coli being suscepible o cefuroxime given suscepibiliy es resuls o oher animicrobials and example of an isolae ABG. Anibioic class Anibioic Prior sus. (%) Cefuroxime Isolae ABG ORsus p 0.1 ORres p 0.1 Penicillins Cephalosporins Carbapenems Penicillin Ampicillin 31 23 0.63 R Ampicillin/Sulbacam 53 6.9 0.40 Amoxicillin/Clavulanae 64 5.1 0.30 R Piperacillin 41 4.5 0.59 R Piperacillin/Tazobacam 89 1.6 0.11 S Mehicillin Cephalohin 33 0.60 R Cefuroxime 83 0 S Cefazidime 91 3.4 0.00 S Cefriaxone 86 4.5 0 S Cefepime 90 2.9 0 S Erapenem 100.0 NS NS Imipenem 99.8 NS NS S Meropenem 99.3 NS NS S Monobacams Azreonam 86 3.4 0.01 S Glycopepides Macrolides Teracyclines Aminoglycosides Quinolones Vancomycin Eryhromycin Minocycline 54 2.2 0.55 S Teracycline 45 2.4 0.63 S Amikacin 89 1.8 0.09 S Genamicin 84 2.0 0.14 S Tobramycin 81 3.7 0.10 S Ciprofloxacin 73 5.0 0.18 S Ofloxacin 74 5.0 0.16 S Oher Chloramphenicol Clindamycin Colisin 98.9 NS NS S Fusidic acid Rifampicin Sulfa-Trim 59 2.0 0.52 S NS: Odds raio no significan. Blank field: Suscepibiliy o his animicrobial is no esed. 3.3.2. Calculaion from naïve Bayes If a naïve Bayes approach is used, hen he poserior probabiliy can be calculaed from he daa available in he suscepibiliy daabase, as shown in he Appendix. This is done by repeaing he procedure given by Eqs. (1) (3), excep ha he single odds raio in Eq. (2) is replaced by ORall, which is he produc of all he bolded odds raios in Table 6. When he es resul is S, we include he OR for cross-suscepibiliy and when he suscepibiliy resul is R, we include he OR for crossresisance. This resuls in he odds raio produc; ORall = 7432 and he poserior probabiliy P(cef ABG) = 100,00%. This resul indicaes a close o perfec predicion of he suscepibiliy o cefuroxime. This aemp a esimaing he poserior probabiliies is repeaed for he oher esed animicrobials, imagining one a a ime ha he es resuls are no available. The resuls are shown in Table 6 in he column labeled Naïve. For mos of he animicrobials he resuls of he suscepibiliy-esimaion are as anicipaed, bu wih a few excepions, noably he suscepibiliy o ampicillin/clavulanae, which is esimaed o 99%, even hough he resul was R (0%). A normalized Brier disance (Appendix Eq. (12)) is used as a measure of he qualiy of he poserior esimaes. This disance can be inerpreed as an average disance beween he poserior probabiliy and he acual es resul. The naïve Bayes esimaes gives a normalized Brier disance of 25.2% for he E. coli isolae considered (las row, Table 6). This can be considered as an improvemen, relaive o he normalized Brier disance for he prior esimaes, which is 29.1%. 3.3.3. Calculaion from semi-naïve Bayes The naïve Bayes assumpion is ha he suscepibiliy es resuls are muually independen. The naïve Bayes approach is known o produce overconfiden resuls, i.e. resuls oo close o 0% or 100%, if he underlying assumpion of independence is no well me, and he naïve Bayes esimaes in Table 6 could indicae ha his is he case. To manage his, a semi-naïve approach is explored. In his approach a more modes number of odds raios are used in he calculaions. We define a version of semi-naïve Bayes, called min1max1, where only he larges and he smalles odds raios from Table 5 are used o calculae he odds raio in Eq. (2). Thus, using he odds raios shaded gray in Table 5 we ge: ORmin1 max 1(cef ABG) = 5.0 0.3 = 0.15 From his odds raio follows ha he poserior probabiliy of suscepibiliy o cefuroxime of 88%, which is a less exreme esimae han he esimae of 100% from he naïve Bayes mehod. The normalized Brier disance for his E. coli isolae over all animicrobials is down o 18%, which suppors he suspicion ha he naïve Bayes resuls may be overconfiden, and ha he more modes seminaïve approach may be beer han he naïve Bayes approach. In he following secion, we compare he performance of he naïve and differen versions of semi-naïve mehods. 3.4. Validaion of he naïve and semi-naïve Bayesian mehods To assess he qualiy of he naïve and semi-naïve poserior probabiliies of suscepibiliy, one suscepibiliy resul a a ime is
214 S. Andreassen e al. / Arificial Inelligence in Medicine 65 (2015) 209 217 Table 6 Anibiogram, Prior, and poserior probabiliy of suscepibiliy of an E. coli isolae. Anibioic class Anibioic Isolae ABG Probabiliy of suscepibiliy (%) Prior Naïve min1max1 Penicillins Cephalosporins Carbapenems Penicillin Ampicillin R 31 0 0 Ampicillin/Sulbacam 53 Amoxicillin/Clavulanae R 64 99 57 Piperacillin R 41 47 11 Piperacillin/Tazobacam S 89 100 88 Mehicillin Cephalohin R 33 30 9 Cefuroxime S 83 100 88 Cefazidime S 91 100 99 Cefriaxone S 86 100 100 Cefepime S 90 100 100 Erapenem 100.0 Imipenem S 99.8 Meropenem S 99.3 Monobacams Azreonam S 86 100 98 Glycopepides Macrolides Teracyclines Aminoglycosides Quinolones Oher Vancomycin Eryhromycin Minocycline S 54 94 87 Teracycline S 45 83 65 Amikacin S 89 100 94 Genamicin S 84 100 92 Tobramycin S 81 100 89 Ciprofloxacin S 73 100 98 Ofloxacin S 74 100 100 Chloramphenicol Clindamycin Colisin S 98.9 Fusidic acid Rifampicin Sulfa-Trim S 59 87 61 Norm. Brier disance (%) 29.1 25.2 18.0 Blank field: Suscepibiliy o his animicrobial is no esed. deleed from he isolae ABG and subsequenly he poserior probabiliy of suscepibiliy is calculaed from he remaining ABG resuls. This is repeaed for all he 3347 isolaes in he daabase using a 5-fold cross-validaion. For each isolae he normalized Brier disance is used o measure he disance beween suscepibiliy es resuls and respecively prior probabiliies and calculaed poserior probabiliies. Table 7 shows hese normalized Brier disances, averaged over all isolaes in he five validaion ses. The firs row presens he normalized Brier disance for prior probabiliies and he following rows presens he normalized Brier disance for differen naïve or semi-naïve Bayesian mehods. The mehods are named afer he number of odds raios used o calculae poserior probabiliies of suscepibiliy. For example, he mehod named min2max2 uses he wo smalles significan odds raios and he wo larges significan odds raios, and he min0max3 mehod includes zero odds raios on cross-resisance and he hree highes significan odds raios on cross-suscepibiliy (if hey exis). The mehods where he odds raios for cross-resisance and cross-suscepibiliy were weighed equally (shaded gray in Table 7) performed beer han boh he naïve mehod and mehods wih an unequal weighing. For example, he min0max3 mehod has a normalized Brier disance, which is even larger han he normalized Brier disance of he prior probabiliies (firs row). The min2max2 mehod performed he bes wih a normalized Brier disance of 25.3% (Ranges from 24.8% 25.6% wihin he five validaions ses). The min2max2 mehod gave a significan improvemen (p < 10 99 ) when compared o he normalized Brier disance for he prior Table 7 Normalized Brier disances for poserior esimaes of suscepibiliy including differen numbers of odds raios. Number of ORs Mehod Norm. Brier disance (%) Range 0 prior 37.7 37.3-38.4 1 2 3 min0max1 35.3 34.8-35.6 min1max0 36.7 36.1-37.5 min0max2 37.2 37.1-37.4 min1max1 25.6 25.3-26.0 min2max0 40.2 39.5-40.8 min0max3 38.2 38.0-38.4 min1max2 26.0 25.7-26.1 min2max1 26.7 26.2-27.2 min3max0 42.1 41.3-42.9 min0max4 38.6 38.4-38.9 min1max3 27.0 26.6-27.3 4 min2max2 25.3 24.8-25.6 min3max1 28.5 28.1-28.9 min4max0 43.0 42.3-43.9 6 min3max3 26.0 25.7-26.4 8 min4max4 26.7 26.4-27.1 All naïve 28.2 27.8-28.5 probabiliies, and he disance was significanly smaller (p < 10 9 ) han he disance for he min1max1 mehod. Fig. 1 shows he performance of he min2max2 mehod on each pahogen group compared o he prior suscepibiliies from he
S. Andreassen e al. / Arificial Inelligence in Medicine 65 (2015) 209 217 215 Normalized Brier diaance 50% 40% 30% 20% 10% 0% Normalized Brier disance 40% 35% 30% 25% prior min2max2 naive prior min2max2 20% 0 0.1 0.2 0.3 0.4 0.5 p value Fig. 1. Normalized Brier disances for each pahogen group for respecively prior suscepibiliies and he min2max2 mehod. insiuional ABG. Normalized Brier disances for each pahogen group were averaged over he five validaion ses. For Gram negaive baceria he min2max2 mehod gave an average reducion in normalized Brier disance of 13.3% from he prior suscepibiliies (from 37.3% o 23.9%). For Saphylococcus spp. he mehod reduced he disance by 12.0% (from 40.6% o 28.6%), and for he res of he Gram posiive baceria he average reducion was 3.5% (from 33.0% o 29.5%). In all he above calculaions, he poserior probabiliies were calculaed from odds raios, which are significanly differen from 1. The limi of significance was arbirarily chosen as p < 0.1. Resuls will depend on he p value chosen and herefore he poserior suscepibiliy deerminaions were carried ou wih differen p values, ranging from 0.02 o 0.5. Fig. 2 shows he mean of esimaed suscepibiliies for he five validaions ses, and he range he ses varied wihin (shaded grey). The abiliy o esimae suscepibiliies was esed using 10 differen p values (from 0.02 o 0.5) for inclusion of significan odds raios. For mos of he 63590 included suscepibiliy es resuls, poserior probabiliies of suscepibiliy could be calculaed from he oher suscepibiliy es resuls. Wih a p value of 0.02 for significance of odds raios, 71.9% of he suscepibiliies could be calculaed, and wih a p value of 0.5, 89.7% of he suscepibiliies could be calculaed. Fig. 3 illusraes ha irrespecive of p value he naïve Bayes mehod had lower normalized Brier disance han he prior probabiliies and ha he min2max2 mehod had lower normalized Brier disances han he naïve Bayes mehod. The normalized Brier disance of he naïve Bayes mehod ended o increase a lile wih he p value while he normalized Brier disance of he min2max2 mehods ended o decrease a lile wih he p value. The decrease of he normalized Brier Esimaed suscepibiliies 100% 90% 80% 70% Mean esimaed suscepibiliies 60% 0.02 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.5 p value Fig. 2. Percen of he 63590 suscepibiliy resuls included in he five validaion ses for which poserior probabiliies could be calculaed by he min2max2 mehod. Fig. 3. Normalized Brier disances averaged over he five validaion ses when using respecively he prior suscepibiliies, he naïve Bayes mehod, and he min2max2 mehod o calculae probabiliies of suscepibiliy. disance of he min2max2 mehod wih he p value could be seen as an argumen for using a high p value. This should however be balanced agains he risk of obaining srange esimaes due o spurious odds raios derived from very few suscepibiliy resuls. Since he reducion in normalized Brier disance was very small for p values above 0.1, p = 0.1 may be an appropriae choice. 4. Discussion This paper explores how inerpreaive reading of he isolae ABG can be used o replace he prior probabiliies sored in he insiuional ABG, wih poserior probabiliies, which are a leas closer o suscepibiliy es resuls han he prior probabiliies. We explored boh a naïve and semi-naïve Bayesian approaches, where he poserior probabiliies were calculaed from he isolae ABG, using daa sored in he insiuional ABG and he insiuional cross-abg. A 5-fold cross-validaion showed ha he goal was achieved. The normalized Brier disance beween he prior probabiliies and he suscepibiliy es resuls for all isolaes in he daabase was reduced from 37.7% o 28.2% by he naïve Bayes mehod. The smalles normalized Brier disance of 25.3% was obained wih he semi-naïve min2max2 mehod. This illusraes ha alhough he naïve Bayes mehod acually gave an improvemen relaive o he prior probabiliies, he naïve Bayes assumpion should for his purpose be used wih resrain. The reason for a more poorly performance for mehods including more han wo odds raios (min3max3, min4max4, and naïve) is probably relaed o he naïve Bayes assumpion; ha he suscepibiliy es resuls are muually independen. The naïve Bayes approach is known o produce overconfiden resuls, i.e. resuls oo close o 0% or 100%, if he underlying assumpion of independence is no well me. The E. coli example in Table 6 indicaes ha his is he case. No surprisingly, he mehods where he odds raios for crossresisance and cross-suscepibiliy were weighed equally (shaded gray in Table 7) performed beer han boh he naïve mehod and mehods wih an unequal weighing. In oher words, giving a higher weigh o eiher cross-resisance or cross-suscepibiliy gives a disorion in he calculaed suscepibiliies. When we looked ino he performance of he min2max2 mehod on differen pahogen groups, he resuls indicaed a beer performance on Gram negaive isolaes compared o Gram posiive isolaes. This is could be relaed o a higher fracion of he cross- ABG being filled for Gram negaive pahogens, i.e. a higher fracion of significan cross-resisances and cross-suscepibiliies. Furher analysis could be done o invesigae his correlaion and he variabiliy in he fracion of he cross-abg being filled.
216 S. Andreassen e al. / Arificial Inelligence in Medicine 65 (2015) 209 217 The proposed mehod can be seen as an exension of he EUCAST inerpreive exper rules because: a) The EUCAST rules are very scarce, covering only a very small par (0.26% in his sudy) of he large marix of pahogens and pairs of animicrobials. The naïve or semi-naïve Bayesian mehods cover a much larger fracion of his marix (17.4% in his sudy). b) In conras o EUCAST he proposed mehod provides an esimae of he probabiliy of suscepibiliy, which is relevan since animicrobials may be given, even when he probabiliy of suscepibiliy is smaller han 100%. c) The proposed mehod is based on saisics, local o he hospial, and will herefore adap o local microbial suscepibiliies, while EUCAST rules are based on inernaional clinical and/or microbiological evidence of varying qualiy [3]. d) EUCAST inerpreaive exper rules gives advice on anibioic suscepibiliy o anibioics in he same anibioic class as he ones esed (e.g wihin he -lacam group) [3]. The proposed mehod works across anibioic classes, as in he example wih cefuroxime and ofloxacin. This means ha he proposed mehod and he inerpreive EUCAST exper rules should be used in combinaion o ge he bes possible predicion of suscepibiliy of a pahogen. The EUCAST rules can be incorporaed as an inegraed par of he insiuional ABG. The min2max2 mehod will in combinaion wih he EUCAST rules be incorporaed ino he TREAT sysem (a compuerized decision suppor sysem for animicrobial sewardship), which have been shown o reduce inappropriae animicrobial reamens a hospials [1,6]. Inerpreaive reading of suscepibiliy resuls is expeced o furher improve performance of he sysem. A limiaion of he sudy is ha he mehods developed are approximae mehods. As poined ou already, an exac Bayesian approach would require populaion of mulidimensional condiional probabiliy marices. This is far beyond reach and in pracice he one-dimensional condiional probabiliies in he cross- ABG represens a pracical upper limi. I may herefore be considered a virue, ha useful poserior probabiliies can be calculaed from he limied informaion compiled in he suscepibiliy daabase. I may also be considered a limiaion ha only probabilisic mehods have been considered. Esimaing suscepibiliies could for example be considered as an impuaion problem, where he missing suscepibiliies should be impued from he available suscepibiliies [7]. Alernaively, classifiers could be build based on neural nes, fuzzy logic or rule inducion. We suspec ha he combinaorial explosion, which prevened a direc applicaion of Bayes heorem, will also presen problems for hese oher mehods, and we have herefore refrained from using hem. The isolae daabase used in his paper provided a raher high number of suscepibiliy resuls - on average each isolae was esed wih 19 animicrobials. Furher validaion on isolaes wih sparser suscepibiliy es resuls would help o srenghen he resuls. In general, he abiliy of he mehod o predic he suscepibiliy o an animicrobial depends on he qualiy of he relevan isolae ABG and he qualiy of he insiuional ABG and cross- ABG. To conclude, a pracical mehod for predicing probabiliy for animicrobial suscepibiliy could be developed based on a semi-naïve Bayesian approach using saisical daa on crosssuscepibiliies and cross-resisances. By using his mehod, we achieved significanly more accurae predicions of pahogen in viro suscepibiliy o animicrobials, han he prior probabiliies sored in he insiuional ABG. Conflic of ineres The auhors Seen Andreassen and Line Sanden are employed par ime a Trea Sysems, a company developing he animicrobial decision suppor sysem TREAT. Trea Sysems did no financially suppor he work. Appendix A. A.1. Calculaion of poserior probabiliy of suscepibiliy from an anibiogram We assume ha a given pahogen has an anibiogram presening a se of T animicrobials: T = {1,...,... T} and a corresponding se of suscepibiliy resuls: A T = {A 1,... A,... A T }, where A is eiher S (a suscepible resul) or R (a resisan resul). The prior probabiliy of suscepibiliy o an animicrobial B, which is no a member of he se of T animicrobials, is denoed as P(b). The corresponding probabiliy of resisance o B is denoed as P( b). The aim is o calculae he poserior probabiliy of suscepibiliy o animicrobial B given a se of suscepibiliy resuls A T, denoed as P(b A T ). A sep owards a simple approximaion of P(b A T ) can be aken by using Bayes heorem o wrie: P(b A T ) = P(A T b)p(b) (1) P(A T ) If we make he naïve Bayes assumpion of condiional independence wihin he se A T, given b, hen P(A T b) = P(A T b) (2) A Bayes inversion of P(A b) gives: P(A T b) = P(b A )P(A ) P(b) Inserion of Eqs. (2) and (3) in (1) gives: [ ] P(b AT P(b A T ) = P(A )P(A T b)p(b) ) P(b) P(b) = P(A T ) P(A T ) [ P(b AT )P(A P(b)] ) [P(A T )]P(b) = P(A T ) We now define he prior odds OD b for P(b) as: OR b = P(b)/P( b) (5) and define he poserior odds for P(b A T ) as: OD b AT = P(b A T )/P( b A T ) (6) Insering Eqs. (4) and (5) ino (6) gives: [ P(b AT P(b)] ) [P(A T )]P(b) OD b AT = [ P( b AT P(b)] ) [P(A T )]P( b) [ ] P(b AT ) P(b) P(b) = [ ] P( b AT ) = OD b P( b) P( b) The odds raio for suscepibiliy is defined as: (3) (4) OD b A OD b (7) OR b A = OD b A /OD b (8)
S. Andreassen e al. / Arificial Inelligence in Medicine 65 (2015) 209 217 217 Hence eq.7 can be wrien as: OD b AT = OD b OR b A (9) Finally he poserior probabiliy can be calculaed from Eq. (6) as: P(b A T ) = OD b AT /(1 + OD b AT ) (10) using ha P( b A T ) = 1- P(b A T ). A.2. Evaluaion of he accuracy of poserior probabiliies of suscepibiliy The abiliy of a mehod o accuraely calculae poserior probabiliies of suscepibiliy was esed by removing one suscepibiliy resul A a a ime from he suscepibiliy resuls A T for a given isolae in he isolae daabase, leaving he se A T \A. Then we use he mehods o calculae P(A A T \A ) and calculae he disance o he rue resul A as he Euclidian disance: Dis = A P(A A T \A ) (11) where A has he value 1 (suscepible) or 0 (resisan). The qualiy of he poserior probabiliies provided by a given mehod was hen assessed from he normalized Brier disance [8,9] across all animicrobials A T for which a suscepibiliy resul was available: A AT Dis2 Norm.Brier dis. = (12) N AT where N AT was he number of members in he se A T. The normalized Brier disance can be inerpreed as a deviaion in percen beween he calculaed poserior probabiliy of suscepibiliy and he measured suscepibiliy es resul. References [1] Paul M, Andreassen S, Tacconelli E, Nielsen AD, Almanasreh N, Frank U, Cauda R, Leibovici L. Improving empirical anibioic reamen using TREAT, a compuerized decision suppor sysem: cluser randomized rial. Journal of Animicrobial Chemoherapy [Online] 2006;58(6):1238 45. Available: hp://dx.doi.org/10.1093/jac/dkl372. [2] Danish Healh Medicines Auhoriy. Guidelines on Prescribing Anibioics [Online] 2013. Available: hp://sundhedssyrelsen.dk/publ/publ2013/11nov/ AnibioPrescribDK En.pdf. (Accessed: 8 July 2015). [3] European Commiee on Animicrobial Suscepibiliy Tesing (EUCAST). EUCAST Exper rules 2011 [Online]. Available: hp://www.eucas.org/exper rules/. (Accessed: 8 July 2015). [4] Zalounina A, Paul M, Leibovici L, Andreassen S. A sochasic model of suscepibiliy o anibioic herapy -The effecs of cross-resisance and reamen hisory. Arificial Inelligence in Medicine [Online] 2007;40(1):57 63. Available: hp://dx.doi.org/10.1016/j.armed.2006.12.007. [5] Andreassen S, Zalounina A, Leibovici L, Paul M. Learning suscepibiliy of a pahogen using daa from similar pahogens. Mehods of Informaion in Medicine 2009;48(3):242 7. Available: hp://dx.doi.org/10.3414/ ME9226. [6] Kofoed K, Zalounina A, Andersen O, Lisby G, Paul M, Leibovici L, Andreassen S. Performance of he TREAT decision suppor sysem in an environmen wih a low prevalence of resisan pahogens. Journal of Animicrobial Chemoherapy [Online] 2009;63(2):400 4. Available: hp://dx.doi.org/10.1093/jac/ dkn504. [7] Sinharay S, Sern H, Russell D. The use of muliple impuaion for he analysis of missing daa. Psychological Mehods [Online] 2001;6(4):317 29. Available: hp://dx.doi.org/10.1037/1082-989x.6.4.317. [8] Brier GW. Verificaion of forecass expressed in erms of probabiliy. Monhly Weaher Review [Online] 1950;78(1):1 3. Available: hp://dx.doi.org/ 10.1175/1520-0493(1950)078<0001:VOFEIT>2.0.CO;2. [9] Gneiing T, Rafery AE. Sricly proper scoring rules, predicion, and esimaion. Journal of he American Saisical Associaion [Online] 2007;102(477):359 78. Available: hp://dx.doi.org/10.1198/016214506000001437.