EVALUATION OF A PEDIATRIC ANTIMICROBIAL STEWARDSHIP PROGRAM IN A TERTIARY CARE MEDICAL CENTER

Transcription

1 EVALUATION OF A PEDIATRIC ANTIMICROBIAL STEWARDSHIP PROGRAM IN A TERTIARY CARE MEDICAL CENTER By Chou-Cheng Lai A dissertation submitted to Johns Hopkins University in conformity with the requirements for the degree of Doctor of Philosophy Baltimore, Maryland January, 2014

2 ABSTRACT Background: The problem of antibiotic resistance is increasing globally. The inappropriate use of antibiotics has been linked to the emergence of antibiotic resistance and other adverse effects. Antimicrobial stewardship programs (ASPs) have been developed to improve antibiotic use, with the goals of maintaining the effectiveness of current antimicrobials and improving patient safety and outcomes. There are several methods by which the use of antimicrobials can be intervened upon by ASPs; most fall into two basic categories: restriction of antimicrobial before they are dispensed initially, often called prior approval and review and feedback regarding antimicrobial use sometime after prescription, often called post-prescription review. Relatively few studies evaluating either approach have been conducted in pediatric settings. This study aims to assess if a prior-approval program combined with post-prescription review program decreases antimicrobial use, reduces the proportion of inappropriate antimicrobial course and is associated with a higher compliance rate with following recommendations compared to a prior-approval program alone among pediatric inpatients. Additionally, the study aims to determine the frequency and risk factors of inaccurate requests submitted in a pediatric web-based prior-approval program. Methods: We conducted a prospective, randomized controlled study at the Johns Hopkins Children Center a 180 bed tertiary pediatric center from September 2011 to November Patients in 4 general pediatric floors who were assessed by ASP team to be receiving inappropriate antibiotics after being on therapy within hours were randomized to ii

3 either receive the intervention (a phone call with the recommendations by ASP team to the treating physician) or no additional feedback. Patients who were cystic fibrosis patients, in oncology-hematology, ICU and patients for whom ID consult had been obtained were excluded from the study. Data collected included days of antibiotic therapy, the proportion of inappropriate antimicrobial course, the acceptance rate of ASP recommendations and some patient s outcome ( such as inconsistence between the antimicrobial susceptibilities of any recovered organism and the recommended alternative therapy, any subsequent infection after ASP's recommendation of stopping therapy) at follow up between the two groups. Wilcoxon rank-sum test was taken to compare measures of antibiotic use. Chisquare test was used to compare the proportion of inappropriate antimicrobial course and the acceptance rate of ASP recommendations. In addition, a retrospective review of patients whose providers ordered antimicrobial using the web-based prior-approval program was carried out from December 2011 to March 2012 for 4 months to determine the frequency of inaccurate information contained within the requests. Multivariate logistic regression was performed to evaluate potential risk factors of inaccurate information in the prior-approval program. Results: The pediatric ASP team identified 60 pediatric patients (30 patients in the intervention group and 30 patients in the control group) for whom use of restricted antimicrobials was inappropriate. There were no significant differences of the amount of restricted antimicrobial use between the intervention group and the control group (median DOTs: 750 vs , p=0.932; median duration of antimicrobial agent per episode of infection (days): 3.5 vs. 5, p=0.094). In the comparison of total antimicrobial use, differences were also not iii

4 significant. However, the prevalence of inappropriate antimicrobial use at follow up was significantly lower in the intervention group than the control group (34.4% vs. 75.8%, p=0.001). The acceptance rate was significantly higher in the intervention group (the treating physician accepted the recommendation) than in the control group (the treating physician auto-corrected antibiotic use so that it was the same as what would have recommended by the ASP team) (67.6% vs. 22.9%, p<0.001). In the retrospective study reviewing prior-approval requests, the result showed that inaccuracy (discrepancies between requests and medical records) occurred in 101 out of 1159 (8.7%) requests. Patients on the surgical service, in the ICU unit, not on oncology service and with prophylaxis as an indication for receiving their antimicrobials were significantly more likely to have inaccurate antimicrobial requests in multivariate logistic regression analysis (p=0.011, p=0.043, p=0.036, p=0.044, respectively). Inaccurate information in the prior-approval requests could potentially affect the decisions of the pediatric ID fellow s approval in about 45% (45 out of 101) of inaccurate requests. Conclusions: Our study demonstrates that a post-prescription review program can successfully decrease the number of inappropriate antimicrobial courses at our institution. These findings might encourage other pediatric centers to pursue similar post-prescription review programs. Although inaccurate information occurred not very frequently among all pediatric prior approval requests, nearly half of them could have influenced pediatric ID fellows decisionmaking regarding approval of the antimicrobial. Targeted review of requests for specific antimicrobials, or for specific patient populations is warranted. iv

5 COMMITTEE OF FINAL THESIS READERS Kenrad Nelson, M.D. Professor and Chair Department of Epidemiology Ruth Karron, M.D. Professor and Dissertation Advisor Department of International Health Sara Cosgrove, M.D., M.S. Associate Professor Department of Medicine, School of Medicine Lawrence H. Moulton, Ph.D. Professor Department of International Health ALTERNATE THESIS READERS William Moss, M.D., M.P.H. Professor Department of Epidemiology Andrea Ruff, M.D. Associate Professor Department of International Health Aaron Milstone, M.D., M.H.S. Assistant Professor Department of Pediatrics, School of Medicine v

6 ACKNOWLEDGEMENTS First and foremost, I would like to express my deepest gratitude to my advisor, Dr. Ruth Karron. I am really fortunate to have such a great advisor. Her continuous guidance, encouragement, patience and immense knowledge always helped me through a lot of obstacles from the beginning of exploring the research topic to every stage of writing the thesis. I cannot thank Dr. Sara Cosgrove enough. Dr. Cosgrove led the intervention project and tutored me in antibiotic stewardship, providing numerous suggestions and answers to my questions about the antimicrobial stewardship program. I would also like to thank Dr. Larry Moulton for his insightful comments and criticisms at different stages of my research. I also truly appreciate the efforts of Dr. Tamma and Dr. Jehn-Hsu, who despite large workloads of their own, completed the post-prescription reviews, helped me collect the data and provided lots of support during the whole process. I am very grateful to my parents, my wife and daughter for their unlimited love, support, and encouragement to pursue my academic career and my good friends Wei-Ju, Yea-Jen, Hsin-Jen, Yi-Fang and Tsung for their continuous support. It is a privilege and wonderful journey to be in the Johns Hopkins University. I will never forget this experience and I could not have finished my thesis without the contributions of so many people in my life. vi

7 Table of Contents ABSTRACT... ii ACKNOWLEDGEMENTS... vi TABLES OF ABBREVIATIONS... x LIST OF TABLES... xi LIST OF FIGURES...xiii 1. Background The Development of Antimicrobial Stewardship Programs (ASPs) The Emergence of Antibiotic Resistance The Significance of Antibiotic Resistance and Other Adverse Outcomes Antibiotic Use and Antibiotic Resistance Adverse Effects of Inappropriate Antibiotic Use The Development of Antimicrobial Stewardship Programs Overview of Antimicrobial Stewardship Programs Active Strategies Supplemental Strategies Current Status of Antimicrobial Stewardship Programs The Influence of Effective Antimicrobial Stewardship Programs Prior Approval Programs Advantage of Prior Approval Programs Limitations of Prior Approval Programs Post-prescription Review Programs Advantages of Post-prescription Review Programs Limitations of Post-prescription Review Programs Pediatric Antimicrobial Stewardship Programs Difference Between Pediatric Patients and Adult Patients Antimicrobial Use in Children in the United States Review of Pediatric ASP studies Pediatric Prior Approval Programs in the Johns Hopkins Children s Center Rationale for this study Hypothesis and Specific Aims vii

8 Specific Aims Methods Methods for Aim 1 and Aim Study Design Outcome Measures Statistical Analysis Methods for Research Aim Study Design Statistical Analysis Results Results for Aim 1 and Demographic Data Comparison of antibiotic use (restricted and total) in the two study arms Reasons for inappropriate antimicrobial use in two groups Proportion of inappropriate antibiotic use on Days 2 and 3 after ASP team review Potential factors associated with inappropriate antimicrobial courses at Day 2 after the ASP team s review Rate of Compliance with the Recommendation at Day 2 and Day 3 after the ASP team s review Outcomes of patients when ASP team recommended alternative empiric therapy or stopping therapy in two arms Results for aim Demographic data Types of inaccurate requests and examples Potential Factors Related to Inaccuracy of Antimicrobial Requests Types of inaccurate requests and potential influences on the approvals of ID fellows Discussions and Recommendations Discussion Strengths and Limitations of the Study Recommendations for Future Study viii

9 4.4. Conclusions References: Appendix: Bibliography: Curriculum Vita: ix

10 TABLES OF ABBREVIATIONS Abbreviations Definition aor Adjusted odds ratio ASP(s) Antimicrobial Stewardship Program(s) CA-MRSA Community-acquired methicillin-resistant Staphylococcus aureus CDAD Clostridium difficile associated disease CDC Centers for Disease Control and Prevention CF Cystic fibrosis CIs Confidence intervals CRE Carbapenem-resistant Enterobacteriacea CRP C-reactive protein CT Computed tomography DDD Daily defined dose DOT Day of therapy EIN Emerging Infections Network ESBL Extended-spectrum β-lactamase ESR Erythrocyte sedimentation rate GI Gastrointestinal hrs Hours ICU Intensive care unit ID Infectious disease IDSA Infectious Disease Society of America KPC Klebsiella pneumonia carbapenemase MALDI-TOF Matrix-assisted laser desorption ionization time-of-flight mass spectrometry MRSA Methicillin-resistant Staphylococcus aureus NDM New Delhi metallo-beta-lactamase NICU Neonatal intensive care unit OR Odds ratio PDRAB Pan-drug-resistant Acinetobacter baumannii PICU Pediatric intensive care units PIDF Pediatric infectious disease fellows STRAMA Strategic Program for the Rational Use of Antimicrobial Agents and Surveillance of Resistance UTIs Urinary tract infections WHO World Health Organization yr Year-old x

11 LIST OF TABLES Table 2.1 Restricted antimicrobials in Johns Hopkins Children s Center Table 3.1 Demographic data for the intervention group and the control group with patient and antimicrobial course as the units of measure Table 3.2 Reasons of inappropriate antimicrobial use (in descending order) Table 3.3 Comparisons of the proportions of inappropriate antimicrobial courses at Day 2 and Day 3 after post-prescription review between two groups (unit of analysis: antimicrobial course) Table 3.4 Potential factors associated with inappropriate antimicrobial courses at Day 2 after the ASP team s review (Unit of analysis: patient) Table 3.5 Recommendations recorded in the data collection forms by the ASP team for two groups Table 3.6 Comparisons of changes noted at Day 2 after post-prescription review between intervention and control groups (N_change/N_total (%)) Table 3.7 Outcome of patients when ASP recommended alternative therapy or no therapy Table 3.8 Basic demographic and clinical data in the pediatric prior-approval requests (unit of analysis: antimicrobial request) Table 3.9 The frequencies of different types of inaccuracies among the inaccurate requests Table 3.10 Examples of different types of inaccuracies of prior-approval requests and the potential influence upon PIDF approval Table 3.11 Bivariate analysis of potential risk factors of inaccurate requests Table 3.12 Multivariate analysis of risk factors for inaccurate requests Table 3.13 Odds ratio (OR), adjusted odds ratio (aor) and adjusted p value for risk factors of inaccuracies aged 1 year and 1 year old Table 3.14 The potential influence of inaccuracies on approval by PIDF Table 3.15 Types of inaccuracies in patients aged 1 year old and > 1 year old xi

12 Table 4.1 Estimated sample size in each group in ascending order if significant reductions of antibiotic use are to be reached by using the results of this study: power 80%, alpha 0.05 and 2-sided test of significance xii

13 LIST OF FIGURES Figure 2.1 Flow chart of post-prescription review for the intervention and control groups 32 Figure 3.1 Comparisons of restricted and total antibiotic use (DOTs), with and without outliers Figure 3.2 Comparisons of inappropriate restricted antibiotic use (DOTs), with and without outliers Figure 3.3 Comparison of median duration (days) of restricted antibiotics and combined restricted antibiotic use (days) per episode of infections between two groups Figure 3.4 Comparisons of median duration of restricted antibiotics (days) per patient per episode of infection in different review timing in the intervention group Figure 3.5 The influence of review timing after the antibiotic was initiated on the proportion of inappropriate antimicrobial courses Figure 5.1 Sample data collection form for Aim 1 and Aim Figure 5.2 Sample data collection form for Aim xiii

14 1. Background 1.1. The Development of Antimicrobial Stewardship Programs (ASPs) Since the discovery of penicillin by Alexander Fleming in , antibiotics have been among the most widely prescribed drugs and have improved patient care greatly. However, their effectiveness is being curtailed by the emergence of antibiotic-resistant bacteria. 2 The inappropriate use of antibiotics has been linked to the emergence of antibiotic resistance. 3 Since the development of new antibiotic classes has slowed in the recent past and is not anticipated to change in the near future 4, 5, it is imperative to maintain the effectiveness of current antibiotics. To respond to the threat of antimicrobial resistance, antibiotic stewardship programs (ASPs) have been developed to promote the judicious use of antibiotics and to prolong the effectiveness of currently available antibiotics The Emergence of Antibiotic Resistance The prevalence of antimicrobial resistance is increasing globally. In the 1940s, Staphylococcus aureus (S. aureus) became the first organism to develop resistance to penicillin. S. aureus subsequently developed resistance to methicillin in 1960s, and to vancomycin in the mid-1990s.2 Furthermore, antibiotic resistant S. aureus is not only confined to hospitals. Community-acquired methicillin-resistant Staphylococcus aureus (CA-MRSA) has become a major problem in the United States, causing skin and soft tissue infections in otherwise healthy children and adults. Based on information from The Surveillance Network Database-USA, an electronic repository of antimicrobial drug susceptibility data, the incidence of CA-MRSA in the United States rose to 66.1% among all MRSA isolates in 2007 with the majority isolated from children. 6 Additionally, the 1

15 incidence of CA-MRSA increased more than 7-fold in outpatient settings (from 3.6% to 28.2%), between 1999 and The problem of antibiotic resistance is not confined to the U.S. For example, about 40% of community acquired S.aureus infections in Algeria were CA-MRSA 8. In Taiwan, MRSA is endemic in most hospitals, accounting for 53 83% of all S. aureus isolates in 12 major hospitals in , and CA-MRSA infections have been reported increasingly in Taiwanese pediatric patients since 2002 with an incidence rate >50% in pediatric cases of community acquired S. aureus infections. 10 In Hong Kong, MRSA has been reported to account for 30% to 40% of all S. aureus isolates in hospitals during the surveillance period. 11 In addition to infections caused by S. aureus, those caused by other gram positive microorganisms such as enterococci are among the most common hospital associated infections in recent years. 12 In , Enterococcus species were the second most common health care associated isolates in the United States, and the majority of Enterococcus faecium isolates from central line-associated bloodstream infection in the U.S. were resistant to vancomycin (82.6%). 12 In Taiwan, the vancomycin resistance in E. faecium also increased significantly from 0.3% in 2004 to 24.9% in 2010 (P <0.001). 13 The optimal treatment for multidrug resistant enterococcal infection has still not been identified. 14 Similarly, multidrug-resistant Gram-negative microorganisms, including Pseudomonas aeruginosa, Acinetobacter baumannii, and extended-spectrum β-lactamase (ESBL) producing or carbapenemase-producing Enterobacteriaceae, are increasingly being reported worldwide. 15 Some of these organisms are extremely difficult to treat 2

16 due to the development of resistance to most or all antibiotics, such as the pan-drugresistant Acinetobacter baumannii (PDRAB), carbapenem-resistant Enterobacteriacea (CRE), including Klebsiella pneumonia carbapenemase( KPC) Enterobacteriacea and the New Delhi metallo-beta-lactamase(ndm) Enterobacteriaceae which emerged in the past decade. 16,17, The Significance of Antibiotic Resistance and Other Adverse Outcomes As antibiotic resistance emerges and increases, many previous effective antibiotic therapies are losing their efficacy. As discussed below, the outcomes of this trend are higher rates of mortality and morbidity, longer hospital stays, and greater medical care expenditures. For example, in a study with careful matching, the costs for care of hospitalized patients with healthcare- and community-associated infections caused by antimicrobial-resistant organisms were estimated to be $ 15,626 and $25,573 greater than for those with infection due to antimicrobial-susceptible organisms. 19 The differences were even larger when costs for these patients were compared with costs for patients without infection. 20 A sensitivity analysis using a regression model to adjust for potential confounding showed that the medical costs attributable to antimicrobial-resistant infection in the U.S. ranged from $18,588 to $29,069 per patient. 21 In addition, antimicrobial-resistant infection prolonged hospital stays by days, and mortality attributable to this type of infection was 6.5%. The societal costs were estimated at $10.7-$15.0 million. In addition, the cost of the gradual loss of efficacy of certain antimicrobial classes, or the increased need for surgical or other procedures due to these infections, is difficult to measure. 20 3

17 Antibiotic Use and Antibiotic Resistance Multiple factors may contribute to the development of antibiotic resistance, but prior antibiotic use plays a key role in this process. Evidence of this relationship is apparent from several studies. For example, studies in the U.S. have shown that the prevalence of resistance for Enterobacter and Pseudomonas species increases in parallel with increases in antimicrobial use. 22,23 Recent exposure to antibiotics was the only predictor that was consistently associated with carbapenem-resistant Enterobacteriaceae and vancomycin-resistant enterococci. 24,25 Areas within hospitals with higher rates of antibiotic resistance also tend to have higher rates of antibiotic use, and increasing the duration of antibiotic treatment also increases the risk of colonization with resistant organisms Adverse Effects of Inappropriate Antibiotic Use In addition to the development of antibiotic resistance, inappropriate antibiotic use also contributes to other adverse effects. For example, Clostridium difficile associated disease (CDAD) is the leading cause of nosocomial diarrhea in industrialized countries. Clostridium difficile infections can cause pseudomembranous colitis and may even lead to toxic megacolon, which is life-threatening. In the U.S., the incidence of hospital admissions complicated by CDAD nearly doubled between 2000 and CDAD was the leading cause of nosocomial infectious diarrhea in hospitalized patients and represents a significant economic burden. 27 The most important risk factor for CDAD is the recent receipt of antibiotics. 28 CDAD was found to be associated with use of a variety of antibiotics, especially ampicillin, clindamycin, 4

18 fluoroquinolones and third-generation cephalosporins. 28 Some studies have shown that programs to restrict inappropriate antibiotic use have decreased the incidence of CDAD. 29,30, The Development of Antimicrobial Stewardship Programs Numerous studies from around the world have shown that up to 50% of antimicrobial use in humans is inappropriate and unnecessary, 32,33,34 with redundant antibiotic use reported in up to 71% of patients who received two or more antibiotics. 35 Since inappropriate use of antimicrobials creates selective pressure for the development of antibiotic resistance, the best strategy to curb the spread of antibiotic resistance is to use available antimicrobials more carefully and more appropriately. The World Health Organization (WHO) has defined optimal prescribing as the cost-effective use of antimicrobials which maximizes their clinical therapeutic effect, while minimizing both drug-related toxicity and the development of antimicrobial resistance. 36 In hospitals, the adoption of an antimicrobial stewardship program (ASP) as a means to achieve optimal prescribing has been widely accepted in recent years, and has also been recommended by the Infectious Disease Society of America, the Society for Healthcare Epidemiology of America, the Centers for Disease Control and Prevention and World Health Organization. 37,38 The primary goal of an ASP is to optimize clinical outcomes while minimizing the unintended consequences of antimicrobial utilization such as the development of resistance and toxicity. 36 The secondary goals include reducing health care costs without adversely impacting 5

19 patient care. 37 Antimicrobial stewardship programs are central to the multifaceted efforts to control the emergence and spread of antibiotic resistance Overview of Antimicrobial Stewardship Programs Effective ASPs may include a number of active and supplemental strategies as described below. There are advantages and disadvantages to the use of each of these strategies. Many programs adopt hybrid strategies, and strict classification is not always possible. 37,39 When choosing a strategy or set of strategies to implement, it is important to consider the local culture, attitudes and available resources Active Strategies Formulary Restriction and Prior Approval Requirement for Specific Agents (Prior Approval Programs) This strategy could lead to immediate and significant reduction of antimicrobial use. 37 The primary care team communicates with the ASP team and requests specific antibiotics or advice. Restricted antibiotics are not released without ASP team approval. A detailed description of prior approval programs is provided in section Prospective Audit with Intervention and Feedback (Post-prescription Review Programs) Post-prescription review can ensure that antimicrobial treatment is optimal in situations where additional microbiological and clinical data become available within hours after initiating antimicrobial treatment. This strategy can also 6

20 lead to a reduction in inappropriate antimicrobial use. 37 A detailed description of post-prescription review programs is provided in section Supplemental Strategies Antibiotic Cycling Antibiotic cycling utilizes the scheduled rotation of antimicrobials from different classes in order to minimize the selective pressure exerted by individual antibiotics. 37 While many institutions no longer cycle antibiotics, several centers and certain units within centers, such as the pediatric intensive care unit (PICU) in the Johns Hopkins Children s Center, were using this practice at the time of our analysis. However, the compliance with cycling might be reduced because of concerns about adverse effects and the belief among providers that there may be better options for antibiotic use in individual patients. 41 Additionally, the available evidence to date is too weak to support cycling of antibiotics as a means of reducing antibiotic resistance rates Education Education is also an important component of any ASP. Education could include teaching sessions, provisions of written guidelines, online learning, etc. to provide a foundation of knowledge that could improve future prescribing behaviors. However, the success of education depends on the motivation of the clinicians. Education alone is marginally effective in changing antimicrobial prescribing practices and is difficult to sustain if not incorporated into programs using other active strategies. 37, 42 7

21 Clinical Pathway and Guidelines Clinical pathways and guidelines can lead to improved antimicrobial use if they incorporate local microbiologic resistance patterns to recommend selection of appropriate antimicrobial agent and dosage and if buy-in is obtained from participating clinicians. 37 For example, in one facility, the ASP team cooperated with general surgical leadership to develop hospital guidelines for management of complicated intra-abdominal infections, based on the 2010 IDSA guidelines. This study showed that the use of antibiotics was improved without significant change in readmission rates, hospital length of stay or rates of CDAD Computer-Based or Assisted Antimicrobial Stewardship Programs Certain tools (such as computer-assisted programs), when used as part of a comprehensive ASP, may also reduce antibiotic use, decrease antibiotic dosing errors, and more readily identify drug-associated adverse events in a timely fashion. 37 Computer-assisted programs have been developed as a means of improving antibiotic selection, dosing and duration. In addition, these programs can also more easily measure antibiotic utilization, monitor adverse events and identify nosocomial infections in a timely manner. 37 One computerized physician orderentry system utilized at Brigham and Women s Hospital showed that prescribers wrote significantly fewer orders for vancomycin when asked to key in a rationale for vancomycin use from one of the categories provided. 44 The authors concluded that a relatively soft educational intervention of displaying criteria for antimicrobial use 8

22 and adding a justification step to ordering antimicrobials can have a substantial effect at controlling prescribing. 45 Another study used the computer decision-support system to adjust the dosing guidelines for pediatric populations to ensure that treatment recommendations were appropriate. 46 The results showed that the system was associated with a 59% decrease in the rate of pharmacy intervention for dosing errors and a 28% decrease in the rates of excess antimicrobial dosing days. Despite the potential advantages of a computer-based system, there are also limitations. Computer-based systems might decrease the opportunity for ID fellows to acquire specific clinical information, and could also decrease the opportunity for instant communication and education. Computer-based systems must also be flexible, user-friendly, and allow for reprogramming with ease when there are updated guidelines or consensus statements Other Strategies Several additional strategies that could be incorporated in the postprescription review program or prior-approval program include but are not limited to: 1) dosage optimization using pharmacokinetic and pharmacodynamic principles and 2) when appropriate, conversion from parenteral to therapy with an oral agent with high bioavailability Current Status of Antimicrobial Stewardship Programs In the US, a survey done in 2009 revealed that among 522 responding physician members of Emerging Infections Network (EIN) who cared for adult patients, 61% 9

23 reported that their hospitals had ASP in place 48, compared to 45% of respondents in a similar survey done in The percentage of institutions implementing ASP increased during this 10 year period, although small community hospitals were still least likely to have ASP programs. The strategies of ASP also shifted from primarily formulary restrictions or prior-approval programs alone to combined sets of strategies designed to provide feedback to the prescribers. In the 2009 survey, 67% of ASP programs reported using post-prescription review as their primary strategy. 48 One limitation of the study described above is that EIN members might be more likely to be interested in infection control and to respond to a survey. A study that may be more representative of all providers was performed in California. In this study, all general acute care hospital campuses were invited to participate in a survey, and the participating hospitals were statistically representative of all the acute care hospitals in the state. 50% of the participating hospitals had an ASP in place and 30% reported planning an ASP. In hospitals that had an ASP in place, 26% had implemented postprescription review. In addition, 22% of the responding hospitals reported that knowledge of the legislation which mandated that all general acute care hospitals develop processes for evaluating the judicious use of antimicrobials had influenced initiation of their ASP. 50 This is the only law of its kind in the U.S. With respect to pediatric hospitals, a 2008 survey of 246 pediatric infectious disease consultants who were members of EIN showed that only about 33% of respondents reported having an ASP 51, and 18% of respondents were planning a program. Obviously, pediatric hospitals have lagged behind general hospitals in the implementation of ASP. 48 Of the respondents with pediatric ASPs, 78% reported using 10

24 prior approval programs, and 33% reported using the prospective audit and feedback strategy. 51 Because of global awareness of this threatening trend of increasing antimicrobial resistance, some countries have launched national programs for antimicrobial stewardship. 52,53 In Sweden, the Strategic Program for the Rational Use of Antimicrobial Agents and Surveillance of Resistance (STRAMA) antimicrobial stewardship initiative reported a reduction of antibiotic use for outpatients and low antibiotic resistance rate for most bacterial species over 10 years without measurable negative consequences. 52 ASPs also have been successfully implemented in certain hospitals in Taiwan, Hong Kong, India and Australia in recent years. 54,55,56, The Influence of Effective Antimicrobial Stewardship Programs There is substantial evidence to indicate that antimicrobial stewardship can reduce medical costs and potentially reduce antibiotic resistance. However, many physicians don t focus on health care costs, and the factors involved in the development of antibiotic resistance are complex and multi-factorial. Therefore, the most important measures of ASP success for clinicians are related to improvement of quality of care and patient health outcomes Improve patient outcomes ASPs have been shown to shorten the duration of antimicrobial use, decrease the re-admission rate and increase being discharged home without antibiotics in patients with community-acquired pneumonia. 59 ASPs have also been shown to significantly decrease nosocomial infection by C. difficile and resistant Enterobacteriaceae, 29,30 increase cure rate and reduce failure rate

25 A critical feature of ASPs is that they should not compromise patient safety by reducing the use of antibiotics when use is appropriate. A prevalence survey conducted in the Netherlands showed that inappropriate lack of treatment was uncommon (0.6%) when antibiotics were indicated. 60 A number of other studies have demonstrated no increase in 30-day readmissions, nosocomial infections, length of stay or mortality, 59,61,62,63,64 even when ASP was implemented in critical care patients Decrease antibiotic resistance Although it often takes years to demonstrate the benefits of less resistance or reduced emergence of resistance, it is still important to note that good antimicrobial stewardship entails more than consideration of the immediate benefit to the individual patient being treated. It also considers the long-term effects of use on the future preservation of susceptibility in the practice population of the prescriber. 65 However, a literature review from Tamma et al. found that there were only a few studies reporting short-term reductions in antimicrobial resistance, and even fewer for long-term reductions. 41 A study of a prior-approval program showed increased susceptibility to all beta-lactam and quinolone antibiotics after a 6 month implementation period. The effect was especially obvious when isolates from intensive care units were examined. 64 Another study demonstrated that restricted access to third generation cephalosporins significantly decreased the prevalence of ESBL-Escherichia coli and Klebsiella species during a 5-year study period. 66 Because there are only a few studies addressing the long-term impact of ASPs on antimicrobial resistance, additional well-designed studies are needed in the future

26 Reduce medical expenditures Effective antimicrobial stewardship programs can reduce medical expenditures which are of great interest to administrators. 67 Comprehensive ASPs have been shown to reduce the use of antibiotics and consequently decrease medical expenditures in both larger academic hospitals and smaller community hospitals. The savings achieved could be used to support ASPs, making them self-sustaining. 37 For example, one study showed that ASP decreased antibiotic expenditure by 46% during its 7-year presence, but that expenditures increased by 32% (approximately $2 million) over a 2-year period after the program was stopped. 68 Another study showed that a combined priorapproval and post-prescription review program could have sustained economic benefits over 11 years with average cost savings of $920,070 to $2,064,441 per year Prior Approval Programs The strategy of prior approval programs is to limit the use of some antimicrobials to certain approved indications. Designated persons, either infectious disease fellows or attending physicians, or infectious disease trained pharmacists, are assigned to implement the approval process Advantage of Prior Approval Programs There is already good evidence that prior approval programs can result in immediate direct and significant reduction in antimicrobial use and cost. 3,64,66,70,71 The first reported study was conducted at Boston City Hospital required prescribers to notify a member of the infectious disease unit before their choice of restricted antibiotic could be dispensed from the pharmacy. 70 This study showed significant decreases in the 13

27 use of certain antibiotics from the restricted list. Other studies showed that similar programs could reduce bacterial resistance. 64,66,72,73 In a study done at the University of Kentucky, formulary restriction combined with a prior approval program reduced the resistance rates of several important pathogens, including multidrug-resistant Pseudomonas aeruginosa and MRSA. 72 Hospitals with policies for restriction of carbapenem use have both lower rates of carbapenem use and lower incidence rates of carbapenem resistance in P. aeruginosa than those without these policies Limitations of Prior Approval Programs Potential challenges for effective prior approval programs exist. Generally, prior approval programs only affect the initial choice of empiric therapy, and broad-spectrum antibiotics are often approved as initial empiric therapy for critically ill patients which could be inappropriate as later clinical information available. However, a multi-center study showed that post-prescription review program could reduce antimicrobial use significantly even in hospitals with highly restricted pre-prescription approvals. 74 In addition, prior approval programs generally do not consider the appropriateness of non-restricted antimicrobials, which are the vast majority of antimicrobials used in the hospital. The restriction of one antibiotic might result in increased use of another antibiotic, a phenomenon which has been described as squeezing the balloon. 75 Prior approval can also be labor intensive. Sufficient staff with expertise in antibiotic use must be available to provide immediate, real-time service to avoid delay in the initiation of empiric therapy. 71 The perceived loss of autonomy by prescribers might influence their acceptance of a prior approval program. One study showed that about 50% of housestaff felt that 14

28 being forced to request approval was frustrating and limited their autonomy. This viewpoint was more common among senior residents than interns (48.8% vs. 8.8%, P<0.005). 76 In addition, members of the antimicrobial approval team might be anxious to maintain good relationships with their colleagues, which might influence their approval practices. Finally, the prescribers also might overstate the severity of patients conditions to gain approval for use of restricted antibiotics, 77,78 or might try to escape the approval period in order to prescribe their targeted antibiotics. 79 An example of this type of escape was shown in a study performed at the Hospital of the University of Pennsylvania 78, where requests for restricted antimicrobials were approved by an infectious disease fellow or infectious disease-trained pharmacist between 8:00 am and 10:00 pm each day. However, outside this time period, prescribers could order any restricted antibiotics but the orders needed to be approved the next morning during ASP active hours in order to be continued. The study found that restricted antibiotics were ordered at a greater rate (restricted antibiotics/ total antibiotics during that period) between 10pm and 10:59pm compared to other hours (57.0% vs. 49.9%; p=.02). In addition, restricted antibiotics prescribed in the first hour after the approval system ended were less likely to have the antibiotics continued compared with the last hour during the approval system active hours. The type of prescribers (surgical or nonsurgical) and the patient s location were not confounders or effect modifiers between the relationship of ordering time and restricted antibiotics. While the reasons for these prescribing patterns are not completely known, the authors of the study had several hypotheses: prescribers might have been concerned that their requests would be denied, might have been too busy to spend time or were reluctant to communicate on the phone, or simply wanted to avoid difficult interactions

29 Another study conducted at the Hospital of the University of Pennsylvania compared the information contained in documented telephone calls from prescribers to the ASP team to information contained in patient s medical records. 78 This study found that inaccurate information was communicated in over one third (39%) of all ASP calls, and that the most common types of inaccuracies included: reports of current antibiotic therapy (12.9% of all calls), microbiological data (11% of all calls), patient body temperature (7.8% of all calls), allergies to medications (5.1% of all calls), and radiological data (3.5% of all calls). ASP calls from surgical services contained more inaccurate information than those from non-surgical services (48% vs. 34%). In a follow up study, these inaccurate communications during prior-approval calls, especially microbiological data, were found to be associated with inappropriate antimicrobial recommendations made from the ASP team (odds ratio 2.2, p=0.03) Post-prescription Review Programs Post-prescription review usually occurs within hours after empiric antimicrobial therapy is initiated: a member of the ASP team contacts the prescriber to optimize antimicrobial use. Sometimes it can also be conducted earlier (within 24 hours) to replace the often complicated on-demand system of the prior-approval program. Earlier review can assure appropriate prescribing of empiric therapy Advantages of Post-prescription Review Programs Post-prescription review allows for reevaluation of empiric therapy with broadspectrum antimicrobials in unstable patients when additional microbiological, radiological and clinical information becomes available. The ASP team can then work with prescribers to optimize antimicrobial use including streamlining therapy or 16

30 modifying therapy to match the targeted pathogen, and thereby reduce the possibility that the approved restricted antibiotic is continued indefinitely or inappropriately used while still allowing for aggressive empirical therapy. 81 This is important in all populations, and especially in pediatrics. For example, one study showed that prolonged initial empiric antibiotic therapy was associated with increased risks of necrotizing enterocolitis or death in extremely low birth weight infants after adjusting gestational age, Apgar scores, race and other confounding factors. 82 Post-prescription review programs have been shown to be effective in a number of settings. A study done in adult patients in a tertiary teaching hospital observed that the intervention rate (defined as the number of courses of therapy in which an intervention was recommended divided by the total number of courses of therapy reviewed) was superior for post-prescription review as compared with a prior approval program (28%-34% vs. 5%). 81 A randomized controlled trial in an internal medicine setting shown that ASP involvement resulted in a 37% reduction in duration of inappropriate antimicrobial use 83, and was superior to provision of indication-based guidelines 84. Clinical pharmacists may be especially effective in the implementation of postprescription review programs. Studies have shown that a post-prescription review program that utilized clinical pharmacists resulted in a significant increase in deescalation of therapy (from 72% to 90%, with an acceptance rate of 91%) 85, and that clinical pharmacists intervene more often than infectious disease (ID) fellows in adult patients (29% vs.9%)

31 Additional evidence suggests that post-prescription review might improve antimicrobial use in specific settings such as intensive care units, 87 long-term care facilities, 88,89 community hospitals with more limited resources, 30,90 ambulatory setting. 91. For example, an intervention in 3 ICUs in a tertiary care center with postprescription review at the 3 rd and 10 th day of therapy showed that monthly broadspectrum antibiotic use, incidence of CDAD and resistance to meropenem were decreased without change in ICU length of stay and mortality. 87 In a medium-sized community hospital, a post-prescription review program showed a reduction of 22% in the use of parental broad-spectrum antimicrobials and a significant decrease in nosocomial C. difficile infections and multidrug-resistant Enterobacteriaceae infections. 30 In hospitals with more limited resources where daily review of antimicrobial use is not feasible, a relatively scaled-down post-prescription review, such as only targeting patients receiving multiple, prolonged or high-cost antimicrobials and limiting recommendations to well-defined clinical scenarios, can still have a significant impact with an estimated 19% reduction in antimicrobial expenditures Limitations of Post-prescription Review Programs One of the limitations of this strategy is the potential for unnecessary antibiotic exposure, cost and toxicity if used in the absence of prior approval or clinical guidelines. 37 In addition, the effectiveness may be reduced if the primary team does not consistently follow suggestions. Post-prescription review programs are also labor intensive and time consuming. It is important for smaller hospitals or hospitals in developing countries to modify their strategies according to their resources. 92 For example, in a hospital with only one 18

32 infectious disease physician available for the ASP program, focusing ASP interventions on the intensive care unit, where the majority of restricted antimicrobials are prescribed, might be the best strategy Pediatric Antimicrobial Stewardship Programs Difference Between Pediatric Patients and Adult Patients Most of the studies published have focused on adult inpatient populations. However, it is difficult to extrapolate from the relative efficacy observed in these studies to pediatric populations because there are some inherent differences between these populations. First, fewer antibiotic treatment guidelines are available for children. 93 Treatment protocols used in adults might not be able to be replicated in children. One meta-analysis showed an increased rate of treatment failure when urinary tract infections (UTIs) in children were treated using the standard guidelines for treatment of UTIs in adult women. 94 Second, there are many fewer pharmacokinetic studies done in young children. Third, many infections in young children are viral and would not require antibiotics. For example, respiratory syncytial virus (RSV) is the most commonly identified cause of lower respiratory tract infection in young children, and is the cause of 50 to 90 percent of hospitalizations for bronchiolitis, 5 to 40 percent of those for pneumonia, and 10 to 30 percent of those for tracheobronchitis. 95 Therefore, the recent published pediatric community-acquired pneumonia guideline states that antimicrobial therapy is not routinely required for preschool-aged children with CAP, because viral pathogens are responsible for the great majority of clinical disease. 96 Fourth, dosing errors might occur more frequently in pediatric patients than adults 19

33 because most antimicrobials are administered by weight in children, whereas standard doses are used in adults Antimicrobial Use in Children in the United States The prescription rate of antimicrobials is extremely high in hospitalized and outpatient children in the United States, although data are limited. Studies show that about 60% of pediatric inpatients receive antimicrobials; the rates reported range from 38-72%. 98 Carbapenems and linezolid use increased enormously from 2002 to 2007 (100% and 279%, respectively) % of children in pediatric intensive care units (PICU) and 43.2% of children in neonatal intensive care units received antimicrobials; the majority of treatment was empiric therapy. 100 On an outpatient service, antibiotics were prescribed during 21% of pediatric visits; 50% of these included broad-spectrum antibiotics. Approximately one quarter of the visits in which antibiotics were prescribed was for respiratory conditions for which antibiotics are not clearly indicated Review of Pediatric ASP studies Although the majority of ASP studies were performed in adult settings, a few recent studies support the use of ASP in pediatric populations. In a study done at the Children s Hospital of Philadelphia which relied on a prior authorization and a postprescription review for re-approval of targeted antimicrobials 102, 45% of requests for restricted antibiotics required an intervention by the ASP. The most common intervention made by the ASP was consultation with the prescribing clinicians, followed by selection of an appropriate agent, or recommendations regarding the dose or 20

34 duration of treatment. The compliance rate was 89% with these interventions, and the clinical outcome of the patients for whom alternative or no therapy was recommended by ASP was acceptable. The high rate of interventions and the reasons for those interventions suggest that pediatric ASPs are needed. One study evaluated the use of the CDC 12-Step Campaign to Prevent Antimicrobial Resistance performed in four tertiary care NICUs. This study found that 28% of antibiotic courses and 24% of all antibiotic days were considered to be non-compliant with at least one CDC 12-Step element. 103 Not targeting the pathogen was the most common violation, such as continued use of vancomycin in methicillin susceptible S. aureus. In addition, inappropriate use was more common during continuation of antibiotics than with initiation of therapy (39% versus 4%), which implied that antimicrobial stewardship focusing on post-prescription review might have a greater effect than prior approval in NICU populations. Another study revealed that inappropriate use of vancomycin and cefepime was greater on the surgical service than the medical service and in the pediatric intensive care unit as compared to the general ward. The most common inappropriate use was failure to stop therapy or de-escalate therapy. 104 Post-prescription review combined with guidelines could decrease targeted antibiotic use and could have other benefits for pediatric patients. An intervention in a district hospital showed that use of revised antibiotic guidelines aimed to avoid broadspectrum antibiotics combined with post-prescription review for those prescribed broad-spectrum antibiotics led to significant decreases in the use of fluoroquinolone and cephalosporins and also to a decrease of the incidence of CDAD. 105 Another study based on post-prescription review and the distribution of antibiotics guidelines showed that there was a 21% decline in targeted antimicrobial doses 3 years after the 21

35 intervention started. Antibiotic susceptibility to broad-spectrum antibiotics remained high for most common gram-negative bacteria isolates in a 7-year follow-up. 106 Finally, ASP could also be successful in pediatric ambulatory settings. One recent study evaluated an intervention focusing on acute sinusitis, streptococcal pharyngitis and pneumonia. This intervention, which combined clinical education (a 1 hour on site session) with personalized quarterly audit and feedback on prescribing, demonstrated that off-guideline antibiotic use reduced from 15.7% to 4.2% in pneumonia cases and 38.9% to 18.8% in acute sinusitis cases for 1 year after the intervention Pediatric Prior Approval Programs in the Johns Hopkins Children s Center In June 2005, Johns Hopkins Children s Center implemented a web-based restricted antimicrobial approval program that was developed by a team of pediatric infectious disease physicians, pharmacists, and information systems experts. 107 Physicians use the web-based tool to submit requests for restricted antibiotics for approval. This web-based tool not only expedites the approval (and disapproval) process, but reduces missed and unnecessary antibiotic doses. The program was shown to improve users satisfaction (from 22% to 68% among prescribers), decrease the number of doses of restricted antimicrobials dispensed (11%), decreased patient-days of restricted antimicrobials (14%), improve multidisciplinary communication, and significantly decrease antimicrobial cost expenditures from restricted antibiotics (22%) without a change of expenditures on unrestricted antibiotics

36 1.6. Rationale for this study The problem of antibiotic resistance is increasing globally. In Taiwan, for example, the excessive use of antimicrobials is very prevalent, 108 and the rate of the antibiotic resistance is one of the highest in the world. 9,10,13,109 While a large number of studies of ASP have been conducted in adults, there have been relatively few studies conducted in pediatric settings, and it is important to add to the evidence base for support and promotion of pediatric ASP studies. On a local level, it is also important to determine whether any improvements could be made to the pediatric ASP at the Johns Hopkins Children s Center, and specifically, whether post-prescription review could be a useful addition to the highly successful web-based priorapproval program. 107 Studies in other settings have shown that post-prescription review programs could reduce antimicrobial use significantly even with a highly restrictive priorapproval program in place, 74 and were more likely to detect an inappropriate antibiotic course than prior approval programs. 81,83,103 Prior approval programs and post-prescription review programs are not mutually exclusive, but could be bundled to better shepherd precious antimicrobial resources and improve antibiotic use. 110 The introduction of a post-prescription review program might enhance patient care in the Johns Hopkins Children s Center. Currently, the pediatric ASP at the Johns Hopkins Children s Center does not have a mechanism for assessing the accuracy of information provided by treating physicians. In the Johns Hopkins Children s Center, the pediatric ID team does not routinely check the accuracy of the submitted requests because of the urgency of the need for approval and time constraints. Previous studies in adult settings have shown that submission of inaccurate information may lead to decreased program effectiveness 78. For this reason, it is also useful to assess this component of the pediatric ASP. 23

37 Most ASP studies used quasi-experimental designs to compare outcomes before and after the intervention. Although the intervention might be the major contributor to the outcome, it is not the only factor and it is often difficult to control for important confounding variables, such as maturation effects associated with changes in patient condition, increase in provider experience, implementation of new guidelines or initiatives during the study period, or seasonal changes in diseases, etc. 65 Unblinded ASP study members could also be biased in assessment of study outcomes if they had prior knowledge that an intervention could affect the appropriateness of the antibiotic use. 111 For these reasons, we introduced an intervention with randomized controlled study design which can reduce some bias mentioned above. The specifics of the study design are described more completely in section

38 1.7. Hypothesis and Specific Aims Specific Aims Aim 1 To determine the effectiveness of the prior-approval program alone and priorapproval combined with post-prescription review in reducing total and restricted antibiotic use and in reducing the proportion of inappropriate antimicrobials used. We hypothesize that a prior-approval program combined with post-prescription review program would: a) decrease total and restricted antibiotic use per patient b) reduce the proportion of inappropriate antimicrobials used compared to a prior-approval program alone Aim 2 To assess both compliance with ASP team recommendations as well as patient outcomes with a prior approval program alone and with a prior approval program combined with post-prescription review. We hypothesize that: a) prior approval combined with post-prescription review would yield a higher compliance rate with the recommendation of the ASP team than prior approval alone b) prior approval combined with post-prescription review program would have similar outcomes for patients as prior approval program alone 25

39 Aim 3 To: a) determine how often inaccurate information was submitted on the pediatric prior approval request forms b) identify risk factors for these inaccurate requests and assess the appropriateness of antimicrobials approved We hypothesize that inaccurate information would be submitted and that risk factors might include clinical service (surgical or medical) and the type of antimicrobial requested. We also hypothesize that some of these inaccuracies could affect the appropriateness of approvals. 26

40 2. Methods 2.1. Methods for Aim 1 and Aim Study Design Study Population and Data Sources Our study was undertaken in the Johns Hopkins Children s, a 180-bed acute care children s hospital which is part of the Johns Hopkins Hospital located in Baltimore, Maryland Center from September 2011 to November The existing stewardship program consisted of a web-based prior approval program which was implemented on June 1st, 2005, to replace the traditional telephone-based verbal approval program. There was no post-prescription review component before this study started. Restricted antibiotics in Johns Hopkins Children s Center are listed in Table 2.1. Some of these antimicrobials are restricted throughout the Children s Center, and some are restricted except for use by certain services. Using the web-based system, prescribers provide the rationale (from an antibiotic-specific list or through free text message) for restricted antibiotics with supporting data, if any. Simultaneously, pediatric infectious disease fellows (PIDF) are paged by the system automatically. PIDF then enter the approval decisions into the system and these are automatically transmitted to the prescribers and to the pharmacy via pager. Pediatric ID attending physicians are available for back-up if needed. The program operates each day from 8 am to 10 pm. Outside of this time period, prescribers can order restricted antibiotics without approval except for a few products (for example, fosfomycin, daptomycin, and palivizumab) and any restricted drug was approved automatically for one or two doses during the time period when PIDF was 27

41 not available. The PIDF would look at the list the next morning for overnight approvals and could choose to approve for longer or to stop further approval. The system is also programmed with certain specific drug-indication combinations for auto-approval to save time and facilitate intervention in more complicated cases. Use of restricted antibiotics was approved up to a specified stop date, but the current system allowed the pediatric pharmacy to dispense the restricted antibiotics after the stop date in order to prevent lapses in dosing for critically ill patients. Once a drug was ordered, the patient could still continue to receive the drug until the treating physician decided to write a discontinuation order, even if the PIDF did not choose to approve it for continued administration. However, resubmission of requests for restricted antimicrobials before the approval stop date was encouraged. Our study intervention included inpatients from 4 hospital locations. We excluded several groups of patients, including those in the ICU, hematologyoncology and cystic fibrosis patients, and patients for whom ID consults had already been obtained. We excluded these patients because they often presented with more complex medical conditions and/or because they were treated using existing antibiotic algorithms. Ethics approval for the study was obtained from the Johns Hopkins Medicine Institutional Review Board. 28

42 Table 2.1 Restricted antimicrobials in Johns Hopkins Children s Center 29

43 Study Design The medical records of the patients from four general pediatric floors who had received restricted antimicrobials within hours after initiation of therapy were identified. After exclusion of the special populations described in Section , medical records were reviewed prospectively by a pediatric ID pharmacist and a pediatric ID fellow in the ASP team with a pediatric ID attending physician providing backup. The ASP team reviewed each case, taking into account local resistance patterns and then decided if the antibiotic use was appropriate. Decisions regarding inappropriateness (Y/N), and reasons why the use was inappropriate, and recommendations for change in antibiotic use were collected on a standardized data collection form. Reviews were not performed on weekends or holidays. Using a random number generated by a third party, all children who were considered to be receiving inappropriate antibiotics were randomized to either receive the intervention (intervention group) or not (control group)(figure 2.1). The intervention consisted of a phone call by pediatric ASP team to the treating physician suggesting a change in the use of antimicrobials or other related recommendations (e.g., obtain an ID consult); however, the final decisions regarding antimicrobial choice were left to the primary treating physicians. In the control group, although pediatric ASP team members wrote down the recommendations in the data collection forms, they did not communicate the recommendations to the treating physicians. The primary providers did not know the contents of the recommendations during the study period. 30

44 Before the introduction of the intervention, one of the ASP team members explained the study to all the pediatric staff to ensure that every primary provider understood that they might receive recommendations to improve antimicrobial use from the ASP team during the study period. 31

45 Figure 2.1 Flow chart of post-prescription review for the intervention and control groups 32

46 Data collection A structured data collection form (Appendix, Figure 5.1) was used to collect basic demographic and clinical information from the electric medical records for each patient. Basic demographic data included age, gender, primary service (defined as the clinical service prescribing orders and writing clinical notes on a patient), review date, and admission date. Clinical information and other data included underlying disease, name and duration of the antibiotic used, total length of stay (hospital days), history of drug allergy, indication for restricted antibiotic therapy, current type of infection, reasons that antibiotic use was considered inappropriate, details of the ASP recommendation, documentation of the treating physician s compliance with the recommendation at Day 2 ( hours) and Day 3 ( hours ) after the ASP team s review, and the safety outcome in cases for which modifying therapy or stopping therapy was recommended. For the indications of restricted antibiotics, the definitions of empiric, directed therapy and prophylaxis were categorized as previously described 103 : Empiric therapy: treating for symptoms or signs of infection Directed therapy: treating a known pathogen Prophylaxis: Preventing infections when patients were asymptomatic or when the antibiotics were used for perioperative phase Outcome Measures Operational definitions of quantitative and qualitative measures of antimicrobial use are defined in detail below: 33

47 Quantitative Measures of Antibiotic Use Days of Therapy Antibiotic use was measured as days of therapy (DOTs) which was abstracted from the pharmacologic database and was normalized to 1,000 patient-days. DOTs were calculated separately for each antibiotic. For example, if a patient received 3 days of vancomycin and 3 days of gentamicin during a 10-day admission, then the patient was considered to have received 300 DOTs/1,000 patient-days of vancomycin and 300 DOTs/1,000 patientdays of gentamicin. For the outcome measure of our study, total DOTs for this patient would be 600 DOTs/1000 patient-days, and restricted DOTs would be 300 DOTs/1000 patient-days. Any dose of a drug in a calendar day was counted as 1 day in quantifying antibiotic use Combined antimicrobial use (days) per patient per episode of infection Antibiotic use was measured as days during the specific episode of infection that the restricted antibiotic was reviewed by the ASP team. It also included the take-home antimicrobials listed in the discharge notes for the continuation of antibiotic therapy for the specific episode of infection. The start of the episode was counted from the start of any antibiotic for the indicated episode of infection. Combined antibiotic use per episode of infection was calculated separately for each antibiotic. 34

48 Median duration (days) of antimicrobial agents per patient per episode of infection The median duration (days) of antimicrobial agents per patient per episode of infection was calculated from all total or restricted antimicrobials used in the specific episode of infection that the ASP team reviewed Measures of Proportions of Antibiotic Courses that were Inappropriate The proportions of antibiotic courses that were inappropriate and the rate of compliance with ASP team recommendations were determined on Day 2 or on Day 3 by examining the patient s pharmacy or medical records to determine whether: 1) the antimicrobial course was still inappropriate; 2) the treating physician accepted the ASP team recommendation or, in the case of the control arm, auto-corrected antibiotic use so that it was the same as what would have been recommended by the ASP team. A course of antimicrobial therapy was considered inappropriate if any of the following criteria was met: Inappropriate dosage was defined as errors in dosage, frequency and/or formulations of antimicrobials based upon dose ranges suggested by the Johns Hopkins Pediatric Infectious Diseases Service, taking into account specific conditions such as renal or liver dysfunction, or bacterial meningitis. Antimicrobial-microorganism mismatching (bug/drug mismatching) was defined as use of a requested or current antimicrobial with suboptimal activity against the microorganism according to the culture and/or susceptibility reports. 35

49 Inappropriate antibiotic selection for documented infection included susceptibility to a narrower-spectrum agent, such as use of vancomycin for treatment of methicillin-susceptible S. aureus in a patient without betalactam allergy. The definition of inappropriate selection also included inappropriate route of administration, such as use of intravenous therapy if the oral form of therapy was considered acceptable. Inappropriate spectrum of coverage included therapeutic duplication (double coverage) which is prescription of two or more antimicrobials with the same antimicrobial activity which is unnecessary (such as piperacillin/tazobactam and metronidazole for treatment of anaerobic infections). No evidence of infection or the infection was a viral infection for which antibiotics were unnecessary were defined as absence of clinical, laboratory or radiographic evidence of infection or the presence of documented or suspected viral infection. Contraindication based on patient s drug allergy history was defined as the prescription of an antimicrobial to patients with known allergies to a particular antimicrobial or antimicrobial class. Prolonged duration of therapy was defined as unnecessarily prolonged therapy for the indicated infection based upon standards established by the Pediatric Infectious Diseases Service (for example, 10 days treatment of acute tracheitis instead of 5 days of therapy; more than 24 hours use of antibiotics after a surgical procedure for perioperative surgical prophylaxis). 36

50 Not requesting an ID consult if ID consultation was considered necessary for the appropriate selection of antibiotics Measures of Compliance with the Recommendation Following the assessment of use of restricted antimicrobials, the ASP team recorded all recommendations in the structured data collection form. Potential recommendations included: 1. Stopping antibiotics (elimination of duplicate therapy or unnecessary therapy) 2. Modifying therapy (adding an antibiotic, prescribing an antibiotic with a narrower or broadened spectrum, adjusting antibiotic dose or duration, changing the route of administration, or recommending an alternative therapy because of patient allergy) 3. ID consult (when the antibiotic choice was complex due to the complicated nature of the patient s condition and ID consultation was considered necessary) 4. Other recommendations: assessing drug levels, monitoring laboratory parameters (for example, CRP, ESR, etc.), recommending sterile site cultures, removal of an infected source (for example, the drainage of an abscess, the removal of a potentially infected device, etc.). For each patient, more than one type of recommendation could be recorded; in this instance, all recommendations were captured on the data collection form. 37

51 The compliance rate was defined as the proportion of all changes made by the treating physicians in compliance with ASP recommendations (treating providers made changes after the communication with the ASP team in the intervention group or the providers made changes on their own without communication with the ASP team) divided by all recommendations recorded in the data collection form by ASP team Measures of patient outcomes evaluated: There were two safety outcomes related to ASP recommendations that we ASP team recommendation at variance with culture results For cases in which alternative therapy (broadened or narrowed empirically) was recommended, we determined whether any of recommended therapies were inconsistent with the antimicrobial susceptibilities of any recovered organisms Developed subsequent infection For cases recommended for stopping therapy, we determined whether the patients developed laboratory-confirmed infections or any infections defined by clinicians and recorded in the medical records within 48 hours following the ASP recommendation. 38

52 Statistical Analysis The analysis was conducted using STATA software version 11 (StataCorp, LP Texas) as described below: Analysis for Reducing Antibiotic Use in Intervention Group Frequency counts and percentages were calculated to describe basic demographic and clinical information in the intervention group and the control group. Student s t-test for continuous variables and Pearson s chi-square test for categorical variables were undertaken to determine whether there were significant demographic or clinical differences between the two groups. If the categorical variable had a cell with fewer than 5 cases, Fisher s exact test was performed. For the comparison of antimicrobial use between the intervention group and control group, the Shapiro-Wilk test was done to determine whether the antimicrobial use data (continuous variables) were normally distributed. If the antimicrobial use data were normally distributed, Student s t-test was used to compare antimicrobial use. If the antimicrobial use data were not normally distributed, Wilcoxon rank-sum test was undertaken with calculation of medians. Several comparisons between the intervention and control arms were made using a t-test or Wilcoxon rank-sum test depending on the results of Shapiro-Wilk test for the analysis of antimicrobial use in our study, including : Restricted and total antibiotic use( DOTs) Inappropriate restricted and total antibiotic use (DOTs) Combined antimicrobial use (days) per patient per episode of infection 39

53 Median duration (days) of antimicrobial agents per patient per episode of infection Cases with extended lengths of stay would potentially influence the calculations of the denominators of DOTs. Because the ASP team did not continuously review antibiotic use in each patient, cases with extended lengths of stay could have extremely low DOTs compared to the cases without extended admissions. Therefore, we determined whether there were high leverage points (defined as the point which was higher than the 3 rd quartile plus 1.5 interquartile ranges) of length of stay and then re-evaluated the comparisons of the days of antimicrobial use in both arms by dropping these cases. Lastly, in order to determine whether an earlier review by the ASP team would have a greater impact on reduction of antimicrobial use than later review, subgroup analysis was also undertaken by t-test or Wilcoxon rank sum test to see if there was difference of median duration of restricted antimicrobial agents per episode of infection by two different review window (Day 2 and Day 3-4 after the restricted antibiotic was initiated) Analysis for Reducing the Proportion of Inappropriate Antibiotic use Chi-square test was used for the comparisons of the proportion of inappropriate antibiotic courses, compliance with the recommendations and for the exploration of the potential factors for inappropriate antibiotics still in use at the Day 2 follow up after the ASP team s assessment. The prevalence ratio (the prevalence of inappropriate antimicrobial course still in use in the intervention group divided by the prevalence in the control group) 40

54 was calculated with point estimates and 95% confidence interval reported. Similar to section , subgroup analysis for different review timing by chi-square test was undertaken for the comparison of inappropriate antibiotic course at Day 2 after the ASP team s assessment Analysis of outcomes when ASP team recommended alternative therapy or no therapy in two arms A frequency count was used to describe the safety outcomes when ASP team recommended alternative therapy or no therapy in two arms Methods for Research Aim Study Design As described above, the clinical setting of this study was the Johns Hopkins Children s Center. The study was a retrospective comparison between the medical records and the prior-approval requests and focused on the accuracy of age, diagnosis, present illness, co-morbid conditions or treatment at the time of requests, laboratory and radiological exams, physical examination findings within the 24 hours preceding the request and current antimicrobial treatment. The study included all documented requests for patients in the Children s Center to the pediatric web-based prior-approval system for 4 months, from December 2011 to March More than one request could be included for a single patient during a given hospitalization. Duplicate requests for a specific antimicrobial were excluded from the study. The patient s medical records, including admission notes, progress notes, consultation notes, laboratory test results, radiological reports, medication administration and patient s history of allergies were used as the source of the data and 41

55 served as the gold standard for comparison with the documented pediatric web-based prior-approval requests. A standardized data collection form was used to abstract data for each antimicrobial request and the corresponding medical record for each patient (Appendix, Figure 5.2). The form included the patient s age, physical examination results (body temperature and blood pressure within the 24 hours before the request, and other pertinent physical findings), underlying diseases and pertinent treatment (for example, immunosuppressive treatment or presence of a central venous catheter, current antimicrobial treatment), history of allergies, clinical laboratory test results (white blood cell and differential counts, CRP or ESR, microbiological results), radiographic test results, and the patient s location and the level and subspecialty of the physician requesting the antimicrobials. Potential risk factors, such as the patient s age, gender, history of prematurity, duration of hospitalization at the time of the request, location (general pediatric ward, NICU, PICU, or oncology ward), level and subspecialty of the requesting physician (attending physician, house staff fellow or resident, surgical or non-surgical) were recorded (the potential importance of these factors is suggested by a previous study in adult patients). The definition of inaccurate requests was clinically significant discrepancies between communication data elements abstracted from documented requests in the web-based system and the data in the medical record. Clinically significant discrepancies were those judged by the study team (Drs. Tamma, Jenh-Hsu and Lai) to be likely to influence antimicrobial prescribing. The absence of information documented from the prior requests was not defined as inaccuracy. 42

56 The definition of each type of inaccurate requests was described as follows: Laboratory data: including significant discrepancies in hematological, biochemical, microbiological or radiographic data. Diagnosis: the diagnosis recorded in the prior request was directly contradicted by the diagnosis recorded in the medical records. Physical exam or vital signs: clinically significant inaccurate physical exam data or vital signs data within the preceding 24 hours of the prior request. History (present illness and past history): including incorrect information regarding drug allergies or past antimicrobial use, or clinically significant inaccuracies in present or past medical history or in presence or absence of an underlying condition. Age: clinically significant inaccurate age of patient The potential risk factors for inaccurate requests were then explored, including age, gender, primary service, whether the patient was in the ICU, or an oncology or cystic fibrosis patient, types of antimicrobials, presence of underlying disease, prematurity (only in patients aged less than 1 year), number of hospital days before the antimicrobial request, types of infections, types of indications for restricted antimicrobial use, whether requests were submitted in off-hours, and whether the ASP team had previously rejected the request, Office hours were defined as the time period from 8 am to 10 pm each day. Other times were defined as off-hours. After the determination of inaccuracies of the prior-approval requests, the ASP team also tried to understand how these inaccuracies might potentially influence the ID fellow s approval. The definition of inaccurate requests which might influence the approvals of PIDF was that the ASP study team re-evaluated all the information from 43

57 patient s medical records retrospectively and felt that PIDF might not approve the antimicrobials if the accurate information had been submitted instead. Because of the limitations of medical records, we categorized inaccurate requests as influential only in well-defined, uncomplicated scenarios. In the case of complicated scenarios, the assumptions were made that inaccuracies were not influential Statistical Analysis The analysis was also conducted using STATA software version 11 (StataCorp, LP Texas) as described below: Analysis for Proportion of Inaccurate Requests First, frequency counts and percentages were calculated to describe basic demographic, clinical information and service characteristics for all of the accurate requests, inaccurate requests and all requests. Second, t-tests for continuous variables and chi-square tests for categorical variables were undertaken to determine whether there were significant difference in those variables between accurate requests and inaccurate requests. Similarly, frequency counts and percentages were calculated to describe the type of inaccuracy in each inaccurate request Analysis for Potential Risk Factors for Inaccurate Requests The potential risk factors for inaccurate requests were evaluated first in a bivariate logistic regression model. A multivariate logistic regression model was then built with independent variables that had P values less than 0.20 in the bivariate analysis. The independent variables were considered independent risk factors if the P value was <0.05 in the multivariate logistic regression model. The 44

58 associations of independent variables with inaccurate requests in bivariate analyses were reported as odds ratios for comparison with the odds ratios calculated from the multivariate logistic regression model. Point estimates with 95% confidence intervals (CIs) for crude odds ratio and adjusted odds ratio (aor) were also calculated Analysis for Proportion of Inaccurate Requests Which Could be Adversely Affected by Inaccurate Information The frequency count and the proportion of each type of inaccurate request (laboratory data, diagnosis, physical examinations or vital signs, history of illness or drug allergy and age) and the total number of inaccurate requests which could potentially adversely affect the approval of PIDF were calculated to determine the magnitude of the potential effect of inaccurate requests. 45

59 3. Results 3.1. Results for Aim 1 and Demographic Data The pediatric ASP team identified 60 pediatric patients for whom use of restricted antimicrobials was inappropriate at post-prescription review, which occurred on Day 2-4 after the restricted antimicrobial was initiated. These patients were randomized to the intervention group (30 patients) or the control group (30 patients). As shown in Table 3.1, most patients were older than 5 years (66.7%). Although there were fewer infants in the intervention group than in the control group (10.0% of patients 1 year old in the intervention group vs. 20.0% in the control group), this difference was not significant. The proportion of male patients was lower in the intervention groups (46.7 vs %), but this difference was not significant. About 30% of cases belonged to the surgical service and two-thirds (68.3%) of cases were reviewed within 3 days of admission. The majority of patients had underlying chronic diseases (83.3%), and the proportion did not differ between groups. The indications for restricted antibiotic included empiric (n=35, 58.3%), directed therapy (n=19, 31.7%) and prophylaxis (n=6, 10.0%). Most of those who received prophylaxis (5 of 6) were in the control group. The most common type of infection was gastrointestinal (30%). There were no significant differences between the intervention and the control groups with regard to age, gender, clinical service, proportion of patients with underlying disease, season of admission, time to review, indications for antimicrobial use or types of infections being treated. A total of 65 restricted antimicrobial courses in 60 pediatric patients were found to be inappropriate by the ASP team, including 32 inappropriate antimicrobial courses 46

60 in the intervention group and 33 inappropriate antimicrobial courses in the control group. The most frequently restricted antibiotics in the study were beta-lactam antibiotics (40.0%) and vancomycin (24.6%). No significant differences were found between the two groups for these courses of antimicrobials. 47

61 Table 3.1 Demographic data for the intervention group and the control group with patient and antimicrobial course as the units of measure Variable Intervention Control Total P value Patient Course Patient Course Patient Course Patient Course N (%) N (%) N (%) N (%) N (%) N (%) N (number) 30(100) 32(100) 30(100) 33(100) 60(100) 65(100) Age Mean (yrs) yrs 3(10.0) 4(12.5) 6(20.0) 8(24.2) 9(15.0) 12(18.5) yrs 4(13.3) 5(15.6) 7(23.3) 7(21.2) 11(18.3) 12(18.5) >5 yrs 23(76.7) 23(71.9) 17(56.7) 18(54.6) 40(66.7) 41(63.1) Male (%) 14(46.7) 16(50.0) 17(56.7) 19(57.6) 31(51.7) 35(53.9) Surgical service 9(30.0) 10(31.3) 9(30.0) 10(30.3) 18(30.0) 20(30.8) With underlying 26(86.7) 28(87.5) 24(80.0) 27(81.8) 50(83.3) 55(84.6) disease Season at 0.29 antimicrobial review Spring 5(16.7) - 5(16.7) - 10(16.7) - Summer 2(6.7) - 5(16.7) - 7(11.7) - Autumn 16(53.3) - 18(60.0) - 34(56.7) - Winter 7(23.3) - 2(6.7) - 9(15.0) - Days of admission at review days 22(73.3) 23(71.9) 19(63.3) 21(63.6) 41(68.3) 44(67.7) 4-14 days 6(20.0) 6(18.8) 6(20.0) 7(21.2) 12(20.0) 13(20.0) >14 days 2(6.7) 3(9.4) 5(16.7) 5(15.2) 7(11.7) 8(12.3) Type of indication Prophylaxis 1(3.3) 2(6.3) 5(16.7) 5(15.2) 6(10.0) 7(10.8) Empiric 19(63.3) 20(62.5) 16(53.3) 17(51.5) 35(58.3) 37(56.9) Directed 10(33.3) 10(31.2) 9(30.0) 11(33.3) 19(31.7) 21(32.3) Type of infections Respiratory 6(20.0) 6(18.8) 6(20.0) 6(18.2) 12(20.0) 12(18.5) GI tract 9(30.0) 9(28.1) 9(30.0) 10(30.3) 18(30.0) 19(29.2) Sepsis 6(20.0) 8(25) 2(6.7) 3(9.1) 8(13.3) 11(16.9) Prophylaxis 1(3.3) 1(3.1) 5(16.7) 5(15.2) 6(10.0) 6(9.2) Others 8(26.7) 8(25) 8(26.7) 9(27.3) 16(26.7) 17(26.2) Type of antimicrobials Penicillin - 12(37.5) - 14(42.4) - 26(40.0) derivatives a Vancomycin - 8(25.0) - 8(24.2) - 16(24.6) Cephalosporin - 3(6.3) - 2(9.1) - 5(7.7) Fluroquinolone - 6(18.8) - 1(3.0) - 7(10.8) Carbapenem - 1(3.1) - 1(3.0) - 2(3.1) Others - 3(9.4) - 6(18.2) - 9(13.9) a Penicillin derivatives include: piperacillin/tazobactam, ticarcillin/clavulanic acid 48

62 Comparison of antibiotic use (restricted and total) in the two study arms Restricted and total antibiotic use measured as days of therapy per 1000 hospital-days (DOTs) As shown in Figure 3.1, restricted antimicrobial use DOTs were lower in the intervention group than in the control group (DOTs median: 750 vs ); however, this did not reach statistical significance (p=0.923). Although several cases with extended length of stay might have influenced the denominator data for DOTs, the differences between the two groups were not significant even when those outlier cases with high leverage (defined as the value of the point higher than the 3 rd quartile plus 1.5 interquartile ranges) were censored (p= 0.767). In addition, the comparison of total antimicrobial use DOTs showed no significant difference between two groups with or without outliers (p=0.830 and p=0.986, respectively) Inappropriate restricted and total antibiotic use measured as days of therapy per 1000 hospital-days (DOTs) As shown in Figure 3.2, the median of inappropriate restricted antimicrobial use was higher in the intervention group than in the control group (DOTs median: vs days) but this difference was not significant (p=0.306). After removing the outliers (3 cases in each group with extended length of stay), the median days of inappropriate restricted antimicrobial use (DOTs) was lower in the intervention group than in the control group (DOTs median: vs. 625). No statistically significant differences in DOTs for inappropriate restricted antibiotic use (p= 0.379) or in total inappropriate antimicrobial use were observed (data not shown). 49

63 Restricted and total antibiotics use measured as median duration (days) of antimicrobial agents per episode of infection and combined antimicrobials use (days) per episode of infection The use of DOTs as an outcome measure has some limitations. In our study, a single assessment of antibiotic use was performed for each patient regardless of length of stay. In some patients with long hospital stays, antibiotics were prescribed for conditions not evaluated during the initial assessment. Specifically, in our study, there were 7 cases (23.3%) in the intervention group and 8 cases (26.7%) in the control group with other episodes of infections in addition to the episode reviewed. Therefore, if the outcome used is total hospitalization days (as defined in DOTs), reduction of antibiotic use from intervention group during the intervention-specific episode might be missed. For this reason, two other measures were introduced for the comparisons of antibiotic use: the median duration (in days) of restricted antimicrobial use per episode of infection and the combined restricted antimicrobial use (in days) per episode of infection (Figure 3.3). The median duration was shorter in the intervention group than in the control group (median days: 3.5 vs. 5 days) although it did not reach statistical significance (p= 0.094). Similarly, the length of combined restricted antimicrobial use was less in the intervention group than in the control group (median days: 5.5 vs. 6 days); however, there was still no significant differences between two groups in this outcome category, perhaps owing to small sample size (p= 0.320). In the comparison of total antimicrobial use in these two measures, differences were also not significant (data not shown). 50

64 The influence of the timing of review on the median duration (days) of restricted antimicrobial agents per episode of infection (Subgroup Analysis) We hypothesized that earlier review compared to later review by the ASP team might lead to earlier cessation of unnecessary treatment, shorter duration of treatment and fewer days of antibiotic therapy. To examine this, we stratified our analysis of the intervention arm into two categories with similar numbers of patients in each category: early review (Day 2 after initiation, n=14) and late review (Day 3 or Day 4 after initiation, n=16). We found that the median duration of restricted antimicrobial agent use per episode of infection was shorter with earlier review than with later review in the intervention group, though it did not reach statistical significance ( p=0.175 Figure 3.4). 51

65 DOTs/ 1000 pt-days DOTs/ 1000 pt-days 1,000 2,000 3,000 4, DOTs/ 1000 pt-days DOTs/ 1000 pt-days 1,000 2,000 3,000 4,000 Figure 3.1 Comparisons of restricted and total antibiotic use (DOTs), with and without outliers 1,000 1,500 2,000 2,500 Restricted antibiotic use (DOTs) between two arms Total antibiotic use (DOTs) between two arms Control Intervention Control Intervention 1,000 1,500 2,000 2,500 Restricted antibiotic use (DOTs) between two arms Without outliers Total antibiotic use (DOTs) between two arms Without outliers Control Intervention Control Intervention 52

66 0 0 DOTs/ 1000 pt-days ,000 1,500 2,000 DOTs/ 1000 pt-days 1,000 1,500 2,000 Figure 3.2 Comparisons of inappropriate restricted antibiotic use (DOTs), with and without outliers Inappropriate restricted antibiotic use (DOTs) between two arms Inappropriate restricted antibiotic use (DOTs) between two arms Without outliers Control Intervention Control Intervention 53

67 0 5 Days Figure 3.3 Comparison of median duration (days) of restricted antibiotics and combined restricted antibiotic use (days) per episode of infections between two groups Median duration of restricted antibiotic use per patient per episode of infection between two arms Control Intervention Restricted antibiotic use per episode of infection between two arms Control Intervention 54

68 0 5 Figure 3.4 Comparisons of median duration of restricted antibiotics (days) per patient per episode of infection in different review timing in the intervention group Median duation of restricted antibiotic use per episode of infection by review timing Day 2 Day N=14 N=16 55

69 Reasons for inappropriate antimicrobial use in two groups Table 3.2 described the reasons for inappropriate antimicrobial use in descending order of frequency. The most common reason was viral infection or no evidence of infection (40%). Overlapping double coverage of antimicrobial agents (13.9%) and susceptible to narrower agent were also frequently found as reasons for inappropriate antimicrobial use. There were no significant differences in reasons for inappropriate antimicrobial use between the intervention and control groups. 56

70 Table 3.2 Reasons of inappropriate antimicrobial use (in descending order) Reason of inappropriate Intervention Control Total p-value Antimicrobial use N (%) N (%) N (%) Viral Infection/No infection 12(37.5) 14(42.4) 26(40.0) Double coverage 3(9.4) 6(18.2) 9(13.9) Susceptible to narrower agent 4(12.5) 4(12.1) 8(12.3) Others 4(12.5) 2(6.1) 6(9.3) Inappropriate duration 4(12.5) 1(3.0) 5(7.7) Bug/drug mismatch 2(6.3) 1(3.0) 3(4.6) Colonization 1(3.1) 2(6.1) 3(4.6) Dose inappropriate 1(3.1) 2(6.1) 3(4.6) Prolonged surgical prophylaxis 0 1(3.0) 1(1.5) Require broader agent 1(3.1) 0 1(1.5) Total 32(100) 33(100) 65(100) 57

71 Proportion of inappropriate antibiotic use on Days 2 and 3 after ASP team review The ASP intervention had a significant impact on reduction of courses of inappropriate antibiotics. At Day 2 after the ASP team s review, the prevalence of inappropriate antibiotic use was significantly lower in the intervention group than the control group (34.4% vs. 75.8%, p=0.001, Table 3.3). At Day 3 after review, 12.0% (3/25) of the antimicrobial courses in the control group had been corrected by the clinical team. However, the effect of the intervention was still significant at Day 3 after the ASP team s review (31.3% vs. 66.7%, p=0.006, Table 3.3). In the intervention group, approximately one third (n=10, 31.3%) of antimicrobial courses still met the definition for inappropriate use at Day 3 after the ASP team s review. Among these antimicrobial courses, the most common instances of inappropriate use were a 10 day course of treatment for acute tracheitis rather than the recommended 5 day course ( 3 cases) and unnecessary double coverage of anaerobic pathogens (such as piperacillin/tazobactam and metronidazole) in another 3 cases The influence of timing of ASP review on the proportion of inappropriate antimicrobial courses Early ASP review (review at Day 2 after the restricted antibiotic was initiated) and later ASP review (review at Day 3 or Day 4 after the restricted antibiotic was initiated) both had a significant impact on the number of courses of inappropriate antimicrobials when assessed 1 day after the ASP team s review (p= and p=0.016) (Figure 3.5). Importantly, there was no significant difference in reduction in inappropriate courses of antimicrobials in the intervention arm when ASP review was conducted at Days 2 or Day 3-4 (p=1.000) (Figure 3.5). 58

72 Table 3.3 Comparisons of the proportions of inappropriate antimicrobial courses at Day 2 and Day 3 after post-prescription review between two groups (unit of analysis: antimicrobial course) Time Intervention Control Prevalence ratio [95% CI] P value group Group (Intervention/Control) Day % (11/32) 75.8%(25/33) [0.271,0.760] Day % (10/32) 66.7% (22/33) [0.266,0.827]

73 Percentage Figure 3.5 The influence of review timing after the antibiotic was initiated on the proportion of inappropriate antimicrobial courses Inappropriate antimicrobial course(%) by review timing Intervention Control Day Day

74 Potential factors associated with inappropriate antimicrobial courses at Day 2 after the ASP team s review In the entire cohort of 60 patients include in this study, 33 were receiving inappropriate antimicrobials at Day 2 after the ASP team s review (Table 3.4). Patients receiving inappropriate antimicrobials were older, more likely to be on the medical service, more likely to be receiving antimicrobial prophylaxis, and more likely to have GI infections than those not receiving inappropriate antimicrobials courses at the time of follow-up. However, none of these factors reached statistical significance. ASP intervention was the only significant independent variable associated with appropriate antimicrobial course (p=0.009, Table 3.4) Rate of Compliance with the Recommendation at Day 2 and Day 3 after the ASP team s review In some instances, more than one antibiotic recommendation was made or recorded for each patient. Overall, there were 37 recommendations made by the ASP team for 32 patients in the intervention group. In addition, 35 recommendations were recorded in the data collection forms for 33 patients in the control group, but these were not communicated (Table 3.5). Stopping therapy, (either eliminating overlapping antibiotic therapy or stopping therapy because there was no evidence of infection) was the most frequent recommendation (55.6%). Modifying therapy was the next most frequent recommendation (26%), and included narrowing of the antibiotic spectrum (11.2%), broadening of the antibiotic spectrum (2.8%), adjustment of dosage (5.6%) or shortening the duration of antibiotic treatment (5 courses, 6.9%). All 5 recommendations for shortened therapy were suggestions of 5 days of therapy for 61

75 acute tracheitis instead of 10 days. In 18% of recommendations, the ASP team recommended clinical consultation with the pediatric infectious disease team because the patient had a complicated infection or the choice of antimicrobials was complex. Overall, the average time spent in communicating with the primary team member was 11.8 minutes in the intervention group. Overall, the compliance rate at Day 2 after post-prescription review in the intervention group (the physicians changed orders according to the recommendations of ID fellows) was significantly higher (67.6% vs. 22.9%) than in the control group (the physicians auto-corrected without prompting by the ASP team) (p<0.001, Table 3.6). Specifically, therapies were stopped significantly more frequently in the intervention group (58.8%) than in the control group (21.7%) (p=0.024). Modifying therapy and ID consults both were changed more in the intervention group than in the control group; however, they did not reach statistical significance (p=0.170 and p= 0.052) probably due to the relatively small sample size. In the intervention group, the compliance rate was highest for ID consult (88.9%). The compliance rate was a little lower for modifying therapy (63.6%) and for stopping therapy (58.8%) Outcomes of patients when ASP team recommended alternative empiric therapy or stopping therapy in two arms Thirty-six antimicrobial courses for 35 patients were further followed up by medical chart review (Table 3.7). Specifically, the ASP team made 4 recommendations for alternative empiric therapies (broadened or narrowed therapy) for the intervention group and recorded 5 alternative therapies for the control group. All the alternative empiric therapies covered the subsequent culture results. In cases for which the ASP 62

76 team recommended stopping antimicrobial therapy because there was no clear indication of antimicrobial use, no patient developed an infection within 48 hours after cessation of antibiotics. 63

77 Table 3.4 Potential factors associated with inappropriate antimicrobial courses at Day 2 after the ASP team s review (Unit of analysis: patient) Variable Inappropriate antibiotic courses No. of patient (%) Appropriate antibiotic courses No. of patient (%) P value N (no. of patients) Age yrs 4(12.1) 5(18.5) 1.1-5yrs 6(18.2) 5(18.5) >5 yrs 23(69.7) 17(63.0) Male (%) 18(54.6) 13(48.2) Surgical service 8(24.2) 10(37.0) With underlying disease 29(87.9) 21(77.8) Season at antimicrobial review Spring 4(12.1) 6(22.2) Summer 3(9.1) 4(14.8) Autumn 21(63.6) 13(48.2) Winter 5(15.2) 4(14.8) Days of admission at review 1-3 days 20(60.6) 21(77.8) 4-14 days 8(24.2) 4(14.8) >14 days 5(15.2) 2(7.4) Type of indication Prophylaxis 4(12.1) 2(7.4) Empiric 17(51.5) 18(66.7) Directed 12(36.4) 7(25.9) Type of infections Respiratory 8(24.2) 4(14.8) GI tract 12(36.4) 6(22.2) Sepsis 2(6.1) 6(22.2) Prophylaxis 4(12.1) 2(7.4) Others 7(21.2) 9(33.3) Intervention 11( 33.3 ) 19( 70.4 ) 0.009* *p<

78 Table 3.5 Recommendations recorded in the data collection forms by the ASP team for two groups Recommendations Intervention N (%) Control N (%) Total N (%) Stopping therapy 17( 45.9) 23(65.7) 40(55.6) Duplicated therapy eliminated 7(18.9) 6(17.1) 13(18.1) No evidence of infection 10(27.0) 17(48.6) 27(37.5) Modifying therapy 11(29.7) 8(17.2) 19(26.4) Narrowed antibiotics 3(8.1) 5(14.3) 8(11.1) Broadened antibiotics 2(5.4) 0 2(2.8) Shortened duration of therapy 4(11.8) 1(2.9) 5(6.9) Adjusted dose 2(5.4) 2(5.7) 4(5.6) ID consult 9(24.3) 4(12.4) 13(18.0) Total 37(100) 35(100) 72(100) 65

79 Table 3.6 Comparisons of changes noted at Day 2 after post-prescription review between intervention and control groups (N_change/N_total (%)) Intervention arm Control arm Total P value Stopping therapy 10/17 (58.8) 5/23(21.7) 15/40(37.5) 0.024* Modifying therapy 7/11(63.6) 2/8(25.0) 9/19(47.4) ID consult 8/9(88.9) 1/4( 25.0) 9/13(69.2) Total cases 25/37(67.6) 8/35(22.9) 33/72(45.8) <0.001 *p<

80 Table 3.7 Outcome of patients when ASP recommended alternative therapy or no therapy Outcome Intervention group Control group Number of alternative therapy (broadened or narrowed) Recommended Alternative therapy used 24 hrs after review Positive cultures 2 3 ASP recommendation covered positive cultures Number of no indication of antimicrobials recommended Therapy stopped within 24 hrs after ASP team s review Developed subsequent infection 0 0 * *In the control group, the outcome was only followed up in the 4 cases in which the antimicrobial was stopped noted at Day 2 after ASP team s review 67

81 3.2. Results for aim Demographic data There were a total of 1159 pediatric prior-approval requests for use of restricted antimicrobials in the 4 month study period (Table 3.8). Inaccuracies (discrepancies between requests and medical records) occurred for 8.7% (95% CI, 7.2%-10.5%) of all requests. Most patients were >5 years old (53.1%), had underlying disease (76.8%), were on the medical service (87.3%), were not oncology patients (78.0%), not cystic fibrosis patients (88.9%) and not ICU patients (64.3%). Most requests were submitted in the first 2 days following admission (52.3%), during normal office hours (8AM-10 PM; 85.4%) and under the indication of empiric therapy (53.0%). Respiratory tract infections (25.7%) and sepsis (23.9%) were the most frequent infections encountered. The antibiotic most frequently requested was vancomycin (32.5%) Types of inaccurate requests and examples The percentages of requests by type of inaccuracy are shown in Table 3.9. The most common types of inaccuracy were errors in laboratory data (34.6%) and in patient history (23.8%). In Table 3.10, some examples of each type of inaccuracy are shown. 68

82 Table 3.8 Basic demographic and clinical data in the pediatric prior-approval requests (unit of analysis: antimicrobial request) Categories Accurate Inaccurate Total P value N (%) N (%) N (%) Number of requests 1058(91.3) 101 (8.7) 1159(100) Age Mean (yrs) yr 300(28.4) 38(37.6) 338(29.2) yr-5 yrs 191(18.0) 14(13.9 ) 205(17.7) 5.1yrs-21yrs 567(53.6) 49(48.5) 616(53.1) With underlying diseases 814(76.9) 76(75.3) 890(76.8) No underlying disease 244(23.1) 25(24.7) 269(23.2) Surgical service 127(12.0) 20(19.8) 147(12.7) Non-surgical service 931(88.0) 81(80.2) 1012(87.3) Oncology patient Non-oncology patient Cystic fibrosis patient 244(23.1) 814(76.9) 118(11.2) 11(10.9) 90(89.1) 11(10.9) 255(22.0) 904(78.0) 129(11.1) Non-cystic fibrosis patient 940(88.0) 90(89.1) 1030(88.9) Days of admission at request 1-2 days 545(51.5) 62(61.4) 607(52.3) days 168(15.9) 10(9.9) 178(15.4) 8 days 345(32.6) 29(28.7) 374(32.3) Off hours( 10pm-8am) 174(16.5) 18(17.8) 192(16.6) Office hours 884(83.5) 83(82.2) 967(85.4) ICU patients 367(34.7) 47(46.5) 414(35.7) Non-ICU patients 691(65.3) 54(53.5) 745(64.3) Rejected by ID fellows 144(13.6) 18(17.8) 162(14.0) Approved by ID fellows 914(86.4) 83(82.2) 997(86.0) Type of indication Prophylaxis 151(14.3) 21(20.8) 172(14.8) Empiric 561(53.0) 53(52.5) 614(53.0) Directed 346(32.7) 27(26.7) 373(32.2) Types of infections Respiratory 273(25.8) 25(24.7) 298(25.7) GI tract 106(10.0) 9(8.9) 115(9.9) Sepsis 263(24.9) 14(13.9) 277(23.9) Prophylaxis 149(14.1) 22(21.8) 171(14.8) Others 267(25.2) 31(30.7) 298(25.7) Antimicrobial type Penicillin derivatives a 117( 11.1) 12(11.9) 129(11.1) Vancomycin 341(32.2) 36(35.6) 377(32.5) Cephalosporin 147(13.9) 17(16.8) 164(14.2) Fluoroquinolone 64(6.0) 6(5.9) 70(6.0) Carbapenem 77(7.3) 3(3.0) 80(6.9) Antifungals 134(12.7) 7(6.9) 141(12.2) Others 178(16.8) 20(19.8) 198(17.1) a Penicillin derivatives included: ampicillin/sulbactam, piperacillin/tazobactam, ticarcillin/clavulanic acid 69

83 Table 3.9 The frequencies of different types of inaccuracies among the inaccurate requests Types N (%) Laboratory data 35 (34.6) History(Present illness and past history) 24 (23.8) Age 21 (20.8) Diagnosis 12 (11.9) Physical exam/vital signs 9 (8.9) Total 101(100) 70

84 Table 3.10 Examples of different types of inaccuracies of prior-approval requests and the potential influence upon PIDF approval Types From requests From medical chart Potential influence Laboratory data Recent positive culture with pseudomonas infection, resistance to amikacin, ciprofloxacin No such resistance data was documented; most recent laboratory data showed pseudomonas sensitive to amikacin, ciprofloxacin Requested ceftazidime for treatment for pseudomonas. PIDF might not approve since the data had shown that the organism was susceptible to amikacin History(Present illness and past history) Age Diagnosis Physical exam/vital signs Post-surgical infection Neonate; persistent fever Diagnosed as cellulitis Patient was febrile with seizure activity No infection was documented; the chart mentioned post-operative antimicrobial prophylaxis Patient was already 16 months old No skin rash or edema was noted; afebrile Afebrile; a case of occipital skull fracture with seizure activity Requested piperacillin/tazobactam for post-surgical infection. PIDF might not approve since no infection was documented Requested vancomycin and piperacillin/tazobactam for suspected neonatal sepsis. PIDF might not approve broad-spectrum antibiotic use since the patient was 16 months old Requested vancomycin for cellulitis. PIDF might not approve because the case was consistent with a viral syndrome based on clinical and lab findings. Requested vancomycin for meningitis. PIDF might not approve because the patient had no fever and the seizure might be due to occipital skull fracture with subdural hematoma found in CT scan 71

85 Potential Factors Related to Inaccuracy of Antimicrobial Requests Crude odds ratio In bivariate analyses (Table 3.11), patients on the surgical service, not on the oncology service, or in the ICU were each significantly more likely to have inaccurate antimicrobial requests (p=0.029, p=0.004, p=0.022, respectively). Infants (ages 0-1 year) and patients with prophylaxis as the indication of restricted antimicrobials were also more likely to have inaccurate antimicrobial requests; however, neither reached statistical significance (p= and p=0.092) Adjusted odds ratio A multivariate logistic regression model was built with independent variables that had a P value less than 0.20 in the bivariate analysis. The results showed that the patients on the surgical service (adjusted OR=2.087), in the ICU unit (adjusted OR=1.629), and non-oncology patients remained significantly more likely to have inaccurate antimicrobial requests (p=0.011, p=0.043, p=0.036, respectively). Additionally, prophylaxis as an indication for restricted antimicrobial use was also significantly more likely to be associated with inaccurate requests in the multivariate logistic regression model (adjusted OR=1.719, p=0.044) (Table 3.12) Subgroup Analysis As shown in table 3.11, requests for use of restricted antimicrobials in infants were more likely to be inaccurate with borderline significance (p=0.052). For this reason, we performed a post-hoc stratified analysis by age (> 1 year old and 1 year old) to determine whether the potential risk factors held true in each age category. As shown in Table 3.13, patients over 1 year of age who were either on 72

86 the surgical service, not on the oncology service,or in the ICU were still significantly more likely to have inaccurate antimicrobial requests (p=0.017, p=0.019, p=0.017, respectively ) in the multivariate logistic regression model. In this age group, requests with prophylaxis as the indication for restricted antimicrobials was no longer significantly associated with inaccurate requests (p=0.097). In contrast, none of the above potential risk factors were significantly associated with the inaccurate requests in infants aged 1 year old (Table 3.13). 73

87 Table 3.11 Bivariate analysis of potential risk factors of inaccurate requests Variable Accurate N (%) Inaccurate N (%) Odds Ratio Number of requests 1058(91.3) 101(8.7) Age 0-1yr yrs 300(88.8) 758(92.3) 38(11.2) 63(7.7) [0.997,2.329] With underlying diseases No underlying disease Surgical service Non-surgical service Oncology patient Non-oncology patient Cystic fibrosis patient Non-cystic fibrosis patient Days of admission at request 814(91.5) 244(90.7) 127(86.4) 931(92.0) 244(95.7) 814(90.0) 118(91.5) 940(91.3) 76(8.5) 25(9.3) 20(13.6) 81(8.0) 11(4.3) 90(10.0) 11(8.5) 90(8.7) [0.567,1.464] [1.073,3.055] [0.215,0.775] [0.506,1.874] P value * 0.004* days 713(90.8) 72(9.2) days 345(92.2) 29(7.8) [0.766,1.884] Off hours( 10pm-8am) 174(90.6) 18(9.4) Office hours 884(91.4) 83(8.6) [0.645,1.881] ICU patients 367(88.7) 47(11.3) * Non-ICU patients 691(92.8) 54(7.2) [1.087,2.472] Rejected by ID fellows 144(88.9) 18(11.1) Approved by ID fellows 914(91.7) 83(8.3) [0.803,2.360] Type of indication Prophylaxis 151(87.8) 21(12.2) Non-prophylaxis 907(91.9) 80(8.1) [0.946,2.627] * p<

88 Table 3.12 Multivariate analysis of risk factors for inaccurate requests Variable Oncology patient (Oncology vs. non-oncology) Surgical service (Surgical vs. non-surgical) ICU patient (ICU vs. non-icu) No. of Prior Requests Percentage of inaccuracy (%) Adjusted Odds ratio ( 95% CI) [0.246,0.952] [1.180,3.691] [1.016,2.614] Adjusted p value Prophylaxis as indication for Restricted antimicrobials (Prophylaxis as indication vs. Other indications) [1.016,2.910]

89 Table 3.13 Odds ratio (OR), adjusted odds ratio (aor) and adjusted p value for risk factors of inaccuracies aged 1 year and 1 year old Variable Oncology patient vs. non-oncology Surgical service vs. non-surgical Age ( yr) No. of Prior Requests Percentage of inaccuracy (%) OR aor adjusted p value [0.295,3.616] [0.166,0.754] [0.462,4.342] [0.289,4.086] [0.177,0.854] [0.432,5.623] * [1.170,3.916] [1.150,4.122] 0.017* ICU patient vs. non-icu [0.486,1.910] [0.495,2.550] [1.108,3.262] [1.128,3.506] 0.017* Prophylaxis as indication for Targeted antimicrobials vs. non-prophylaxis * p< [0.715,3.389] [0.744,2.929] [0.717,3.420] [0.896,3.736]

90 Types of inaccurate requests and potential influences on the approvals of ID fellows We found that incorrect information could have potentially affected the ID fellows approval in about 45% of the cases (95% CI, 34.7%-54.8%) (Table 3.14). Some examples are shown in Table In specific type of inaccuracies, inaccuracies in diagnosis or patient history were more likely to influence approval decisions of approvals than inaccuracies in laboratory data (p < 0.05). Age appeared to be a less important factor Subgroup Analysis When considering patients aged 1 year old (Table 3.15), most inaccuracies (76.3%) were judged to be non-influential (95% CI, 59.8%-88.6%). Specifically, most inaccuracies occurred in patient history or patient age. Of the age discrepancies, only 5.9% (95%, 0.2%- 28.7%) were thought to potentially influence the PIDFs approvals. Among these 17 cases of age discrepancies, 7 cases were requests for palivizumab use and were not influential because they were either preterm babies or suffered from congenital heart disease and were thus eligible for palivizumab use at the time of the request. In contrast, for patients aged >1 year old (Table 3.15), inaccuracies were more likely to potentially influence the PIDF s approval (57.1%) (95% CI, 44.0%- 69.5%). Most inaccuracies occurred in laboratory data and patient history. Of the inaccuracies in patient history, about 81.3 %( 95% CI, 54.4%-96.0%) were judged to potentially influence the PIDF s approval. 77

91 Table 3.14 The potential influence of inaccuracies on approval by PIDF Types Influence No influence Total P value N (%) N (%) N (%) Laboratory data 14(40.0) 21(60.0) 35(100) --- Diagnosis 9(75.0) 3(25.0) 12(100) Physical exam/vital signs 3(33.3) 6(66.7) 9 (100) History(Present illness 17(70.8) 7 (29.2) 24 (100) and past history) Age 2 (9.5 ) 19(90.5) 21(100) Total 45(44.6) 56(55.4) 101(100) 78

92 Table 3.15 Types of inaccuracies in patients aged 1 year old and > 1 year old Types Influence N (%) No influence N (%) Total N (%) 1 yr > 1 yr 1 yr > 1 yr 1 yr > 1 yr Laboratory data 1(20.0) 13(43.3) 4(80.0) 17(56.7) 5(100) 30(100) Diagnosis 1(100) 8(72.7) 0 3(27.3) 1(100) 11(100) Physical exam/vital signs 2(28.6) 1(50.0) 5(71.4) 1(50.0) 7(100) 2(100) History(Present illness 4(50.0) 13(81.3) 4(50.0) 3(18.7) 25(100) 16(100) and past history) Age 1(5.9) 1(25.0) 16(94.1) 3(75.0) 17(100) 4(100) Total 9(23.7) 36(57.1) 29(76.3) 27(42.9) 38(100) 63(100) 79

93 4. Discussions and Recommendations 4.1. Discussion Only a few previous studies have explored the effectiveness of pediatric stewardship programs in reducing the amount of antibiotic use and reducing the proportion of inappropriate antibiotic courses. Our study, an intervention with a randomized, controlled design, was focused on determining the effectiveness of post-prescription review. Our study showed a significantly lower proportion of inappropriate antibiotic courses in the intervention group than in the control group at Day 2 (p=0.001) and Day 3 (p=0.006). Auto-correction of antimicrobial therapy occurred in a small number of cases in the control group (12% or 3 courses out of 25 courses) from Day 2 to Day 3. We also did not find adverse outcomes associated with post-prescription review. These two findings demonstrate the utility of postprescription review for enhancing appropriate antimicrobial use in pediatric patients. Approximately one third of antimicrobial courses (31.3%) in patients in the intervention group met the definition for inappropriate antimicrobial use at Day 3 after the ASP team s review. It is important to consider why almost a third of the ASP recommendations were not followed in the intervention group. The most common recommendations that were not followed were the prolonged treatment for tracheitis and unnecessary double coverage of anaerobes. Why these specific recommendations were not followed is uncertain, but lack of knowledge of recent literature, 48 the perception that the antimicrobials are rarely harmful, 74 diagnostic uncertainty, the fear of the failure to treat a treatable infection, and the absence of clear guidelines might have been contributing factors. 112 Our study demonstrated a statistically significant reduction in the proportion of inappropriate treatment courses, but not in DOTs or median duration of therapy. The most common choices for the measures of antibiotic use include defined daily dose (DDD) and days 80

94 of therapy (DOT). DDD is defined as the assumed average maintenance dose per day of a drug used for its main indication in a 70-kg adult. WHO currently uses DDDs methodology to measure antimicrobial use. 113 DDDs are normalized in most studies to 1000 patient-days to control for difference in hospital census. However, DDD is not appropriate in analyzing drug use in children because the maintenance dose varies significantly in children depending on age and weight. Instead, the DOT measurement is preferred for pediatric populations because it is independent of age and body weight difference. 113 One DOT is defined as the administration of a single agent on a given day regardless of the number of the doses administered or dosage strength. 114 It is also often normalized to 1000 patient-days. Because of the above reasons, we used DOTs instead of DDDs as one of the measures of antibiotic use in our study. Total DOT is an attractive outcome measure because it includes all antimicrobial use in every episode of infection and can easily be used as a benchmark for comparisons in different institutions because it is independent of differences in restricted antimicrobials, and definitions of inappropriateness of antibiotics. However, given our study design, assessment on a per episode for this study might have been more useful than DOTs because additional episodes of infections could have contributed to DOTs but were not assessed in our study. Therefore, median duration (days) of antimicrobial agents per episode of infection and combined antimicrobial use (days) per episode of infection was a better measure for our study. In addition, it was important to measure total antibiotic use in additional to restricted antibiotic use because the reduction of study agents might result in increased used of non-restricted antimicrobials. 75 There are several possible reasons why were unable to show differences in DOTs or median duration of therapy with the use of post-prescription review. First, it is difficult to detect small but meaningful reductions with the relatively small sample size that we had in our 81

95 study (30 cases in each arm). Using the calculations derived by Noether, 115 we estimated the sample size needed for each group to measure various antimicrobial use outcomes (Table 4.1). As shown, our sample size was sufficient to measure significant changes in the proportion of inappropriate antibiotic courses, but not for other outcome measures. For some measures, such as median duration (days) of restricted antibiotic agents per episode of infection, the sample size is small enough that a study to measure this outcome might be conducted in a single large center. However, other outcome measures, such as DOTs, probably need to be applied to multicenter studies. Second, all of the cases included were receiving broad-spectrum restricted antibiotics and often had complex medical problems. Although the basic demographic and clinical information was similar in two groups in our study, there might be some unmeasured confounders differentially distributed in two groups, especially in pediatric patients in a tertiary center, such as different pre-existing medical conditions with various severities, which could influence antimicrobial use. Third, although the post-prescription review program can be an effective practice to reduce antimicrobial use, it takes time to buy into the process 74, and the program which was still in its earliest stages at the time of our study. Fourth, more than half of the recommendations in the intervention group were to modify therapy or obtain an ID consult, which might not necessarily reduce the amount of the antibiotic use although they might improve the appropriateness of the antibiotic use. Fifth, our study was limited to a single post-prescription review, and some studies have shown that additional reviews provide more opportunities to review the antibiotic use, which might lead to greater impact. 87 With regard to the timing of post-prescription review, we hypothesized that earlier review might be more likely to lead to antimicrobial cessation or modification. However, we were unable to demonstrate this in our subgroup analysis. This may be because of small 82

96 sample size and might have to be evaluated in a larger study. However, it is helpful to know that intervention at Day 3 was still associated with a significant reduction in inappropriate courses of antibiotics. The compliance rate (66.7%) in the intervention group was similar to some other studies, 74,81 ; however, higher compliance rates have been documented in several studies. 87,102,106 For example, a study of a post-prescription review program in a pediatric hospital demonstrated a compliance rate of 92%. 106 However, most of their recommendations were for dose adjustment which might be easier for treatment physicians to accept. Another study in the adult ICU setting with post-prescription review twice for the targeted antimicrobials (3 rd and 10 th day of the therapy) with ID physicians approved every identified inappropriate case and making suggestions also showed high compliance rate (82%). 87 The recommendations after 10 days of therapy might also be more convincing because more clinical and microbiological information was available. Besides, the expertise and trust provided by ID attending physician might be better than by ID fellows. The authors found that active interaction with the treatment team from the early stages of ASP program planning also played a major role for their success. Finally, a survey found that the prescribing etiquette could also have major influences upon the compliance rate. 116 If the ASP team did not communicate directly with the leader of the treating team attending physician, as seen in our study, the compliance might not be very high. We found that the compliance rate with the recommendation to stop therapy was lower compared to the other 2 categories in the intervention group. Several potential reasons for this finding are listed below. First, in our study, no evidence of infection and double coverage of certain pathogens were the most common reasons for inappropriate antimicrobial use. The suggestions of stopping therapy therefore comprised the most frequent recommendations with the greatest statistical power. Second, the reluctance to stop rather than modify 83

97 antimicrobials reflects the discomfort that some prescribers have with stopping therapy if a patient has improved on therapy, even when an infection etiology is not identified. 74 Furthermore, the physicians might perceive that antibiotics rarely harmful or might have difficulty in acknowledging the undesirable consequences (such as bacterial resistance) because the prescription and the consequence could be so widely separately in time. These also contributed to the lower acceptance rate of stopping therapy. 74,117 In order to modify this kind of physician s behavior, the incorporation of rapid diagnostic tests such as using low levels of procalcitonin (low likelihood of severe bacterial infection) to guide the discontinuation of antibiotics, 118 additional studies supporting shorter duration of therapy, particularly in the pediatric population, 119 additional studies that demonstrate the improved patient outcome, and campaigns to raise awareness of the problem of bacterial resistance might be useful. 117 Consistent with other studies, 102 our study showed no antibiotic treatment failure and no inadequate coverage when the ASP team recommended narrowed therapy or cessation of therapy (Table 3.7). We chose these outcome measures for a number of reasons. Few studies have tried to study the outcome of mortality rate as the effect of ASP although it is the most objective measure, 120 because fortunately, mortality is relatively rare in children. Therefore, microbiological treatment failure, as in our study, might be considered. In addition, readmission rate, need for more advanced care (such as need for ICU care, cardiovascular or respiratory support) could be potential patient outcomes if the sample size is adequate. 40 Antimicrobial resistance is the most difficult outcome measure since it often takes a long time to develop. The available data are often not patient-specific and thus demonstrate weaker association

98 In aim 3, the results showed that inaccuracy (discrepancy between requests and medical records) occurred in 8.7% of all requests, most of which were not discovered by the pediatric ID fellows. Encouraging fellows to access medical records for specific requests or indications (for example, vancomycin constituted 35.6% of all the inaccuracies in our study) might be feasible. The inaccuracy rate (8.7%) that we observed was less than another study evaluating adult patients (39%). 78 There may be several reasons for this. First, our web-based prior approval system might have fewer inaccuracies than phone communications since the contents of the requests are all documented and the prescribing physicians might be less inclined to provide inaccurate information in the documented forms. Second, web-based systems might decrease the opportunity for ID fellows to acquire specific clinical information through instant communications, 47 and thus might have less chance to acquire inaccurate information. In addition, the operation definitions of inaccuracies might not be the same in different studies. Patients on the surgical service, in the ICU unit, non-oncology patients and those with prophylaxis as indication for antimicrobials were significantly more likely to have inaccurate antimicrobial requests in a multivariate logistic regression analysis (p<0.05). Our findings are consistent with a study in adults, which showed that calls from surgical services were also more likely to have inaccurate communications. 78 There are some possible explanations for the findings in our study. First, PIDF and medical house staff work together more frequently than PIDF and surgical house staff because of the rotation system, which might lead to more accurate descriptions of the patient data by the pediatric housestaff. Secord, steeper hierarchy among surgeons might cause surgical residents to feel pressured to obtain antimicrobials. 78 In addition, most antimicrobial requests in oncology patients in the Johns Hopkins Children s Center were based on algorithms which were less likely to be inaccurate (data not shown). 85

99 Based on our finding that some risk factors for inaccurate requests were distributed differently in patients age 1 year old and > 1 year old,, we conducted a subgroup analysis and found that age acted as an effect modifier for several risk factors (Table 3.13), although the interactions between these risk factors and age were not statistically significant as measured by a likelihood test using multivariate logistic regression (p=0.13, p=0.22 and p=0.89 for nononcology service, ICU service and surgical service interaction with age, respectively). Providing an overall estimate would have masked this heterogeneity Strengths and Limitations of the Study Our study used a prospective, patient-level randomized controlled study design to explore the benefit of the post-prescription review intervention. A prospective study has a number of advantages over a retrospective review. 102 The randomized controlled design meant that the ASP team was not able to choose cases with easy interventions because they did not know the group assignment when they reviewed the cases, which should help to avoid overestimation of the intervention effects. Weaknesses of a patient-level randomization study design are that it is logistically difficult and time-consuming for the study team, as well as the possibility of a contamination effect if the treating physicians simultaneously take care of patients from two groups and recommendations for the intervention group influence physicians antimicrobial use in treating the patients in the control group. In our intervention study, there were total of 39 attending physicians taking care of 60 patients. The rate of autocorrection in the control group was higher in instances in which the attending physician had ever cared for cases from the intervention group (4 changes out of 13 recommendations, 30.8%) than when they had not (4 changes out of 22 recommendations, 18.2%), although this 86

100 difference was not statistically significant (p=0.391). In theory, this could potentially be evidence of a contamination effect, which would bias the intervention effect toward the null. A cluster randomized controlled study, including randomization of different units or hospitals to receive the intervention or not, is less likely to produce a contamination effect. However, it is difficult to randomize these clusters because the units or hospitals may be fundamentally different which could result in confounding or effect modification. 40 Unlike a randomized controlled design, a quasi-experimental design comparing outcomes before and after the intervention is simple and quick to implement, but it could be influenced by maturation effects. 67 An interrupted time series approach could help alleviate such confounding, although it still could be difficult to determine whether a change noted is due to the intervention or to other factors, and it is more time consuming. A cross-over design could also decrease the confounding because each patient or unit serves as its own control, but the carryover effect from intervention could influence the effect of the intervention if the washout period is not long enough. For aim 3, the web-based prior approval programs were a well-documented source of information for comparisons of discrepancies. Studies using the data from the web-based information system are likely to have fewer abstraction errors than studies from phone-based prior approval programs. There are several potential limitations in our study. First, only one hospital was involved in our study, and the Johns Hopkins Children s Center had already had a very successful web-based prior-request system. These factors could limit the study s generalizability to other institutions. In addition, we excluded ICU, CF, oncology and ID consult patients from our intervention although these patients might receive the most restricted 87

101 antimicrobials. These special populations are especially vulnerable to infections and the empiric therapy for them may tend to be broader spectrum antimicrobials; 40 also, some institutional algorithms (such as in oncology or CF patients, antibiotic cycling in NICU) might preclude meaningful ASP intervention. Only restricted antimicrobials were reviewed for the purposes of our study, so we could have missed inappropriate use of unrestricted antibiotics. Also, a single review was performed for each patient in our study, and a convenience sample of cases was used. We therefore cannot compare the intervention rate to other studies because we did not systematically review the cases; however, our approach might reflect the reality of non-study situations since even the most established ASP cannot guarantee intervening in every possible case. 74 For aim 3, the medical record may have been incomplete, leading to misclassification of the accuracy of the submitted request. However, we did not count inadequate provision of patient s data as an inaccuracy. Therefore, it likely biased toward recognizing fewer inaccuracies if misclassification occurred. 78 Besides, using more objective definitions of inaccuracies in our study might decrease the potential of misclassifications. We also did not check the accuracy of requests throughout an entire academic year because of time constraints. Such a study might be useful to determine whether housestaff experience makes a difference in the proportion of inaccurate requests Recommendations for Future Study Future studies might incorporate new laboratory technologies to enhance the ASP program. For example, it might be possible to initiate appropriate antimicrobial therapy earlier if the ASP incorporates new technology such as MALDI-TOF for rapid species-level 88

102 identification of pathogens. 121 Use of some biomarkers such as procalcitonin could guide the initiation and discontinuation of antibiotics. 118 Multicenter collaborative studies could increase statistical power and generalizability. However, comparisons across centers might also have some limitations such as different ASP structure, team personnel, list of restricted drugs, etc. Risk adjustment by focusing on specific inpatient populations and the addition of disease severity indexes could allow comparisons of antimicrobial use across different settings. 40 Similarly, more focused interventions such as decreasing the duration of therapy in specific infections may make it more likely to observe an effect, 40 such as studying minimal acceptable duration of therapy in community-acquired pneumonia (CAP) or urinary tract infection (UTI). In addition, to understand the potential influence of inaccurate communication upon the PIDF s approval, future studies of the associations between inaccuracies of requests and clinical outcomes might be helpful Conclusions We demonstrated that a post-prescription review program could successfully decrease the number of inappropriate antimicrobial courses at our institution. These findings might encourage other pediatric centers to pursue similar post-prescription review programs. Although inaccurate information occurred not very frequently among all web-based pediatric prior approval requests, we believe that almost half of them could potentially influence pediatric ID fellows decision-making. While it is not practical for a pediatric ID fellow to check the accuracy of each request, targeted review of requests for specific antimicrobials, or for specific patient populations is warranted. 89

103 Table 4.1 Estimated sample size in each group in ascending order if significant reductions of antibiotic use are to be reached by using the results of this study: power 80%, alpha 0.05 and 2- sided test of significance From the results of some outcome measures Estimated sample size in each group Significant difference in our study ( Y/N )* Proportion of inappropriate 26 Yes antimicrobial course at Day 2 after postprescription review Median duration (days) of restricted 83 No antibiotic agents per episode of infection Median duration of total antibiotic agents 208 No per episode of infection Combined restricted antimicrobial use 230 No (days) per episode of infection Inappropriate restricted antimicrobial use 271 No ( DOTs) after dropping some cases with long hospitalizations Inappropriate restricted antimicrobial use 279 No ( DOTs) Restricted antimicrobial use (DOTs) after 2442 No dropping some cases with long hospitalizations Restricted antimicrobial use (DOTs) No Total antimicrobial use (DOTs) after No dropping some cases with long hospitalizations Total antimicrobial use (DOTs) Not estimated because No higher rank sums (more antibiotic use ) in intervention group Combined total antimicrobial use (days) per episode of infection Not estimated because higher rank sums (more antibiotic use ) in intervention group No * In our study: Intervention group: 30 patients, 32 antimicrobial courses. Control group: 30 patients, 33 antimicrobial courses 90

104 5. References: 5.1. Appendix: Figure 5.1 Sample data collection form for Aim 1 and Aim 2 91