An Electronic Chart Prompt to Decrease Proprietary Antibiotic Prescription to Self-pay Patients

Transcription

1 ACAD EMERG MED d March 2005, Vol. 12, No. 3 d An Electronic Chart Prompt to Decrease Proprietary Antibiotic Prescription to Self-pay Patients Steven L. Bernstein, MD, David Whitaker, DO, Jonathan Winograd, DO, John A. Brennan, MD Abstract Objectives: Emergency physicians unaware of patients insurance status may prescribe expensive proprietary antibiotics for patients who cannot afford them. The objective of this study was to develop a clinical decision support system to display patient insurance status before prescription writing for outpatient conditions. Methods: This was a 26-week before-and-after trial at an urban emergency department (ED) with 78,000 visits/year treating a medically underserved population. Sixty-one prescribers, including attending physicians, residents, and physician assistants, participated. All patients older than 18 years of age discharged from the ED receiving antibiotic prescriptions were eligible. The electronic ED chart is linked to prescription-writing software, which includes a menu of 74 antibiotics. The system was programmed so that when an emergency physician accessed the prescription menu, a prompt appeared displaying insurance status. Prescribers also received educational interventions. The main outcome measure was the percentage of prescribers who reduced their prescription writing of proprietary antibiotics to selfpay patients. Data were analyzed with cluster techniques using SPSS 10.0 (SPSS Inc., Chicago, IL). Results: Of 594 prescriptions written for self-pay patients before prompt insertion, 158 (26.6%) were for proprietary antibiotics. After the intervention, self-pay patients received 564 antibiotic prescriptions, of which 117 (20.7%) were for proprietary drugs. Analyzed by prescriber, the reduction in the prescription rate for proprietary antibiotics was statistically significant (p = 0.03, x 2 test). Patients with respiratory or urinary infections also had a statistically significant reduction in proprietary antibiotic prescription (p = 0.03). Conclusions: A clinical decision support system, integrated into a prescription-writing program, can decrease the prescription of proprietary antibiotics for self-pay patients in the ED. Key words: clinical decision support; antibiotics; information systems; emergency department; compliance. ACADEMIC EMERGENCY MEDICINE 2005; 12: Patients in the emergency department (ED) frequently receive prescriptions for antibiotics on discharge. In 2000, antimicrobial agents were prescribed or administered 25.4 million times in U.S. EDs, accounting for 14.6% of all drug mentions. 1 Patients often do not fill these prescriptions, 2 in part because of lack of insurance to pay for them. Although insurance status is a required data element of ED charts, physicians may not always be aware of the patient s insurance status From the Department of Emergency Medicine, Newark Beth Israel Medical Center, St. Barnabas Health Care System (SLB, DW, JW, JAB), Newark, NJ. Received May 13, 2004; revision received September 27, 2004; accepted September 29, Presented at American College of Emergency Physicians scientific assembly, Seattle, WA, October Supported by a grant from the Harvey Nussbaum Research Institute (Livingston, NJ). This work was performed while Dr. Bernstein was a Visiting Fellow at the Robert Wood Johnson Clinical Scholars Program (Yale University School of Medicine, New Haven, CT). Address for correspondence and reprints: Steven L. Bernstein, MD, Department of Emergency Medicine, Albert Einstein College of Medicine, Montefiore Medical Center, 111 East 210th Street, Bronx, NY Fax: ; sbernste@montefiore.org. doi: /j.aem at the time they write the prescription, resulting in patient inability to fill the prescription, suboptimal care, and potential adverse outcomes. The Institute of Medicine report on medical error 3 addressed medication errors in the inpatient setting, focusing on dosing mistakes, unrecognized allergies, drug drug interactions, or drug selection. The Institute of Medicine report has focused interest on the use of clinical decision support systems to assist physicians clinical decision making. 4,5 Clinical decision support systems have been intensively studied in infectious diseases, primarily to assist with antibiotics prescribing in the inpatient setting To date, few studies have examined clinical decision support systems in the ED setting. 13,14 Aronsky and Haug developed a clinical decision support system to identify ED patients with pneumonia eligible for treatment using a practice guideline. 13 The system had a sensitivity of 95% and a specificity of 68.5%. Schriger et al. designed a clinical decision support system, embedded within the electronic medical record, to assist with management of patients with occupational exposure to body fluids. 14 The system improved documentation of elements of patient history from 57% to 98% and elements of aftercare instruction from 31% to 93%.

2 226 Bernstein et al. d PROPRIETARY ANTIBIOTICS AND SELF-PAY PATIENTS Clinical decision support systems come in two general types: synchronous and asynchronous. 4 Synchronous decision support is given in real time to guide physician prescribing during care of the patient. Ideally, these systems should be fast, be simple to use and implement, raise red flags only when clinically pertinent, and be integrated into the hospital s extant order-entry system. Asynchronous decision support uses software to inform physicians after the fact if a problem has been detected. This study describes the development and use of a simple synchronous decision support system, using an on-screen prompt, to alert physicians to patients insurance status before prescribing antibiotics for patients to be discharged from the ED. The clinical decision support system was not designed to guide or force prescription of specific antibiotics for individual diagnoses. METHODS Study Design. This was a before-and-after trial covering 26 weeks: one half before the intervention and one half after. The study was approved by the hospital s institutional review board, which waived the requirement for informed consent. Study Setting and Population. The study was conducted at an urban ED with 78,000 visits/year serving a medically underserved population. The ED is the primary teaching site for an emergency medicine residency. All health care workers with authority to write prescriptions using the electronic prescription writer participated. This included all attending emergency physicians, physician assistants (PAs), emergency medicine residents, medical and PA students, and residents rotating from other services who work in the ED. (Prescriptions written by students require a cosignature by an attending physician or PA.) All patients aged 18 years or older discharged from the ED receiving antibiotic prescriptions were considered eligible. Demographic and clinical data recorded included age, gender, insurance status, and ICD-9 diagnostic codes. Study Protocol. The hospital s electronic ED patient chart system contains 400 templates, based on presenting complaint. This charting system is linked to an electronic prescription-writing system, which contains a menu of 200 medications in various dosing schedules. Included in the prescription menu are 74 antibiotics in 140 different dosing schedules. These antibiotics are listed in Table 1 (24 of the antibiotics are not specified). The menu is not restrictive; a free-text option is available to allow physicians to prescribe drugs not included on the menu or to alter the dosing schedule of drugs that are included. TABLE 1. Antibiotics Available in Electronic Menu Drug Proprietary Generic Oral antibacterials Azithromycin Amoxicillin Clarithromycin Ampicillin Cefuroxime Cephalexin Levofloxacin Penicillin Ciprofloxacin Clindamycin Amoxicillin/ Dicloxacillin clavulanic acid Cefuroxime Doxycycline Cefaclor Erythromycin Cefprozil Metronidazole Cefadroxil Minocycline Nitrofurantoin Trimethoprimsulfamethoxazole Tetracycline Oral antivirals Famciclovir Acyclovir Oseltamivir Valacyclovir Ocular antibacterials Ofloxacin Sulfacetamide Ciprofloxacin Gentamicin Erythromycin Tobramycin Antifungal agents Fluconazole Clotrimazole Terconazole Nystatin Itraconazole Ketoconazole Miconazole Griseofulvin Otic antibacterials Ofloxacin Neomycin/colistin Topical antibacterials Mupirocin Bacitracin Silver sulfadiazine Scabicides Lindane Permethrin The system was programmed so that when a provider filled out a patient chart and accessed the prescription system, a prompt appeared on the screen displaying the patient s insurance status, with bold red letters reminding the provider to check the patient s insurance status before antibiotic prescription (Figure 1). Residents and attending physicians received a brief ED-based in-service when the prescription prompt was introduced, a 30-minute didactic lecture, and a brief reminder one and four weeks after the prompt was introduced. In all of these sessions, physicians were encouraged to consider prescribing generic medications, particularly antibiotics, if clinically appropriate. If a patient received prescriptions for more than one antibiotic, we included each drug. Assessment of the availability of each drug in generic form was determined by consulting the hospital pharmacist and local retail pharmacies. We elected to study prescribing patterns for those classes of antibiotics that offered both generic and proprietary medications. Hence, we excluded otic suspensions, because neomycin/colistin is filled in generic form only and is prescribed almost exclusively in our institution for mild otitis externa. We excluded

3 ACAD EMERG MED d March 2005, Vol. 12, No. 3 d Figure 1. The electronic prescription menu, with insurance prompt. When in use, the prompt ( Please review. ) appears in red. scabicides, all of which are available in generic form. Finally, we excluded the proprietary antiretroviral drugs taken by patients with human immunodeficiency virus infection or by hospital employees with needlestick injuries. The classes of antibiotic in the study include antibacterial, antiviral, and antifungal agents. Patients were further categorized by diagnosis, based on site of infection and primary ICD-9 code: respiratory, urinary, genital/sexually transmitted disease, skin/soft tissue, gastrointestinal, ophthalmologic, viral, and other. One preplanned subgroup analysis was the change in the prescription rate for patients with urinary or respiratory infections, because these are the most common infections for which antibiotics are prescribed in our ED and the ones with the broadest choices of medication. Measures. In this study, the intervention was allocated to the individual writing the prescription, either an attending physician, resident, PA, or medical student, and not the prescriptions or the patients themselves. Prescriptions written by residents must be cosigned by an attending physician. (Our ED has two PAs, who practice independently in fast track.) We chose to analyze the data by the actual prescriber (attending, resident, PA, or student), not by the attending physician or PA cosigning the prescription. We used a clustered model to analyze data that considered the main outcome measure to be the change in the percentage of prescriptions for proprietary antibiotics written for self-pay patients, grouped by prescriber. Data Analysis. Data were transferred from the health care system s mainframe computer into Microsoft Excel 2000 and then to SPSS 10.0 (SPSS Inc., Chicago, IL). Categorical variables were recoded as numerical variables by double data entry by two research assistants. The two data sets were then compared and discrepancies resolved by a third investigator. Categorical data were analyzed by chi-square test, with the chi-square statistic adjusted for a variance inflation factor caused by clustering around prescriber. 15 a was set at 0.05 for significance. A correction for multiple comparisons was not made. Sample Size Calculation. A pilot study of 12 days data (March 21 to April 1, 2001) showed that 419 antibiotic prescriptions were written, of which 85 (20.3%) were for patients with no insurance. Of these 85 prescriptions, 38 (44.7%, 95% confidence interval [CI] = 34.1% to 55.3%) were for proprietary antibiotics. We chose to consider as quantitatively significant a 25% reduction in the prescription of proprietary antibiotics to self-pay patients. To demonstrate a 25% reduction in the use of proprietary prescriptions, with a power of 0.8 and an a of 0.05, a sample size calculation showed that 324 patients would be needed in each time period of study. To account for clustering around the prescriber, the sample size was increased by a variance inflation factor (VIF) calculated from

4 228 Bernstein et al. d PROPRIETARY ANTIBIOTICS AND SELF-PAY PATIENTS pilot data. A VIF of 1.22 was calculated, based on a corrected mean cluster size of and intraclass correlation coefficient of 0.022, using standard methods and the equation VIF = 1 1 (m# 2 1)r, where m# is corrected mean cluster size and r is intraclass correlation coefficient. 15,16 To account for clustering effects, the calculated sample size was increased proportionately. The sample size was enlarged further, to approximately 520 prescriptions per group, to provide for an additional period of follow-up to test the durability of the intervention. RESULTS There were 61 prescribers in the study, consisting of 38 emergency medicine residents, 16 attending emergency physicians, two PAs, residents from internal medicine and pediatrics, and medical and PA students. The providers authorized to sign the prescriptions (16 physicians and two PAs) consisted of 13 men and five women and were in practice for a mean of 7.2 years (range, 1 22 years). (The electronic medical record did not record the names of individual non emergency medicine residents or students. Hence, they are grouped together and analyzed as clusters of either students or residents.) Patient demographic and clinical characteristics are given in Table 2. Overall, 1,158 antibiotic prescriptions were written for 1,057 patients, a mean of 1.1 prescriptions/patient. The effect of the prompt on antibiotic prescribing is shown in Table 3. The 61 prescribers yielded 107 clusters, 55 before the prompt and 52 after, for a mean of 10.8 prescriptions/cluster. Not every prescriber prescribed antibiotics during both phases of the study. The percentage of proprietary antibiotics prescribed before and after insertion of the electronic prompt was 26.6% and 20.7%, respectively, for a statistically significant reduction of 22% (p = 0.03, VIF-adjusted x 2 ; 95% CI = 2% to 42%). The most commonly prescribed antibiotics before and after the prompt was inserted are listed in Table 4. TABLE 2. Characteristics of Discharged Adult Patients Who Received an Antibiotic Prescription Control Intervention No. of patients Women (%) 330 (60.8%) 334 (65.0%) Age, yr (median, interquartile range) 31 (23 40) 28 (22 40) Diagnostic category Respiratory Urinary Genital/sexually transmitted disease Skin/soft tissue Gastrointestinal 6 3 Ophthalmologic Viral Other 9 9 TABLE 3. Antibiotic Prescribing for Self-pay or Charity-care Patients, before and after Insertion of the Prescription Prompt Before Prompt After Prompt Total Generic (%) 436 (73.4) 447 (79.3) 883 Proprietary (%) 158 (26.6)* 117 (20.7)* 275 Total *p = 0.03, chi-square test. In a preplanned subgroup analysis, the decision prompt was found to reduce prescription of proprietary antibiotics to self-pay patients with respiratory and urinary infections. A total of 523 prescriptions were written for 490 patients with either a urinary or a respiratory infection (1.07 prescriptions/patient). For patients with these infections, the percentage of proprietary antibiotics prescribed before and after insertion of the electronic prompt was 44.4% and 31.3%, respectively, for a statistically significant reduction of 30% (p = 0.005, VIF-adjusted x 2 ). These data are summarized in Table 5. Table 6 summarizes the change in prescribing patterns for patients with respiratory or urinary tract infections. Data were pooled for these infections because 1) these represented two of the three most common infection categories seen; 2) the second most common infection category, skin/soft tissue, was overwhelmingly treated with cephalexin; and 3) pneumonia and urinary tract infection have important, evidence-based clinical practice guidelines DISCUSSION This study was able to demonstrate a statistically significant reduction in the prescription of proprietary antibiotics for self-pay patients discharged from an urban ED. The main outcome measure narrowly TABLE 4. Antibiotics Prescribed to All Self-pay Adult Patients Antibiotic Before Prompt After Prompt Amoxicillin/clavulanate* Azithromycin* Ciprofloxacin* 4 8 Clarithromycin* 3 1 Levofloxacin* Amoxicillin Cephalexin Clindamycin Doxycycline Erythromycin 4 8 Metronidazole Nitrofurantoin Penicillin Trimethoprim-sulfamethoxazole Total *Proprietary drugs.

5 ACAD EMERG MED d March 2005, Vol. 12, No. 3 d TABLE 5. Antibiotic Prescribing for Self-pay or Charity-care Patients with Respiratory or Urinary Tract Infections, before and after Insertion of the Prescription Prompt Before Prompt After Prompt Total Generic (%) 145 (55.6) 180 (68.7) 325 Proprietary (%) 116 (44.4)* 82 (31.3)* 198 Total *p = 0.005, chi-square test. missed the predetermined level of quantitative significance (25% vs. a 22% actual reduction). For the secondary outcome measure, prescription of proprietary antibiotics to self-pay patients with respiratory or urinary tract infections, quantitative significance was reached (30%). It is possible that a greater reduction in prescription of proprietary antibiotics was reached in patients with respiratory or urinary tract infections because numerous appropriate antibiotic choices exist to treat these conditions. At the same time, the reduction in proprietary prescribing was modest in general. The reasons for the apparent resistance to change are several and probably pertain to the lack of forcing functions, ingrained physician prescribing patterns, and the duration of follow-up. Our decision prompt did not contain a forcing function ; in other words, physicians were free to prescribe any antibiotic for any patient. Choices were not constrained based on the patient s insurance or diagnosis. Forcing functions have been shown to be an effective way to reduce medication errors, such as overdosing or prescribing drugs to which the patient is known to be allergic. 20,21 We elected not to constrain prescribing options based on insurance because of ethical concerns about economic discrimination. The reduction in proprietary drug prescription was modest. Even with the prompt, one prescription in five to a self-pay patient was for a brand-name antibiotic. TABLE 6. Numbers of Prescriptions Written to Selfpay Adult Patients with Respiratory or Urinary Tract Infections Antibiotic Before Prompt After Prompt Amoxicillin/clavulanate* 14 9 Azithromycin* Clarithromycin* 3 1 Ciprofloxacin* 1 7 Levofloxacin* Amoxicillin Doxycycline Nitrofurantoin 8 10 Penicillin Trimethoprim-sulfamethoxazole Other (cephalosporins, clindamycin, erythromycin, metronidazole) Total *Proprietary drugs. In these patients, the reasons for the persistent use of proprietary drugs remain unclear. Typically, generic alternatives were available for the drugs prescribed. For example, doxycycline or erythromycin may be used to treat community-acquired pneumonia in a previously untreated immunocompetent adult, instead of the commonly prescribed levofloxacin or azithromycin. 17,18,22,23 Similarly, trimethoprim-sulfamethoxazole is an acceptable first-line treatment for uncomplicated cystitis or pyelonephritis. 19,23 Summaries of recent clinical practice guidelines addressing antibiotic treatment for immunocompetent adults with communityacquired pneumonia or urinary tract infection are presented in Table 7. The difference in pricing between generic and proprietary antibiotics can be striking. Table 8 summarizes cost data from three local pharmacies for standard treatment regimens for the most commonly prescribed antibacterial agents in this study. Using the data in Table 6, the intervention would have resulted in a decrease in cost to the patient from $35.37/ prescription to $ Our hospital does not have a pharmacy for discharged ED patients to fill their prescriptions. It seems reasonable to assume that prescriptions for cheaper antibiotics are more likely to be filled than prescriptions for the proprietary drugs shown. One notable exception is clindamycin, which, although generic, still is quite expensive. Another possible explanation remains for the persistence of proprietary drug prescribing. Pharmaceutical representatives meet from time to time with the residents and attending physicians of our department, chiefly through sponsorship of lunchtime conferences and, although discouraged, sporadic visits to the ED. These contacts may influence physician behavior sufficiently to account for the persistent and pervasive use of the more expensive proprietary drug. 24 These are in addition to the advertisements in medical journals and mass media and appearances at national conferences, to which physicians are exposed. The Institute of Medicine report on health care quality said that high-quality care should be safe, effective, patient centered, timely, efficient, and equitable. 6 Increasing the likelihood that a self-pay patient could fill a prescription for antibiotics (or any other needed medicine) would enhance the effectiveness and patient-centeredness of care. Several investigators have urged physicians to explicitly ask patients about their ability to pay for their care 25,26 ; a clinical decision support system like the one in this study provides an additional technique to inform physicians of their patients ability to pay. Future work will examine whether the prescription prompt improves patient adherence with filling the prescription and taking the medication and whether expanding the decision support system to include practice guidelines about the treatment of specific infections influences physician behavior.

6 230 Bernstein et al. d PROPRIETARY ANTIBIOTICS AND SELF-PAY PATIENTS TABLE 7. Clinical Practice Guideline Recommendations for Treatment of Community-acquired Pneumonia or Urinary Tract Infection in Immunocompetent, Healthy Adults Source Antibiotic Recommendations Community-acquired pneumonia Sanford Guide 23 Azithromycin, clarithromycin, erythromycin, doxycycline, fluoroquinolone, amoxicillin/clavulanic acid, cephalosporin American College of Emergency Physicians 22 Azithromycin, clarithromycin, erythromycin, doxycycline, or fluoroquinolone American Thoracic Society 17 Azithromycin, clarithromycin, or doxycycline Infectious Diseases Society of America 18 Azithromycin, clarithromycin, erythromycin, or doxycycline Urinary tract infection Sanford Guide 23 Trimethoprim-sulfamethoxazole or fluoroquinolone; alternatives: nitrofurantoin, doxycycline, amoxicillin/clavulanic acid Infectious Diseases Society of America 19 Trimethoprim-sulfamethoxazole or fluoroquinolone; alternative: nitrofurantoin LIMITATIONS The study has four main limitations. First, we did not demonstrate improvement in the percentage of ED patients who actually filled and took their antibiotic as prescribed. This was not the main outcome measure of the study, and the trial was not powered to detect this. We chose to examine whether physician behavior can be altered by an electronic prompt. The use of a process outcome, rather than a clinical one, may limit somewhat the clinical utility of the study. 27 However, because of the evidence linking financial barriers as a reason for failure to fill prescriptions, we believe it likely that giving self-pay patients a cheaper drug will increase the likelihood of their filling the prescription. Second, we did not deliver the intervention as a randomized trial. We chose the before-and-after format because we believed that once emergency physicians became aware of the prompt, their behavior might have changed whether or not the prompt appeared on the screen. Hence, the study was structured as a before-and-after trial of an administrative intervention, with the before group serving as a historical control. A methodologically sounder technique to test the efficacy of the intervention would have TABLE 8. Average Retail Cost of Common Antibiotic Treatment Regimens from Three Local Pharmacies Drug Regimen Retail Cost Levofloxacin* 500 mg daily for 10 days $96.00 Amoxicillin/ clavulanate* 875 mg BID for 7 days $85.00 Clindamycin 300 mg QID for 10 days $76.30 Ciprofloxacin* 500 mg BID for 3 days $35.33 Nitrofurantoin 100 mg BID for 7 days $34.70 Azithromycin* 250 mg daily for 4 days $33.66 Cephalexin 500 mg QID for 7 days $20.00 Amoxicillin 500 mg TID for 10 days $14.66 Doxycycline 100 mg BID for 10 days $13.00 Penicillin VK 500 mg QID for 5 days $9.33 Trimethoprimsulfamethoxazole 1 tablet BID for 3 days $6.33 *Proprietary drugs. been to randomize the prescription prompt across multiple EDs with separate provider staffs. Third, we only followed prescription-writing behavior for 13 weeks after initiation of the prompt. We do not know whether emergency physicians will resume prescribing proprietary antibiotics in selfpay or charity-care patients. The intervention may be strengthened by providing a stronger education program for providers or designing a clinical decision support system with a forcing function. A planned one-year follow-up study will address this question. Fourth, we could not isolate the effect of the prompt itself versus the educational interventions (in-service, lecture, and reminders). These latter maneuvers accompanied the intervention group only and may have strengthened the treatment effect. Longer follow-up, without additional educational interventions and with natural turnover in residency classes, would allow us to isolate the effect of the prompt only. We did not examine whether allergy history may have influenced prescribing behavior. This is unlikely to have affected prescribing behavior in a substantial fashion, because there are many generic alternatives to b-lactams, the most common antibiotic allergy. Potential drug drug interactions may have influenced choice of therapy. Further, the insurance status as listed in the chart may not always be accurate. Some patients listed as self-pay may in fact have insurance because of a change in employment status or inaccurate transcription by a clerk. However, this is likely to have had a small effect 28 and unlikely to have affected the results, because we were studying prescriber behavior, not whether the prescription was filled. Last, prescribing patterns in the pediatric ED were not studied, because there are very few self-pay children in our population. Most are enrolled in Medicaid or a managed care plan. CONCLUSIONS A decision support system that did not constrain physician choice was able to increase the percentage of generic antibiotics written for self-pay or charity-care

7 ACAD EMERG MED d March 2005, Vol. 12, No. 3 d patients, including those with respiratory or urinary infections. Lasting and statistically significant increases in generic prescribing may require constraints in the form of forcing functions or additional physician education. This study did not examine the most clinically pertinent end point: whether the decision support system resulted in more patients filling, and taking, their medication as directed. The authors thank Victor Lindberg for programming the prompt, Alvan R. Feinstein, MD (deceased), for support and guidance, and Robert Wears, MD, MS, for assistance with data analysis. References 1. McCaig LF, Ly N. National Hospital Ambulatory Medical Care Survey: 2000 emergency department summary. Advance data from vital and health statistics; no Hyattsville, MD: National Center for Health Statistics, Saunders CE. Patient compliance in filling prescriptions after discharge from the emergency department. Am J Emerg Med. 1987; 5: Institute of Medicine. To Err is Human: Building a Safer Health System. Washington, DC: National Academy Press, Bailey TC, McMullin ST. Using information systems technology to improve antibiotic prescribing. Crit Care Med. 2001; 29(suppl):N Bissell MG. The effect of benchmarking clinical practice with the clinical laboratory. Clin Lab Med. 1999; 19: Institute of Medicine. Crossing the Quality Chasm: A New Health System for the 21st Century. Washington, DC: National Academy Press, Bates DW, Cohen M, Leape LL, Overhage JM, Shabot MM, Sheridan T. Reducing the frequency of errors in medicine using information technology. J Am Med Inform Assoc. 2001; 8: Christakis DA, Zimmerman FJ, Wright JA, Garrison MM, Rivara FP, Davis RL. A randomized controlled trial of point-ofcare evidence to improve the antibiotic prescribing practices for otitis media in children [abstract]. Pediatrics. 2001; 107:e Kristensen B, Andreassen S, Leibovici L, Riekehr C, Kjaer AG, Schonheyder HC. Empirical treatment of bacteraemic urinary tract infection. Dan Med Bull. 1999; 46: Leibovici L, Gitelman V, Yehezkelli Y, et al. Improving empirical antibiotic treatment: prospective, nonintervention testing of a decision support system. J Intern Med. 1997; 242: Warner H, Blue SR, Sorenson D, et al. New computer-based tools for empiric antibiotic decision support. Proc AMIA Symp. 1997; Soumerai SB, Avorn J, Taylor WC, Wessels M, Maher D, Hawley SL. Improving choice of prescribed antibiotics through concurrent reminders in an educational order form. Med Care. 1993; 31: Aronsky D, Haug PJ. Automatic identification of patients eligible for a pneumonia guideline. Proc AMIA Symp. 2000; Schriger DL, Baraff LJ, Rogers WH, Cretin S. Implementation of clinical guidelines using a computer charting system: effect on the initial care of health care workers exposed to body fluids. JAMA. 1997; 278: Wears RL. Advanced statistics: statistical methods for analyzing cluster and cluster-randomized data. Acad Emerg Med. 2002; 9: Fleiss JL. Statistical Methods for Rates and Proportions. New York, NY: Wiley, American Thoracic Society. Guidelines for the management of adults with community-acquired pneumonia. Am J Respir Crit Care Med. 2001; 163: Mandell LA, Bartlett JG, Dowell SF, File TM Jr, Musher DM, Whitney C. Update of practice guidelines for the management of community-acquired pneumonia in immunocompetent adults. Clin Infect Dis. 2003; 37: Warren JW, Abrutyn E, Hebel JR, Johnson JR, Schaeffer AJ, Stamm WE. Guidelines for antimicrobial treatment of uncomplicated acute bacterial cystitis and acute pyelonephritis in women. Clin Infect Dis. 1999; 29: Bates DW, Gawande AA. Improving safety with information technology. N Engl J Med. 2003; 348: Sittig DF, Stead WW. Computer-based physician order entry: the state of the art. J Am Med Inform Assoc. 1994; 1: American College of Emergency Physicians Clinical Policies Committee. Clinical policy for the management and risk stratification of community-acquired pneumonia in adults in the emergency department. Ann Emerg Med. 2001; 38: Gilbert DN, Moellering RC Jr, Sande MA. The Sanford Guide to Antimicrobial Therapy Hyde Park, VT: Antimicrobial Therapy, Inc., Gill KS, Katz ED, Mahoney H. Effect of pharmaceutical representatives on prescribing practices of an emergency medicine residency [abstract]. Acad Emerg Med. 2003; 10: Weiner S. I can t afford that! Dilemmas in the care of the uninsured and underinsured. J Gen Intern Med. 2001; 16: Braham RL. Teaching about cost-effective use of medical resources: still trying after all these years. J Gen Intern Med. 2001; 16: Feinstein AR. Clinical Epidemiology: The Architecture of Clinical Research. Philadelphia, PA: Saunders, Price J, Estrada CA, Thompson D. Administrative data versus corrected administrative data. Am J Med Qual. 2003; 18: Where to Find AEM Instructions for Authors For complete instructions for authors, see the January or July issue of Academic Emergency Medicine; visit the SAEM web site at inform/autinstr.htm; or contact SAEM via at aem@saem.org, via phone at , or via fax at