EVALUATION OF THE SENSITIVITY AND SPECIFICITY OF THE EHRLICHIA CANIS DIAGNOSTIC TEST: Anigen Rapid E.canis Ab Test Kit FINAL REPORT Research contract (art. 83 of the L.O.U) between the Ehrlichiosis Diagnostic Service at the Universidad Complutense de Madrid and Urano Vet, S.L.
Madrid, October 2013 Research team The research team belongs to the Department of Animal Medicine and Surgery at the School of Veterinary Science, Universidad Complutense de Madrid (UCM), and comprises: Ángel Sainz Rodríguez (coordinator) Mercedes García Sancho Téllez Fernando Rodríguez Franco Alejandra Villaescusa Fernández Definition of the research The aim of this trial is to compare the results obtained using the immunochromatography test to be evaluated with those obtained using the reference serological technique, namely indirect immunofluorescence, using Ehrlichia canis (Madrid strain) previously isolated by the UCM research group as antigen. The indirect immunofluorescence assay has been checked beforehand in the UCM research group's laboratory. This phase includes the immunochromatographic analysis of 50 samples previously titrated using the IFI technique. These samples are obtained from 15 animals that are either seronegative or have antibody titres below the cut-off point and 35 seropositive animals with different antibody titres (between 1:80 and 1:5120). Once the samples have been analysed, the results obtained using the immunochromatography test will be analysed on the basis of the concordance correlation coefficient and kappa index, together with the sensitivity, specificity and predictive values.
Introduction Ehrlichioses are a group of vector-borne diseases caused by obligate intracellular Gramnegative bacteria that can affect both wild and domestic animals and humans. A significant amount of progress has been made in understanding Ehrlichia spp. infection in the past few years. Thus, since Donatien and Lestoquard first identified Ehrlichia canis in tickinfested dogs presenting a febrile syndrome in 1935 (Donatien y Lestoquard, 1935), a large number of species from the genus Ehrlichia (or the closely related Anaplasma) that are able to infect humans and different animal species have been described. These include the species formerly known as E. platys (currently Anaplasma platys), E. risticii (Neorickettsia risticii) and E. equi, E. phagocytophila and the causative agent of human granulocytic ehrlichiosis (currently Anaplasma phagocytophilum), and E. ewingii and E. chaffeensis. Indeed, three new species, namely E. ewingii (Anderson et al., 1992), E. chaffeensis and Anaplasma phagocytophilum (Johansson et al., 1995), were described in dogs in the 1990s alone. Classification of the genus Ehrlichia and other genetically closely related genera was initially based on highly variable morphological, ecological, epidemiological and clinical characteristics. However, novel genetic analyses of the 16S rrna genes and surface proteins highlighted the existence of major differences between several species previously included in the same genus. This led to a proposed restructuring and, subsequently, a new classification of a large number of species from the genera Ehrlichia, Anaplasma, Cowdria, Neorickettsia and Wolbachia (Dumler et al., 2001). These genera are characterised by being a group of obligate intracellular bacteria that live inside vacuoles in eukaryotic cells. Specifically, some Ehrlichia species have been reclassified into other genera, whereas species from other genera are currently considered to belong to the genus Ehrlichia. The current classification is as follows: - The genus Ehrlichia includes E. canis, E. chaffeensis, E. ewingii, E. muris and the species previously known as Cowdria ruminantium (currently Ehrlichia ruminantium). - The genus Anaplasma includes Anaplasma marginale and the species previously known as E. platys (currently A. platys), E. equi, E. phagocytophila and the causative agent of human ehrlichiosis. Moreover, the differences between the latter three species are so minor that they are currently considered to be the same species, namely Anaplasma phagocytophila. - The genus Neorickettsia comprises N. helminthoeca as well as Ehrlichia risticii (currently N. risticii) and Ehrlichia sennetsu (currently N. sennetsu). The first literature reference to ehrlichiosis in Spain was a case reported in dogs in Catalonia (Font et al., 1988). Since then, however, the onset of immunological diagnostic techniques (mainly indirect immunofluorescence) has allowed cases compatible with canine ehrlichiosis to be diagnosed essentially throughout the country, coinciding with those regions in which the tick Rhipicephalus sanguineus is present. The epidemiological studies undertaken to date in the Community of Madrid suggest a prevalence of 6.5% (Sainz et al., 1998). The seroprevalence rates reported for other Spanish regions, such as Galicia, Castilla y León and Catalonia, vary between 3.1% and 19.2% (Sainz et al., 1998; Amusategui et al., 2000). These values, together with the experience accumulated in our diagnostics laboratory, suggest that E. canis infection is widespread throughout Spain. Despite this, the first isolate from this species of Ehrlichia in Spanish dogs was obtained by our group in 2004 (Aguirre et al., 2004).
In summary, our current understanding of canine ehrlichiosis in Spain indicates the existence of a large number of clinical cases seropositive for Ehrlichia canis with symptomatology and blood analyses consistent with ehrlichiosis. This, together with the isolation of this species from dogs with clinical symptoms consistent with infection means that E. canis is considered to be the most important Ehrlichia species in our country. Serological techniques are most commonly used to diagnose canine ehrlichiosis by vets in our geographical setting. Although many such serology-based diagnostic techniques are available, indirect immunofluorescence (IFI) is considered to be the reference technique (Neer et al., 2002). Despite this, other techniques, such as ELISA or immunochromatography, are equally valid alternatives that are known to provide similar results to those provided by the IFI technique (Waner et al., 2001; Neer et al., 2002). Materials and Methods Samples A total of 50 serum samples previously titrated by indirect immunofluorescence using a native strain of Ehrlichia canis (Madrid strain) previously isolated by the UCM research team were analysed. Of these, 15 were seronegative for Ehrlichia canis and 35 were seropositive but with different antibody titres (between 1:80 and 1:5120). All samples had been stored frozen, at a temperature of 20 ºC, in the serum bank maintained by the Ehrlichiosis Diagnosis Service of the School of Veterinary Science at the UCM. Indirect immunofluorescence (IFI) technique The indirect immunofluorescence technique for the detection of anti-ehrlichia canis antibodies (IgG) used in this study has previously been checked in the UCM research group's laboratory and referenced in several publications in this respect. The cut-off point is established at the 1:80 dilution. The antigen used comprises the canine DH82 cell line infected by Ehrlichia canis obtained from a naturally infected dog in Madrid. This isolate was genetically characterised and included in GenBank with the name Ehrlichia canis Madrid strain and accession number AY394465. The flask from which the antigen was obtained presented a cell infection percentage of more than 40%. Fixation of the antigen to the slides was performed after removal of the supernatant and washing three times with PBS (ph 7.2) by centrifugation. The antigen was then deposited on special IFI slides and fixed by air-drying. All slides were stored frozen at 20 ºC until use. The IFI protocol in our laboratory comprises the following steps: 1. All serum to be analysed are serially diluted (with an initial dilution of 1:10). The normal working dilutions are 1:40, 1:80 and 1:320. In the event of positive serum with a final titre higher than 1:320, the technique is repeated using higher dilutions (up to a maximum titre of 1:10,240). A positive control serum (using its final titre) and a negative control serum from samples already checked by our laboratory are also included.
2. The slides are thawed prior to use. 3. Test sample dilutions are deposited in the wells on IFI slides. One dilution of a positive control and one of a negative control are included on each slide systematically. 4. Each slide is then incubated at 37 ºC for 30 minutes. 5. The samples are removed from the wells and the slides washed three times with PBS (ph 7.2) while shaking. 6. The slides are allowed to dry at room temperature. 7. Dog anti-igg conjugated with fluoresceine isothiocyanate at a dilution of 1:100 in PBS (ph 7.2) is added to each well, with Evans blue as dye. 8. The slides are then incubated again at 37 ºC for a further 30 minutes. 9. The conjugate residues are removed from the wells and the slides washed three times with PBS (ph 7.2) while shaking. 10. The slides are allowed to dry at room temperature. 11. The slides are mounted for reading, using commercial buffered glycerine. 12. The IFI slides are always read under a fluorescence microscope by the same person. Immunochromatography test All previously thawed samples were analysed blind using the immunochromatography test strictly according to the manufacturer's instructions. All products analysed belonged to batch 2105073 of Uranotest Ehrlichia or Anigen rapid E. canis Ab, with expiry date of June 2015. As established in the instructions, samples were considered to be: - Negative, when a single band was observed in the control region ( C ) of the results window, and - Positive, when two bands were seen, one in the control region ( C ) of the results window and the other in the test region ( T ), irrespective of which of the two appeared first.
Repetition of anomalous analyses Both the indirect immunofluorescence assay and the immunochromatography test were repeated for any sample for which anomalous results were obtained. Results analysis The results obtained using the immunochromatography test and those obtained using IFI were compared using the concordance correlation coefficient and Cohen's kappa (κ) coefficient. By convention, and according to the proposal of Landis and Koch, the degree of agreement, or concordance, between two techniques varies on the basis of said coefficient as follows: - Concordance is slight when κ = 0 0.2 - Concordance is fair when κ = 0.2 0.4 - Concordance is moderate when κ = 0.4 0.6 - Concordance is substantial when κ = 0.6 0.8 - Concordance is almost perfect when κ = 0.8 1.0 Moreover, analysis of the results obtained was completed by determining the sensitivity, specificity and predictive values. Results and Discussion Results concerning the methodology employed (chromatographic immunoassay) The Uranotest Ehrlichia or Anigen rapid E. canis Ab kit from BioNote, Inc. is easy to handle and comes with very clear and simple instructions, which we consider to be very promising due to its potential use. None of the material provided with the kit is superfluous and the presentation is appropriate (photo 1). Photo 1: Presentation of the kit
Despite this good presentation, we would like to point out that the different kits contained different quantities of diluent ( Assay diluents ), a fact that could readily be seen when evaluating various units simultaneously (photo 2) despite the fact that the user manual states that the volume of this diluent is 3 ml. A greater homogeneity of the volume of diluent added would be advisable. Photo 2: Different volumes of diluent in kits As regards occasional findings with the product's presentation, we noted that in one of the kits used the bag of silica gel included in the packaging had become stuck to the kit's plastic support (photo 3). However, as expected, this did not interfere with performance of the test or reading of the results. Photo 3: Bag of silica gel stuck to the plastic support in one of the kits. On another occasion roughness was observed around the sample-addition zone (photo 4), although this did not interfere with either sample and diluent migration or the results obtained (the sample was analysed using a second kit with no variation in the results obtained).
Photo 4: Roughness on the sample- and diluent-addition membrane As regards the test process, we note that the membrane in the results window began to turn purple immediately after addition of the sample and diluent (photo 5), as should occur according to the manufacturer's instructions. Once the coloured region had covered the membrane, the purple colour began to disseminate until only the corresponding bands (either the C band alone or both bands C and T ) were present when reading the results. As this migration was noted for all the kits used, no additional diluent needed to be added (as specified in the manufacturer's instructions if migration does not occur after 1 minute). Photo 5: Migration of the purple colour to the results window membrane after addition of the test sample and diluent In those cases in which the sample used had been haemolysed (these samples were used precisely to determine whether haemolysis interfered with the results), the membrane in the results window remained brownish (appearing dirty ) after migration of the sample and diluent (photo 6), and, in contrast to the situation with non-haemolysed samples, the original white colour was not recovered. Despite this, the results could be read without problem as the C and T bands were readily visible. Photo 6: Kit used with haemolysed sample. Staining of the results window membrane
In one case a small purple region was seen to remain in the upper left corner of the results window membrane after the rest had become colourless (photo 7). This did not interfere with reading of the results for this kit. Photo 7: A small purple region remains on the results window membrane The coloured band in the control region (C) appears rapidly and is usually homogeneous in terms of line shape (photo 8). As the C band was observed for all the kits used, all results obtained were considered to be valid (negative or positive). 1 2 3 4 5 6 Photo 8: Gradual appearance of the control band ( C ) during a test However, the C band varied in intensity between the different kits used (photo 9), although it was always readily and clearly identified, with no doubts in any case.
Photo 9: Two kits with different intensities of the C band As regards the test region (T), the band begins to appear very quickly for positive samples (photo 10). 1 2 3 4 5 Photo 10: Appearance of the T band for an IFI-positive sample On occasions, this band appeared even before complete migration of the purple colour to the results window membrane, and even before appearance of the C band (photo 11). Generally speaking, this fast appearance of the T band was associated with very high IFI-derived antibody titres. Photo 11: Appearance of an intense T band prior to appearance of the C band (sample with a titre of 1/5120 according to IFI) As regards results reading, and as will be discussed in further detail in the following section of this report, all IFI-negative samples (titres <1/80) gave clearly negative results with the kit, with a single band being clearly visible in the control region.
As far as positive samples are concerned, the intensity of the test band ( T ) was found to be highly variable (photo 12). In most cases the intensity appears to depend on the IFI-based antibody titre of the test sample, with more intense T bands being observed for samples with titres above 1/640. Photo 12: Comparison of the intensity of the T band for two samples with different IFI-based titres (1/80 for the kit in the upper part of the photo and 1/640 for the kit in the lower part) Despite the apparent relationship between the intensity of the T band and the antibody titre, higher intensities were often observed for samples with lower titres and lower intensities for samples with higher titres. This finding is not considered to be of particular importance as regards our evaluation of the kit as this is a qualitative test that only establishes whether a sample is positive or negative (the fact that the band is more or less intense is therefore completely unrelated to a higher or lower concentration of E. canis antibodies). However, the fact that the T bands for seven samples (two with a titre of 1/80, one with 1/160, two with 1/320 and two with 1/640) were so weak that the observer had great difficulty in deciding whether the sample was positive or negative could affect our evaluation of the kit. Finally, and in accordance with the manufacturer's recommendations, these samples were considered to be positive, although it should be stressed that, on occasions, these findings could lead to an error. In light of these findings, it would be advisable to stress in the instructions for use that the appearance of any T band, irrespective of its intensity, indicates a positive result. Photo 13: Kit used to test an IFI-positive sample (titre of 1/80) with a very weak T band
Photo 14: Two kits with very different T band intensities used to test two samples with similar IFIbased titres (1/80). Specifically, the band for the kit in the upper part of the photo is barely visible Moreover, as will be described and discussed in the following section, some of the IFI-positive samples gave negative results in the immunochromatographic test. Specifically, the samples that tested negative using the kit corresponded to two samples with an IFI-based titre of 1/80 and one with a titre of 1/160. If we consider these samples together with those with a weak T band mentioned above, the fact that only two of the eight samples with low IFI-based titres (1/80 and 1/160) analysed gave clearly positive results in the immunochromatography tests stands out. This finding is related to the manufacturer's observations in the Test limitations section, which states that although this is a very accurate method for detecting Ehrlichia canis antibodies, false negative results may be obtained in some cases. Moreover, as also described above, four samples with titres higher than 1/160 (two with a titre of 1/320 and two with a titre of 1/640) also gave results that could be considered to be uncertain. Results concerning degree of concordance obtained The results obtained in this study can be found in the following table, in which those cases for which the band was weak but visible and those in which the band was so weak that it was barely visible, are highlighted. Reference no. IFI Titre E. canis IFI Result E. canis Uranotest Ehrlichia or Anigen rapid E. canis Ab result Qualitative concordance 1 - - - YES 2 - - - YES 3 - - - YES 4 - - - YES 5 - - - YES 6 - - - YES 7 - - - YES 8 - - - YES 9 - - - YES 10 - - - YES 11 - - - YES 12 - - - YES 13 - - - YES
14 - - - YES 15 - - - YES 16 1/80 + + (very weak band, barely visible) YES 17 1/80 + - NO 18 1/80 + + YES 19 1/80 + + (very weak band, barely visible) YES 20 1/80 + - NO 21 1/160 + - NO 22 1/160 + + (weak) YES 23 1/160 + + YES 24 1/320 + + YES 25 1/320 + + YES 26 1/320 + + YES 27 1/320 + + (weak) YES 28 1/320 + + YES 29 1/320 + + (weak) YES 30 1/320 + + YES 31 1/320 + + YES 32 1/640 + + (weak) YES 33 1/640 + + (weak) YES 34 1/640 + + YES 35 1/640 + + YES 36 1/640 + + YES 37 1/1,280 + + YES 38 1/1,280 + + YES 39 1/1,280 + + YES 40 1/1,280 + + YES 41 1/1,280 + + YES 42 1/1,280 + + YES 43 1/1,280 + + YES 44 1/2,560 + + YES 45 1/2,560 + + YES 46 1/2,560 + + YES 47 1/2,560 + + YES 48 1/5,120 + + YES 49 1/5,120 + + YES 50 1/5,120 + + YES In all anomalous cases, repetition of both the IFI assay and immunochromatographic test confirmed the results obtained in the initial analysis. Although the number of samples retested is rather low, this may nevertheless suggest a good inter-test reproducibility. A summary of the results obtained can be found below: Positive Test Negative Test Total IFI Positive T band barely visible: 2 32 Weak T band: 5 Intense T band: 25 3 35 IFI negative 0 15 15 Total 32 18 50
The concordance correlation coefficient was 0.94 and the Kappa coefficient (κ) 0.865 (95% CI: 0.717 1), thus meaning that the concordance between the results obtained using both techniques is almost perfect. Taking the results obtained using the IFI technique as reference, the sensitivity of the test would be 0.912 (0.82 1) and the specificity 1 (1 1). Likewise, the positive predictive value would be 1 (1 1) and the negative predictive value 0.83 (0.66 1). However, these values should be interpreted with great care as the size of the sample analysed does not allow further conclusions to be reached and the characteristics of the samples do not represent the reality of such infections in most regions in Spain. Moreover, the fact that all tests used came from the same batch could affect the evaluation of this technique. On taking a closer look at the results, it can be seen that the concordance obtained when analysing negative samples for E. canis is very high, with all samples that tested negative by IFI also testing negative by immunochromatography (titres <1/80), although the number of samples analysed does not allow further conclusions in this respect to be reached. As far as the positive samples are concerned, the immunochromatography test was found to be able to unambiguously detect all samples with high antibody titres (specifically, those titres higher than 1/640), whereas anomalous results (or uncertain kit results) were obtained when analysing lower titres. As discussed above, it was found that only two of the eight samples with low IFI-based titres (1/80 or 1/160) analysed gave clearly positive results with the Uranotest Ehrlichia or Anigen rapid E. canis Ab kit. A further three tested negative and the remaining three gave very weak T bands that could lead to a negative result if read by an inexperienced analyst. Generally speaking, in light of the results obtained, the Uranotest Ehrlichia or Anigen rapid E. canis Ab test appears to present an excellent specificity and very good sensitivity. However, the presence of several false negative results for samples with low antibody titres suggests that some modifications may need to be made to the technique in order to increase its sensitivity, provided that this is one of the objectives to be achieved. Similarly, the existence of results that are difficult to interpret (samples with a barely visible T band, normally at low antibody titres) suggests the need for greater emphasis to be placed on how to correctly interpret the results of this technique in the instructions, especially in the event of uncertain findings. However, the low number of samples analysed suggests that the conclusions reached in this regard should be interpreted with care.