ABSTRACT
Syphilis is a health problem of increasing incidence in recent years that may have severe complications if not diagnosed and treated at an early stage. There are many diagnostic tests available for syphilis, but there is no gold standard, and diagnosis therefore usually relies upon a combination of tests. In this multicenter study, we evaluated the treponemal Elecsys syphilis assay for use in the diagnosis of syphilis in routine samples, i.e., when syphilis is suspected or during antenatal or blood donation screening. The sensitivity and specificity of the Elecsys syphilis assay were compared head to head with those of other treponemal assays used in routine clinical practice and were assessed in potentially cross-reactive samples from patients with Epstein-Barr virus, HIV, and Lyme disease. In a total of 8,063 syphilis-negative samples collected from routine diagnostic requests and blood donations, the Elecsys syphilis assay had a specificity of 99.88%. In 928 samples previously identified as syphilis positive, the sensitivity was 99.57 to 100% (the result is presented as a range depending on whether four initially indeterminate samples are included in the assessment). The specificity of the Elecsys syphilis assay in patients with other infections was 100%; no false-positive samples were identified.
INTRODUCTION
Syphilis is sometimes considered an old and often forgotten infection, but many countries have recently observed an increased incidence and large localized outbreaks of the disease (1–9). Epidemiologic data (from 2005) estimate a global annual incidence of 3.19 in men and 3.01 in women per 1,000, with 11 million new cases in 2005 and 36 million people infected with Treponema pallidum subsp. pallidum (the spirochete causative agent of syphilis) (7). In 2013 in the United States, the Centers for Disease Control and Prevention reported the prevalence of syphilis to be 0.05 cases per 1,000; as such, syphilis is still very much a current problem (10).
In the primary and secondary stages of syphilis, symptoms, such as painless sores, tiredness, and headaches, may be missed or mistaken for other conditions (11). If undetected in the primary or secondary stages, syphilis then enters the lengthy asymptomatic latent period; therefore, many people continue to be unaware that they are infected and that they can transmit the infection via sexual intercourse or during pregnancy. If untreated, the infection eventually progresses to the more serious symptomatic tertiary stage, which can cause significant complications, such as cerebral and vascular involvement (e.g., aortic aneurysm, general paresis, and tabes dorsalis) (12). However, syphilis can be successfully treated, particularly if it is diagnosed in the early stages (11). Diagnosis is therefore crucial so that treatment may be initiated earlier to improve outcomes and prevent transmission (11).
As well as being sexually transmitted, syphilis can be passed from the mother to the fetus during pregnancy. There is also a small risk of transmission via blood transfusion (13, 14), which is largely theoretical due to contamination concerns in the processing of blood products. It is estimated that 1.5 million pregnancies are affected globally each year and, if untreated, approximately 50% will suffer adverse outcomes, including miscarriage, stillbirth, or congenital syphilis (15). However, if syphilis is detected during pregnancy, treatment can reduce the risk of adverse outcomes associated with syphilis; because of this, syphilis screening is part of routine antenatal care (11, 16).
The diagnosis of syphilis requires the use of either serologic or nonserologic tests. Nonserologic methods directly identify T. pallidum; these include dark field microscopy (DFM) and PCR. Nonserologic methods are restricted to use in patients with particular clinical conditions (17). Other limitations include that the performance of DFM relies on the experience of the examiner, PCR is not widely available, and there is not an internationally approved method for T. pallidum identification (17, 18). As such, serologic tests are the preferred methods for diagnosing syphilis.
Serologic methods are relevant throughout the different stages of infection (i.e., primary, secondary, latent, and tertiary infection, once there is an immune response); serologic methods exploit the immune response elicited during infection. A number of different tests are available for detecting treponemal or nontreponemal antibodies. Treponemal tests, such as enzyme immunoassays (EIA), detect antibodies directed against T. pallidum antigens. Nontreponemal tests (including rapid plasma reagin [RPR]/Venereal Diseases Research Laboratory [VDRL] tests) use the phospholipid antigen cardiolipin to detect nonspecific serum antibodies present in most patients during early and partially treated T. pallidum infections (19).
To prevent the transmission of T. pallidum (the causative agent of syphilis) via blood transfusion, the World Health Organization (WHO) and the International Union against Sexually Transmitted Infections (IUSTI) recommend mandatory screening of all blood donations for specific treponemal antibodies (11, 20). The risk of syphilis via transfusion is now very small, with one case reported in the previous 50 years in the United States (13, 14), because T. pallidum does not typically survive >120 h outside the human host and under the cooled storage conditions of banked blood. However, the WHO still recommends the testing of blood samples (20–22), because the diagnosis of previous or active syphilis might somehow be a surrogate marker for behaviors that exposed the subject to the risk of acquiring other sexually transmitted viruses, such as HIV, hepatitis C virus (HCV), and hepatitis B virus (HBV).
As there is no gold standard for syphilis diagnosis, T. pallidum testing and algorithms for screening and confirmation vary between countries. In Europe, an EIA or agglutination assay is recommended for primary screening, followed by confirmatory testing if the result is equivocal or positive with a different treponemal antigen test (T. pallidum particle agglutination assay [TPPA]/T. pallidum particle hemagglutination assay [TPHA] if EIA is used for screening, EIA if TPPA/TPHA is used for screening) (11). The European guidelines do not recommend nontreponemal tests for the initial screening of syphilis; one reason for this is a lack of sensitivity in the late stages of infection (11). Testing in the United States traditionally has been performed using a different algorithm, initially using nontreponemal tests for screening, but now many centers are adopting treponemal tests as the primary screening assay (23, 24). In addition to their value in diagnosing syphilis at all stages, the use of treponemal tests has become more widespread due to the availability of T. pallidum recombinant polypeptides, which overcome the previous problem of antigenic variability of the proteins obtained from spirochetes grown in rabbit testes (25, 47).
The Elecsys syphilis assay (Roche Diagnostics, Mannheim, Germany) is an immunoassay for the in vitro qualitative determination of total antibodies against T. pallidum in human serum and plasma samples. It was developed to provide a fully automated treponemal screening assay that can be run on the same system as other Elecsys tests for infectious diseases.
Here, the aim was to evaluate the specificity of the Elecsys syphilis assay in routine samples that were sent by clinical request for syphilis testing, including samples from pregnant women and blood donors. The samples were assessed at clinical laboratories in Europe and Asia and included samples from patients and blood donors of various ethnic backgrounds. The specificity of the Elecsys syphilis assay using samples from patients with other infections (i.e., Epstein-Barr virus [EBV] and Lyme disease) that might be cross-reactive was evaluated. The Elecsys syphilis assay was also assessed for sensitivity using samples previously identified as syphilis positive and for specificity and sensitivity using samples from patients with confirmed HIV infection. The Elecsys syphilis assay was also compared head to head with other commercially available syphilis screening assays to evaluate the overall clinical performance of this novel method.
MATERIALS AND METHODS
Participating laboratories.Eight independent laboratories from six countries (Germany, Austria, Italy, Turkey, Thailand, and China) were involved in the evaluation, although not all investigations were performed by every laboratory. The role of each laboratory is indicated in Table 1.
Comparator assays, confirmation methods, Elecsys syphilis assay reagent lots, and samples tested at each of the laboratories
Samples.Fresh serum or plasma samples were provided by each laboratory to assess the specificities of the assays. All were leftover samples from routine requests or blood donations. All samples were anonymized or single coded prior to use for this study, and no follow-up and repeat testing of patients were possible.
The leftover samples were used to determine assay sensitivity and specificity in patients with HIV, Lyme disease, or EBV, which might cause cross-reactivity with the assay (special cohorts). The majority of these samples were stored at −20°C or −80°C prior to testing. Further details regarding the samples are given in Table 1. For the confirmed-positive samples with a defined clinical stage, the stage was determined by the clinician who provided the samples for testing.
Assays.The Elecsys syphilis assay is a double-antigen sandwich (DAGS) assay that simultaneously detects anti-treponemal IgG and IgM antibodies. A sample is incubated with a mixture of biotinylated and ruthenylated thermostable recombinant TpN15, TpN17, and TpN47 antigens to form a DAGS in the presence of the corresponding antibodies. Streptavidin-coated microparticles are then added, and the immune complexes bind to the solid phase by biotin-streptavidin interaction. The microparticles are magnetically captured on the electrode, and a voltage is applied to induce chemiluminescence, which is measured by a photomultiplier. The results are calculated automatically by the analyzer software, and the total assay time is 18 min.
Each site used the Elecsys syphilis assay, and the majority also used at least one of the following treponemal comparator assays: Architect syphilis TP (Abbott Laboratories, Wiesbaden, Germany), LIAISON Treponema screen (DiaSorin; Saluggia, Italy), Serodia-TPPA (Fujirebio, Tokyo, Japan), Vitros syphilis TPA (Ortho Clinical Diagnostics, High Wycombe, United Kingdom), Enzygnost syphilis (Siemens Healthcare Diagnostics, Marburg, Germany), and the Immulite 2000 syphilis screen (Siemens Healthcare Diagnostics). All comparator assays were performed according to the manufacturers' instructions.
Reagent lots.Three validation lots of the Elecsys syphilis assay were assessed to demonstrate reproducible specificity. The majority of the laboratories used lot A of the Elecsys syphilis assay to evaluate the performance of the test and demonstrate intersite comparability. Three sites used two additional reagent lots of the Elecsys syphilis assay (lots B and C) to allow interlot comparisons. Further details of the reagent lots used are given in Table 1.
Methods and analyses.The Elecsys syphilis assay results were expressed as a signal/cutoff (s/co) ratio, with an s/co of <1.00 indicating a negative result and an s/co of ≥1.0 a positive result. All routine or blood donation samples giving an initial reactive result were retested in duplicate using the Elecsys syphilis assay and considered to be repeatedly reactive if either of the results had an s/co of ≥1.0. The samples were then subjected to confirmatory testing.
For the majority of the comparator assays, the results were also expressed as an s/co ratio and were interpreted according to the manufacturers' guidelines. The exception was the Serodia-TPPA assay, for which the agglutination pattern was interpreted visually, and the results were expressed as titers and interpreted according to the manufacturers' guidelines.
Routine or blood donation samples that were initially reactive or equivocal using one of the comparator assays were retested (in duplicate or singly), even if the information for the users did not specify a requirement for retesting initially reactive samples. The samples with at least one repeat positive result or with repeat borderline results were considered to be repeatedly reactive and were subjected to confirmatory testing.
The methods used to confirm all initially or repeatedly borderline and reactive samples are shown in Table 1. Not all laboratories had all the necessary methodologies in place, and some samples requiring confirmatory testing were sent to a second participating laboratory or an independent external service laboratory to allow testing with all methods. The final assessment of a sample's status was based on the MIQ16 algorithm (25). In this study, following the use of this algorithm, all samples with a positive or equivocal screening result using a treponemal test were retested using a treponemal assay based on another test principle. The first retest was a fluorescent treponemal antibody (FTA) absorbance (ABS) IgG or polyvalent immunoglobulin assay, and the second was an IgG immunoblot. The samples reported to be confirmed positive were additionally tested by FTA-ABS IgM, IgM immunoblot, and a nontreponemal assay (e.g., VDRL or RPR) to evaluate the activity of the infection.
The precharacterized positive samples used for the sensitivity assessment were tested as single determinations, while repeat and confirmatory testing was undertaken for nonreactive samples (using FTA-ABS IgM, FTA-ABS IgG or polyvalent, TPPA or T. pallidum hemagglutination assay [TPHA], IgM immunoblot, IgG immunoblot, and VDRL or RPR). Similarly, the samples from the special cohorts were tested singly, with additional testing for the samples that gave discordant results between the assays.
RESULTS
Number of samples tested.A total of 8,079 samples were tested for specificity using the Elecsys syphilis reagent lot A. Of these, 3,500 were routine samples and 4,579 were leftover samples from blood donation screening. Fourteen confirmed-positive samples were excluded from the analysis, as were two samples with indeterminate results according to the applied confirmation algorithm.
Using lots B and C, a total of 2,681 leftover samples were assessed; 1,000 were routine samples, and 1,681 were samples from blood donation screenings. One sample assessed using lots B and C had indeterminate results and was excluded from the subsequent analyses.
The sensitivity of the Elecsys syphilis assay was tested in 928 banked samples previously identified as syphilis positive. A total of 922 samples were found to be positive, two samples were excluded from the sensitivity analysis due to indeterminate confirmation results, and four samples were initially nonreactive but were positive upon retesting.
The specificity using samples that were potentially cross-reactive was assessed in 225 samples from patients with HIV infection, 91 patients with EBV, and 10 patients with Lyme disease. None of the samples from patients with EBV or Lyme disease were reactive. Of the samples from patients with HIV infection, seven gave discrepant results, three were found to be positive on confirmatory testing, and four remained inconclusive.
The sensitivity of the Elecsys syphilis assay was also assessed in 228 samples from patients with confirmed HIV infection. A total of 226 were confirmed to be syphilis positive, and two samples were indeterminate but were later confirmed to be positive.
Specificity.The overall specificity of the Elecsys syphilis assay determined using lot A was 99.88% (8,053/8,063) and was superior or similar to that of the comparator assays (Table 2). The specificities of the Elecsys syphilis assay using the routine samples and blood donor samples as well as those of the comparator assays are shown in Table 2. The specificities of the Elecsys syphilis assay determined by the individual sites ranged from 99.59% to 100.00%, while those of the comparator assays were similar and ranged from 99.40% to 100.00%. Laboratory-specific data on each of the assays are given in Table S1 in the supplemental material.
Specificities of the Elecsys syphilis assay using lot A and specificities of comparator assays
The overall specificities as determined using lots B and C of the Elecsys syphilis reagents were equivalent, as shown in Table 3, and they were also similar to the specificities of the comparator assays. The specificities of lots B and C were also comparable to those achieved using lot A. However, in the comparator analysis using lots B and C, the overall specificity of the Architect syphilis TP assay was lower than that found in the comparator assays using lot A (99.21% versus 99.71%); this appears to be due to a lower observed specificity in the patient cohort from Izmir, Turkey.
Specificities of the Elecsys syphilis assay using lots B and C and specificities of comparator assays
To further assess the Elecsys syphilis assay, the specificities using samples from a cohort of patients with confirmed HIV infection were determined for the Elecsys syphilis and Immulite 2000 syphilis screen assays using 225 samples. No false-positive samples were identified for either test, and the specificity of the Elecsys syphilis assay in this special cohort was 100% (95% confidence interval [CI], 98.37 to 100.00%). Furthermore, no potential cross-reactivity was observed in the 101 samples positive for either EBV or Lyme disease. All 101 samples tested were negative using the Elecsys syphilis assay; 20/20 samples positive for EBV or Lyme disease tested negative using the Architect syphilis TP, LIAISON Treponema screen, and Immulite 2000 syphilis screen assays, while 81/81 samples positive for EBV tested negative using the LIAISON Treponema screen and Immulite 2000 syphilis screen assays.
Sensitivity.Although predefined as being positive for syphilis, four samples from two laboratories initially gave negative results using the Elecsys syphilis assay. These samples were found to be positive upon repeat testing and with the subsequent confirmation procedure. The sensitivity of the Elecsys syphilis assay was 100% when these samples were excluded from the analysis due to probable handling errors. However, if these samples are considered false-negative results, the sensitivity is 99.57%. The results observed at the individual sites, as well as the overall sensitivities for the Elecsys syphilis and comparator assays, are shown in Table 4.
Sensitivities of the assays tested, including individual site data
The clinical stage of infection was known for 698 samples. The sensitivities of the Elecsys syphilis assay according to stage of infection were as follows: primary, 100% (n = 101); secondary, 100% (n = 124); and latent, 99.37 to 100% (n = 473 to 470, including nine samples from the tertiary stage of infection). The ranges for the latent group depend on whether the three initially negative samples are considered to be false negative or are excluded from the analysis; the clinical stage was not known for the fourth initially negative sample.
In patients with both syphilis and HIV infection, the sensitivity of the Elecsys syphilis assay was 99.12% (95% CI, 96.87 to 99.89% [226/228]), compared with 98.68% (95% CI, 96.20 to 99.73% [225/228]) for the Immulite 2000 syphilis screen assay. For these investigations, the interpretation of the results considered the known effects of HIV coinfection on the immune response in addition to the confirmation algorithm; this included the potential downregulation of the immune response to TpN15 and TpN17 when interpreting the Mikrogen IgG recomLine assay (V. Sambri, personal communication).
DISCUSSION
This evaluation demonstrates that the Elecsys syphilis assay has an overall specificity (99.88%) and sensitivity (99.57 to 100.00%) that support its routine use. In addition, the specificity obtained using routine samples (99.80%), including predominantly samples from antenatal care, blood donations (99.93%), and samples from individuals with HIV infection (100.00%), demonstrates its suitability for screening for syphilis in all relevant patient groups and samples from blood donors.
The above results were obtained using reagent lot A, but the specificity of the Elecsys syphilis assay was also determined using two additional validation lots. The overall specificity was 99.85% for lot B and 99.89% for lot C, confirming that the assay is suitable for routine use.
Two of the laboratories assessed all three reagent lots, although different samples were used in the assessment of lot A versus lots B and C. At these laboratories, the specificities determined using lot A were slightly lower than those obtained using lots B and C. However, this is most likely an effect of sample size, as the 95% confidence intervals demonstrably overlapped.
The overall specificity results reported here were similar for the Elecsys syphilis assay and all other assays tested (Table 2) and compare favorably with those reported in the literature (26–40), as shown in Fig. 1A. With the Elecsys syphilis assay, there is clear discrimination between positive and negative results based on the s/co ratio and therefore no requirement for a gray zone to handle equivocal results. Hence, retesting is required to confirm positive results only.
Specificity (A) and sensitivity (B) ranges (with associated references) in routine samples and blood donors previously reported for the comparator assays compared with the individual site data from this evaluation. The Elecsys syphilis results in this figure were obtained using reagent lot A only.
When performing a head-to-head comparison using daily routine blood donations, a preselection bias is present. For safety reasons, blood banks exclude donors with false-positive results for a particular infection from future blood donations. A new assay containing unique or different components may therefore report a larger number of discrepant results than that with the routine assay. Hence, the assay that is routinely used by the blood bank is favored by such comparisons. Despite this, the results from individual laboratories comparing the Elecsys syphilis assay with the assays in routine use for blood donation screening demonstrated a higher or equivalent specificity for the Elecsys syphilis assay, further confirming the suitability of the Elecsys syphilis assay for routine use in blood banks.
In this evaluation, the confirmation of a positive screening result was based on the MIQ16 algorithm (25), but, as described earlier, confirmatory algorithms vary between countries, as there is no gold standard (11, 23, 24). Given the specificities of the immunoassays tested in this evaluation, it appears to be appropriate to use these assays for screening and to confirm positive results using an alternative treponemal test (19, 23).
In the sensitivity analysis, all four samples that were previously determined to be syphilis positive but gave an initial negative result with the Elecsys syphilis assay were confirmed to be positive using up to four separate tests, as well as on retesting using the Elecsys syphilis assay. The initial negative result was likely due to a technical factor, and the most probable reason is that frozen rather than fresh samples were tested. Frozen samples can be more challenging to handle due to the formation of fibrin filaments, and it is possible that sample handling issues might have contributed to a negative result. If all initially negative samples are assumed to be false negative, the overall sensitivity of the Elecsys syphilis assay is 99.57%. However, as these were banked samples that were not treated in the same way as the routine samples, it is appropriate to exclude them from the analysis; using this scenario, the sensitivity of the Elecsys syphilis assay is 100%. Both results are included here.
The sensitivity of the Elecsys syphilis assay was comparable to that of the other assays tested, and the results also compare favorably with those reported in the literature, as shown in Fig. 1B (26–29, 31–41). Furthermore, a reliable detection of syphilis was observed at all stages of infection and in patients coinfected with HIV. Coinfection with syphilis and HIV is common, as syphilis is associated with an increased risk of acquiring HIV, partly due to the association between HIV infection and symptomatic genital ulcer disease (42). In addition, HIV can adversely affect the serologic response to syphilis and may affect the ability to diagnose the infection (42).
The Elecsys syphilis assay uses three antigens to ensure appropriate sensitivity. The response against TpN47 is one of the earliest detectable responses in the disease course; therefore, TpN47 is useful in the diagnosis of primary syphilis (43, 44). The reliable diagnosis of early syphilis is essential given the potential use of the assay to screen samples collected during pregnancy and from blood donors. The incorporation of the TpN15, TpN17, and TpN47 antigens, which are detectable throughout the disease course (45), can optimize the sensitivity in patients with HIV and possibly an abnormal immune response. In contrast, the LIAISON Treponema screen and Immulite 2000 syphilis screen assays use the TpN17 antigen only. As demonstrated here, the triple antigen combination of the Elecsys syphilis assay had no apparent loss in specificity compared to that of the comparator assays; furthermore, the antigen combination was not generally associated with cross-reactivity (46).
In summary, the Elecsys syphilis assay represents a state-of-the-art reliable treponemal test for screening for syphilis in both clinical and blood donation settings.
ACKNOWLEDGMENTS
Funding for the study was provided by Roche Diagnostics GmbH (Penzberg, Germany). Rebecca Gardner (Elements Communications Ltd., Westerham, United Kingdom) provided medical writing assistance supported by Roche Diagnostics GmbH.
We thank S. Richardt and R. Hofweber and the Clinical Research Team at Roche Diagnostics GmbH for their assistance with the evaluation.
V. Sambri declares speaker's fees from Novartis Diagnostics and Roche Diagnostics. V. Sambri has received research funding from Roche Diagnostics, Novartis Diagnostics, DiaSorin, and bioMérieux. H. Schennach received travel fees for attendance at an advisory board for Roche Diagnostics.
FOOTNOTES
- Received 18 July 2014.
- Returned for modification 15 August 2014.
- Accepted 20 October 2014.
- Accepted manuscript posted online 29 October 2014.
Supplemental material for this article may be found at http://dx.doi.org/10.1128/CVI.00505-14.
- Copyright © 2015, American Society for Microbiology. All Rights Reserved.
The authors have paid a fee to allow immediate free access to this article.
REFERENCES
