Previous Article | Next Article ![]()
Clinical and Vaccine Immunology, December 2007, p. 1545-1549, Vol. 14, No. 12
1071-412X/07/$08.00+0 doi:10.1128/CVI.00120-07
Copyright © 2007, American Society for Microbiology. All Rights Reserved.

The Johns Hopkins University, School of Medicine, Division of Infectious Diseases, Baltimore, Maryland,1 National Institute of Allergy and Infectious Diseases, National Institutes of Health, Baltimore, Maryland,2 Johns Hopkins Bloomberg School of Public Health, Baltimore, Maryland,3 Department of Laboratory Medicine, University of Washington, Seattle, Washington4
Received 12 March 2007/ Returned for modification 20 June 2007/ Accepted 27 September 2007
|
|
|---|
2 analysis, might also affect the clarity of the Biokit test. The recommended index cutoff value of 1.1 for the Focus and Kalon assays was found to be optimal for this population. |
|
|---|
The Focus assay for HSV-2 has previously been shown to have sensitivities ranging from 97% to 100% and specificities ranging from 52% to 100% compared to the results of HSV-2 Western blotting, the current "gold standard" in HSV-2 detection technology (20, 22). The Focus assay is currently the only assay for HSV-2 cleared by the Food and Drug Administration for clinical use in the United States. The Kalon ELISA for HSV-2 has likewise been shown to perform well, with sensitivity estimates being between 92.3% and 100% and specificities ranging from 97.7% to 100% (15). The Biokit rapid assay is used as a point-of-care test for the detection of HSV-2 antibody. The major benefits of a point-of-care method are that it requires no additional materials beyond the components of the kit and the results can be given to the patient immediately. Premarket evaluation showed that the Biokit assay has a sensitivity of 96% and a specificity of 98% (4). The Biokit assay is especially advantageous for researchers or clinicians working in remote locations with limited access to clean water, reliable electricity, and laboratory equipment, such as a plate reader.
Variations in the performance of these assays most likely result from differences in population compositions and locales. Many of the studies that have evaluated the Focus, Kalon, and Biokit assays have been done with sexually transmitted disease (STD) clinic populations; however, these studies generally did not identify the particular STDs present in the population. Therefore, a primary goal of our study was to determine if other sexually transmitted infections had any effect on these assays, as well as to investigate the overall performance of the assays.
|
|
|---|
The serum samples were tested for hepatitis C virus (HCV; HCV ELISA; Ortho Raritan, NJ) and HIV (Vironostika HIV-1 ELISA; Biomerieux, Durham, NC). Confirmation of the results for the HIV-positive samples was performed by HIV-1 Western blotting (Bio-Rad Laboratories, Redmond, WA). Testing for the presence of infection by Chlamydia trachomatis, Neisseria gonorrhoeae, Trichomonas vaginalis, or Mycoplasma genitalium was performed with urine samples by the protocols described below.
The urine samples were evaluated for the presence of C. trachomatis and N. gonorrhoeae infections by the Aptima Combo 2 assay (GenProbe, San Diego, CA), and the results for samples with positive results were confirmed with the Aptima CT and Aptima GC confirmatory assays, respectively. Testing for Trichomonas vaginalis was performed by use of the Aptima TMA research assay (GenProbe) and a real-time T. vaginalis PCR assay (Johns Hopkins University, Baltimore, MD) simultaneously (9). A positive result by both assays was required to consider the patient infected with T. vaginalis. Concurrent testing for M. genitalium was performed by using a multitarget real-time PCR assay (Johns Hopkins University) and a transcription-mediated amplification research assay (GenProbe) (10). The results of both assays were required to be positive for the patient to be considered infected with M. genitalium. All assays were performed according to their respective manufacturers' package inserts or instructions.
Samples with sufficient remaining sera (n = 279) were further tested for the presence of HSV-2 antibodies by three commercially available immunosorbent assays: the Biokit assay, the Focus assay, and the Kalon assay. Any samples that tested positive or equivocal by an assay for HSV-2 were analyzed for HSV-1 and HSV-2 proteins by Western blotting, which was performed at the University of Washington, Seattle.
The Biokit rapid test results are dependent on the subjective judgment of the technician who performs the testing. Therefore, the results of all Biokit tests were independently reviewed by two technicians for validation. Specimens that had a clearly distinguishable test spot were considered positive, and specimen test devices with no test spot were considered negative (Fig. 1). The results for specimen test devices with faint colorations were considered indeterminate. Additionally, if the technicians disagreed on a test result, it was considered indeterminate. A few methods of interpretation of these faint test spots have been noted, mainly, that they should be considered negative, that the result should be verified by an alternative assay, or that the sample should be retested (22). For the purposes of comparison, a third interpretation, in which faint coloration is considered a positive result, was reviewed. Our recommendation for the best possible method of interpretation was based on the greatest sum of sensitivity and specificity. Multiple logistic regression analysis was used to assess and quantify the effects of individual STD infections on the indeterminate results of the Biokit assay for HSV-2 for the entire study population. When the "true disease" status, as determined by Western blotting, was not available for all subjects,
2-test values were calculated to evaluate the association.
![]() View larger version (66K): [in a new window] |
FIG. 1. Positive, indeterminate, and negative results by the Biokit assay for HSV-2. Note that the test spot for a positive result is most often not as bright as the control spot but is still clearly discernible.
|
All data were analyzed with Intercooled software (version 9; STATA Corp., College Station, TX). All P values were considered significant if they were less than or equal to 0.05.
|
|
|---|
|
View this table: [in a new window] |
TABLE 1. Present STD coinfections by HSV-2 serostatus in men attending Baltimore STD clinics
|
![]() View larger version (28K): [in a new window] |
FIG. 2. HSV-2 serological status of 136 male patients from Baltimore STD clinics, as determined by the Kalon, Focus, and Biokit assays for HSV-2 and as confirmed by Western blotting. HSV-1 serological status was determined solely by Western blotting. Only the data for samples whose results were confirmed by Western blotting are presented. Note that four samples had atypical HSV-2 Western blot results and those results are not included here. pos, positive; neg, negative.
|
Neisseria gonorrhoeae infection showed a significant effect on the indeterminateness of the Biokit assay for HSV-2. After the other STDs within the population, except for HSV-1, were controlled for, it was found that patients infected with N. gonorrhoeae were 3.88 times more likely to have an indeterminate Biokit assay result than a clearly positive or a clearly negative test spot (P = 0.001; 95% confidence interval, 1.78, 8.45) (Table 2).
|
View this table: [in a new window] |
TABLE 2. Multiple-regression analysis of effect of concurrent STD infections on Biokit assay results (determinate or indeterminate) in men attending Baltimore STD clinics
|
2 test). For this STD clinic population, when four samples with atypical Western blotting results are excluded, it was found that the Focus assay for HSV-2 had a sensitivity much lower (82.6%) than that published previously, but it was still highly specific (Table 3). The Kalon assay performed very well and in accordance with previous performance estimates. Receiver operating characteristic analysis confirmed an optimal index cutoff value of 1.1 for both the Kalon and Focus assays (results not shown).
|
View this table: [in a new window] |
TABLE 3. Summary of performance analysis for the Focus, Kalon, and Biokit HSV-2 antibody detection assays in men attending Baltimore STD clinics, using Western blotting results as the gold standard
|
|
|
|---|
Assessment of the performance of the Biokit rapid assay for HSV-2 is largely dependent on the interpretation of an indeterminate, faint coloration of the test spot. Of the Biokit assay results for the 279 subjects tested in this study, 19.7% were unclear to the two technicians. Outside of these ambiguous results, the Biokit assay was fairly specific, although it was not as sensitive as estimated previously. Others have noted similar difficulties in the evaluation and the frequency of faint coloration in near-patient assays for the diagnosis of HSV-2 infection and have cautioned against the overinterpretation of such results, which would cause a decrease in the specificity of the assay compared to that of Western blotting (6, 19).
In this study, over half of the subjects with indeterminate Biokit assay results were HSV-1 positive and HSV-2 negative. This significant association suggests a possible cross-reactivity of the kit components with HSV-1-specific antibodies in the absence of IgG specific to HSV-2. Morrow et al. also noted this relationship (17). Conversely, it has been shown that patients who have low-positive index values from the Focus assay for HSV-2 and who are serologically negative for HSV-2 by Western blotting are more likely to have recently seroconverted (11). Perhaps this faint coloration in the Biokit assay is also a reflection of early infection in these patients. Seroconversion data were not available for analysis in this study.
Additionally, there was a significant association between the detection of N. gonorrhoeae in patients and an indeterminate Biokit assay result (odds ratio = 3.88; P = 0.001). This relationship may be due, in part, to the cross-reactivity of N. gonorrhoeae-specific antibodies with the assay's target antigen or additional components of the assay. If N. gonorrhoeae infection elicits a general immune response with elevated levels of acute-phase reactants, these reactants may interact with the components of the Biokit test. The Biokit assay manufacturer's protocol mentions that possible cross-reactions could occur; however, it does not specifically include N. gonorrhoeae, which was highly prevalent in this population. We were not able to control for HSV-1 infection status in the regression model, which may also affect the clarity of the Biokit assay result.
The suggested method of Biokit assay indeterminate result interpretation, which considers a faint coloration to be a negative outcome, produced a combined sensitivity and specificity that was lower than those reported previously (4, 12, 17). The combined sensitivity and specificity was higher if the samples with indeterminate results were tested by a subsequent assay rather than if samples with indeterminate results were considered negative. Therefore, we recommend that the Biokit assay not be used without confirmatory testing of samples with indeterminate results by an alternate assay or repeated testing.
Each of the three assays has particular strengths and weaknesses. For instance, the Biokit test is extremely easy to use and a result can be obtained within minutes; however, the final result can be ambiguous. The Kalon and Focus assays show higher sensitivities and specificities but take several hours to run. They also require equipment beyond that which is provided in the kits, such as a plate reader, a plate washer, reliable electricity, and clean water, to name a few. To enhance the sensitivity and the specificity of testing for HSV-2 in a primarily African-American male population, we looked at 15 serial and parallel algorithms that use combinations of the Kalon, Focus, and Biokit assays. We found that the Kalon assay alone outperforms all of the possible algorithms. It is important to note, however, that the Kalon assay is currently not cleared for clinical use by the Food and Drug Administration.
We would like to reiterate that the results discussed above pertain primarily to a male high-prevalence population and may not extend to female or general populations. Another limitation to interpreting the results is that not all samples were tested for HSV-1 and/or HSV-2 by Western blotting. This could influence the calculated sensitivities and specificities, even though appropriate mathematical adjustments were made. Finally, previous studies have shown that the times from primary infection to detectable seroconversion differ between assays for HSV-2 and that these differences can affect the comparative performance of any of these assays (2).
This research was supported by the Division of Intramural Research, NIAID, NIH.
Published ahead of print on 3 October 2007. ![]()
|
|
|---|
| |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Copyright © 2009 by the American Society for Microbiology. For an alternate route to Journals.ASM.org, visit: http://intl-journals.asm.org | More Info»