ABSTRACT
A multitest sequential algorithm based on rapid and simple (R/S) assays was applied for the diagnosis of HIV infection among participants in a phase 3 microbicide effectiveness trial. HIV testing was performed on finger-prick blood samples obtained from patients after their enrollment in the trial. The specimens were tested in a serial procedure using three different rapid tests (Determine HIV-1/2 [Abbott], SD Bioline HIV-1/2 3.0 [Standard Diagnostics], and Uni-Gold HIV [Trinity Biotech]). In the event of discordant results between the Determine HIV-1/2 and SD Bioline HIV-1/2 3.0 tests, the third assay (Uni-Gold HIV) determined the final outcome. When the final outcome was positive, a second specimen was collected and tested with the same algorithm, only if a positive result was obtained with this sample the participant was informed of her positive serostatus. A total of 5,734 postenrollment specimens obtained from 1,398 women were tested. Forty-six women tested positive according to the testing algorithm performed on the first collected specimen. Confirmatory testing results obtained at the ITM confirmed that 42 women were truly infected. Two of four initial false positives tested negative upon analysis of a second blood specimen. The other two tested false positive twice using specimens collected the same day. A high percentage of specimens reactive with the Determine HIV-1/2 assay was only observed at the study site in Kampala. This result did not appear to be associated with pregnancy or malaria infection. We conclude that HIV testing algorithms, including only R/S assays, are suitable for use in clinical trials, provided that adequate quality assurance procedures are in place.
INTRODUCTION
Phase 2b/3 trials assessing the effectiveness of products to prevent the heterosexual acquisition of HIV are almost exclusively conducted in African and Asian countries with HIV epidemics and high incidence of infection. Even in such settings, effectiveness trials must be conducted in multiple centers to observe the large number of HIV infection events required to achieve adequate statistical power to detect a treatment effect.
Study clinics may be located in remote areas, small towns or villages, where well-equipped laboratories are lacking. There is often access only to a small on-site laboratory where rapid and simple (R/S) assays to detect HIV antibodies are performed. Testing using enzyme-linked immunosorbent assays is preferable when large numbers of specimens are to be tested; however, equipment, storage of reagents in refrigerated conditions, and extensive training of laboratory staff is required for these assays. In contrast, R/S assays require little to no equipment and have no storage restrictions (if ambient temperatures are below 27°C). In addition, the use of R/S assays (compared to traditional multiwell immunoassays) has the advantage that results can be provided the day samples are collected, greatly reducing the likelihood that individuals will fail to receive their test results. As with multiwell immunoassays, there exists a time interval between infection and detectable antibody levels (i.e., a window period, often measured in weeks), and false-positive results can never be excluded (4). Nonetheless, once patients are out of the window period, the sensitivity and specificity of the R/S HIV assays are high and comparable to multiwell immunoassays (2, 24, 25, 26).
The HIV testing algorithms described in the literature often include the confirmation of reactive rapid or immunoassay tests by supplemental testing such as Western blot (WB) (3, 4, 19). Performing WB assays requires additional training of the laboratory staff and specific equipment and is expensive. The UNAIDS/WHO program on HIV/AIDS testing proposes three testing strategies using R/S assays without the requirement for additional confirmatory testing, making them affordable and suitable for application in small laboratories (2, 7, 13, 14). To maximize the specificity and positive predictive value, multitest sequential algorithms have been implemented. However, the sensitivity of any such algorithm can only be as high as that of the first test in the sequence (1). Here, we present the results obtained with a sequential HIV multitest algorithm based on R/S assays used for the diagnosis of HIV infection among participants in a phase 3 microbicide effectiveness trial.
MATERIALS AND METHODS
Study design.The study in which the R/S algorithm was used has been described elsewhere (23). Briefly, the study was a randomized double blind placebo-controlled trial of a candidate vaginal microbicide (cellulose sulfate gel) for HIV prevention. Women were recruited at five sites: a community clinic and a clinic for sexually transmitted infections in Cotonou, Benin; the Y. R. Gaitonde Center for AIDS Research and Education (Y.R.G. CARE) in Chennai, India; the Medical Research Council in Durban, South Africa; the Mulago Hospital (Makarere University) in Kampala, Uganda; and clinics in Mudhol and Jhamkandi in Karnataka, India (in collaboration with the Karnataka Health Promotion Trust, Bangalore). Eligibility criteria included negative HIV antibody tests at screening and enrollment and agreement to visit the clinic for HIV testing 1, 3, 6, 9, and 12 months after enrollment.
HIV testing was performed on serum samples at screening, on plasma samples at enrollment, the final visit, and at any visit at which product use was discontinued, and on whole blood obtained by finger prick at the follow-up visits at 1, 3, 6, and 9 months. The sites applied their national or local HIV testing algorithm for the determination of HIV infection at screening and enrollment, but the sequential R/S study algorithm (Fig. 1) was applied on all other visits. We report here the results and performance of the HIV algorithm used during follow-up and at the final visit and do not include the evaluation of the national or local testing algorithm used at screening and enrollment.
HIV testing algorithm used in the trial during follow-up visits and at final visits.
Ethical approval.The study was approved by the Institutional Review Board of the Eastern Virginia Medical School and of the Institute of Tropical Medicine and by local ethics committees at each site where women were recruited. All approvals were granted prior to study initiation. Participants gave written informed consent before screening and enrollment. The trial was conducted under the Food and Drug Administration's Investigational New Drug application number 69,107 and registered at clintrial.gov (study number NCT00153777).
Specimens.Whole blood was collected in an EDTA Microtainer (Becton Dickinson, Sparks, MD) by finger prick using a Glucolet 2 device (Bayer HealthCare LLC, Mishawaka, IN) at 1-, 3-, 6-, and 9-month follow-up visits. The collected volumes ranged between 250 and 500 μl. At the final visit, permanent product withdrawal visit or in the event the first finger-prick blood sample was reactive for HIV antibodies, 7 ml of blood was collected in an EDTA blood collection tube and processed within 4 h. After centrifugation at 800 × g (± 2,000 to 2,200 rpm) at 20 ± 5°C for 10 min, the plasma was transferred to a 15-ml conical tube under sterile conditions. The tube was centrifuged for 30 min at 1,200 × g (± 2,200 to 2,400 rpm) at 20 ± 5°C. After HIV R/S testing on site, plasma aliquots were stored at −70°C until shipment on dry ice to the Institute of Tropical Medicine (ITM), Antwerp, Belgium, for additional testing.
HIV assays.Postenrollment specimens were tested according to a sequential HIV R/S algorithm based on the modified WHO/UNAIDS testing approach for strategy III, which was also included as point-of-contact algorithm 4 in the status report on HIV testing algorithms published by the Association of Public Health Laboratories (1, 21). Specimens were first tested with the Determine HIV-1/2 test (Abbott Laboratories, United Kingdom). If a sample was reactive, it was tested with the SD Bioline HIV-1/2 3.0 test (Standard Diagnostics, South Korea). In the event of a discordant result, the Uni-Gold HIV test (Trinity Biotech, Ireland) was used. The final result of the algorithm scored a sample as positive if two of the three R/S HIV assays were reactive. In this case, a second specimen was collected from the participant, preferably on the same day, and tested with the same algorithm. The participant was informed of her positive serostatus only if a positive result with the second specimen was obtained.
The ITM tested additional specimens to exclude the possibility that participants entered or exited the study during their window period. To this end, all enrollment plasma specimens from participants with detectable HIV antibodies within 3 months after enrollment and all plasma specimens collected at the final visits of patients without detectable HIV antibodies were tested by using a qualitative HIV-1 RNA PCR assay (COBAS AmpliScreen HIV-1 test; Roche Molecular Systems). Due to the large number of specimens, pools of a maximum of 24 final visit plasma samples were tested according to the manufacturer's specifications.
In the context of the quality control established for the study, aliquots of stored plasma specimens from participants who tested HIV positive after enrollment were also tested at the ITM to confirm these results using a different testing algorithm, which is presented in Fig. 2.
HIV testing algorithm used for retesting and final confirmation of HIV seroconversion at the Institute of Tropical Medicine, Belgium. EIA 1, Enzygnost Anti-HIV 1/2 Plus (Dade Behring GmbH, Germany); EIA 2, Vironostika HIV Uni-Form II Plus O (Organon Teknika, France); LIA, INNO-LIA HIV I/II Score (Innogenetics, Belgium); Ag, INNOTEST HIV Antigen MAb (Innogenetics, Belgium); IND, indeterminate; +, positive; -, negative.
The details regarding guaranteeing the quality of the HIV results obtained at the different study sites have been reported previously (5). In brief, this quality control assessment consisted of the assessment of the study site laboratories, provision of hands-on training, conduction of supervisory visits, distribution of quality and batch control panels, and retesting of predefined specimens.
Pregnancy and malaria testing.Participants were tested for pregnancy at all visits using a urine HCG rapid test. When malaria was suspected, a thick blood smear film was examined for the microscopic visualization of Plasmodium parasites.
Statistical analysis.Overall, 38.8% of the potential participants were HIV positive at screening and thus excluded from study participation (Table 1). Here, we summarize the results of the sequential R/S HIV algorithm as applied to 5,734 postenrollment specimens of 1,398 enrolled women, who had at least one follow-up HIV test specimen, as well as confirmatory specimens collected after an initial positive result (specimens collected after a confirmed positive HIV result and specimens contributed by three women later found to have been in their window period at enrollment are excluded). Descriptive tests of associations between participant characteristics and discordant rapid test results were conducted using logistic regression with generalized estimating equations to account for repeated measures on each enrolled participant.
HIV prevalence rates in women who presented for screening at the five study sites
Final confirmation of positive results was performed at the ITM according to the testing algorithm presented in Fig. 2 and/or using the HIV-1 RNA PCR assay. Negative results were confirmed by testing all plasma samples collected at final visits with the HIV-1 RNA PCR assay.
RESULTS
A total of 46 first collected specimens tested positive according to the sequential R/S testing algorithm at the sites (Table 2), with all but one specimen reactive with both Determine HIV-1/2 and SD Bioline HIV-1/2 3.0 (Fig. 3). Confirmatory results were available for all 46 women, with 42 (91.3%) confirmed to be HIV-1 positive and 4 (8.7%) confirmed to be HIV negative (1 of the 42 confirmed positives became infected after the administrative censoring date of the trial and was excluded from the primary effectiveness analysis reported in reference 23). HIV-2 infection was not detected in the postenrollment specimens. According to the confirmed results of a total of 5,734 specimens tested, the sequential R/S HIV testing algorithm showed an initial sensitivity of 100% (95% confidence interval [CI] = 89.6 to 100%) and an initial specificity of 99.9% (95% CI = 99.8 to 100%). One of the four women with negative confirmatory results was subsequently lost to follow-up, two of the four had HIV-negative results at three or more subsequent visits, and one had a positive HIV result on her first but a negative result on her second collected specimen, followed by negative results at her last two visits. The initial positive results for these four participants were considered false positive.
Positive HIV rapid test results of first specimens collected at follow-up
HIV test results on the first specimens collected.
For two of the participants, including the participant with an unconfirmed positive result at her next visit, the final result was false positive. We cannot exclude that the false-positive results were partially caused by the use of deteriorated Determine HIV-1/2 test strips. Indeed, we observed that the Determine HIV-1/2 test strips were not stored properly and were exposed to humidity at this test site. In addition, the SD Bioline HIV-1/2 3.0 assay had a false reaction leading to a final false-positive result.
There were 124 specimens from 98 women with a reactive result in the Determine HIV-1/2 test only (i.e., both SD Bioline HIV-1/2 3.0 and Uni-Gold HIV were not reactive; Table 3). Five of the 98 women were not tested again, 2 were HIV antibody positive according to the R/S algorithm at their next study visit, and 91 were subsequently tested but never diagnosed with HIV. Of the 91, 22 (24.1%) were reactive only in the Determine HIV-1/2 test on multiple occasions (Table 4).
Specimens reactive only in the Determine HIV-1/2 assay
Women with reactive results only in the Determine HIV-1/2 assay
The proportion of specimens that were only reactive in the Determine HIV-1/2 assay was significantly higher in Uganda (7.2%) than in the study as a whole (2.2%; P < 0.001). Given this observation, we sought to explore the possible influences of pregnancy and malaria on the performance of the Determine HIV-1/2 assay. Pregnancy and malaria were both highly prevalent in Uganda, and there is evidence in the literature to suggest that these conditions may impact the specificity of the Determine HIV-1/2 assay (9, 10, 11, 15, 18, 28). However, we found no significant difference in the proportion of specimens in which only the Determine HIV-1/2 test was reactive between pregnant and nonpregnant women (3.6% reactive among women testing positive for pregnancy, 2.1% for women not testing positive for pregnancy; P = 0.24). Malaria was only diagnosed in Benin and Uganda. Restricting analysis to those two sites, we likewise found no significant difference in the proportion of specimens in which only the Determine HIV-1/2 test was reactive according to the malaria status (4.7%, regardless of previous malaria diagnosis; P = 0.61).
Having a reactive result only in the Determine HIV-1/2 assay was significantly and positively associated with the probability of obtaining a reactive result only on the Determine HIV-1/2 test at a subsequent visit (P = 0.014), but this result is confounded by the effect of site (20 of the 22 women with repeat samples only reactive in the Determine HIV-1/2 test were from Uganda). Given these results, we are unable to explain the high prevalence of specimens reactive only in the Determine HIV-1/2 test in Uganda.
Using the qualitative HIV-1 RNA PCR assay on stored plasma specimens, three participants were found to be HIV infected at enrollment. These three participants were from Uganda, Benin, and South Africa; sites which used a different testing algorithm at enrollment than at follow-up and hence do not provide information on the utility of the R/S algorithm presented here. Another participant was found to be recently infected at her final visit.
DISCUSSION
HIV testing plays a crucial role in HIV prevention clinical trials as it determines the eligibility of potential participants and is used to identify the primary study endpoints.
The use of R/S HIV tests provides the opportunity for small or field-based laboratories to deliver HIV results on the same day as the study visit. Moreover, R/S tests do not require refrigeration as long as temperatures are below 27°C, nor do they require special equipment, processing of specimens in batches, or highly skilled laboratory staff.
In general, multiwell immunoassays are considered to be superior to R/S tests for the detection of HIV antibodies (16). However, all but one infection taking place during the present study were detected using the sequential R/S HIV algorithm. The single missed infection was detected at the patient's final visit using a PCR RNA test and was most likely in the window period. This recent infection would thus probably not have been detected using a third-generation immunoassay either.
The advantage of most R/S assays is that they can be performed on finger-prick blood, in contrast to multiwell immunoassays, for which venous blood has to be collected. We chose to collect 250 to 500 μl of whole blood from finger pricks in an EDTA Microtainer because the procedure is less invasive and often more acceptable to participants and provides sufficient volume to perform the three R/S assays included in the testing algorithm.
Testing of a second specimen following an initial positive HIV algorithm result was done to exclude transcription errors and to control for possible specimen mix-ups (20). This procedure is routine practice in Belgium but not in other countries. The first specimen tested was blood obtained via finger prick, which in some instances may influence the reliability of the visual reading of the R/S test (12). Among the four participants with HIV false-positive results, one participant had substances in her blood that cross-reacted at all testing occasions with the SD Bioline HIV-1/2 3.0 assay. For this participant only, the sequence of the rapid assays was changed to the Determine HIV-1/2 test first and in the event of a reactive result, the Uni-Gold HIV test was performed instead of the SD Bioline HIV-1/2 3.0.
We found 124 of 5,734 (2.2%) specimens that were only reactive in the Determine HIV-1/2 test, with a much higher rate (7.2%) in Uganda than at any other site. The lack of specificity of the Determine HIV-1/2 assay with specimens from Uganda has been reported previously (6, 12, 22). We hypothesize that in Kampala, Uganda, in particular, participants had undiagnosed endemic non-HIV infections or circulating substances in their blood that may have cross-reacted with the Determine HIV-1/2 test (12). Cross-reactivity of HIV assays is not uncommon; a recent publication reported cross-reactivity of the Murex HIV Ag/Ab combination enzyme immunoassay with S. haematobium IgG and a study from our group showed cross-reactivity of the Determine HIV-1/2 test with Trypanosoma brucei gambiense (8, 17). The 124 results came from 98 women, and only 2 were ultimately diagnosed with HIV during the trial. This counters the impression expressed by laboratory staff in the field that the Determine HIV-1/2 test is more sensitive at detecting recent infections.
It should be noted that the Determine HIV-1/2 rapid test was initially developed by Abbott; however, in May 2005 Abbott entered an agreement to sell the rapid test to Inverness Medical Innovation. At present, the rapid test is manufactured by Alere Medical in Japan. According to the new manufacturer, the performance characteristics of the assay have not changed with the change in manufacturer.
However, we note that some of the false-reactive results obtained with the Determine HIV-1/2 assay could be attributed to the deterioration of the test strips that was observed at the start of the study and the overinterpretation of reddish “shadows” or “ghost lines” especially when finger-prick blood was tested (15). Retraining on the storage and handling of the strips and test procedure, including the interpretation of “reactivity,” was provided (5). It should be noted that according to the guidelines of the World Health Organization (WHO), the discordance of test results between first and second or tiebreaker tests should not exceed 5% (27). Although the number of false-reactive Determine HIV-1/2 assays obtained in Kampala decreased following training and thorough supervision, it remained ca. 5%. This observation reinforces our hypothesis that interfering blood substances played a role in the observed false-reactive Determine HIV-1/2 results. In addition, this finding emphasizes the need for and importance of assay and algorithm validation in the populations and geographical areas in which assays will be used (7, 12).
Although rapid assays are simple, training in the test procedure and reading of the test devices is still necessary (5, 12). Likewise, quality control and assurance procedures should be put into place for any clinical trial. First, the quality of the test devices should be assessed when received by the laboratory. Once the test devices pass batch-entry control, internal controls should be run at regular intervals. The staff administering the tests should also be assessed at a predefined frequency by testing quality control panels (20). Although the assays in the present study do not require refrigeration, they cannot be stored at temperatures higher than 27°C, and the tests should not be performed in conditions exceeding those temperatures. In addition, the test devices are susceptible to humidity and need to be stored in sealed pouches containing a desiccant.
With the use of a qualitative HIV RNA PCR assay, we were able to more accurately determine the time of infection and exclude three participants from the analysis who were found to be HIV infected but still in their window period at enrollment. We were also able to detect one early/recent infection in a participant who did not develop HIV antibodies at her last study visit and who was likely in her window period. Not only could this endpoint be included in the data analysis and improve the precision of the estimated treatment effect, the use of PCR on the final visit sample allowed the participant to be referred for further HIV infection management. Excluding this recent infection detected at a final visit, no additional infections other than those determined by the HIV testing algorithm were detected using HIV RNA PCR. This observation confirms the sensitivity of the HIV testing algorithm and its suitability in clinical trials.
In conclusion, our results demonstrate that HIV algorithms using only rapid and simple HIV assays with results confirmed on a second specimen provide reliable HIV results and can easily be implemented in HIV prevention trials conducted in resource-limited areas. If more accurate time point determinations of HIV infection are required for outcome analysis, we recommend using HIV RNA PCR in a look-back procedure on stored specimens.
ACKNOWLEDGMENTS
We thank all study participants and study staff, especially the laboratory staff, at the study sites for their dedication. We thank Marjan Mangelschots, Mirriam Van Rooy, and Sergio Garcia for the HIV testing and Wendy Thys for her data entry work.
This study was supported by grants from the U.S. Agency for International Development and the Bill & Melinda Gates Foundation.
No potential conflict of interest relevant to this article was reported.
FOOTNOTES
- Received 6 April 2011.
- Returned for modification 5 May 2011.
- Accepted 6 July 2011.
- Accepted manuscript posted online 13 July 2011.
- Copyright © 2011, American Society for Microbiology. All Rights Reserved.
REFERENCES
