Previous Article | Next Article ![]()
Clinical and Vaccine Immunology, November 2007, p. 1458-1464, Vol. 14, No. 11
1071-412X/07/$08.00+0 doi:10.1128/CVI.00482-06
Copyright © 2007, American Society for Microbiology. All Rights Reserved.

Wellcome Trust-Mahosot Hospital-Oxford Tropical Medicine Research Collaboration, Microbiology Laboratory, Mahosot Hospital, Vientiane, Lao PDR,1 Centre for Tropical Medicine, Nuffield Department of Clinical Medicine, Churchill Hospital, University of Oxford, Oxford, United Kingdom,2 Faculty of Tropical Medicine, Mahidol University, 420/6 Rajvithi Road, Bangkok 10400, Thailand,3 World Health Organization Regional Office for the Western Pacific, Manila, Philippines,4 Armed Forces Research Institute of Medical Sciences, Bangkok 10400, Thailand,5 Military Infectious Diseases Research Program, U.S. Army Medical Research and Materiel Command, Fort Detrick, Maryland6
Received 23 December 2006/ Returned for modification 14 May 2007/ Accepted 9 August 2007
|
|
|---|
|
|
|---|
We recently compared the performance of eight RDTs for dengue by using a panel of reference sera from patients with and without dengue who had been characterized previously by gold standard methods (1). Performance characteristics of the dengue RDTs were poor, with only one RDT considered potentially clinically informative. Here we present the results of a complementary prospective study undertaken in the Lao People's Democratic Republic (Laos) to determine the diagnostic performance characteristics of the same eight RDTs and to determine the tests suitability for acute dengue virus infection diagnosis in a clinical, limited-resource setting in an area of dengue endemicity.
|
|
|---|
RDT selection. Eight RDTs were selected, one each from the manufacturers Core, Diazyme, Globalemed, Minerva, Panbio, Standard Diagnostics, Teco, and Tulip, after a survey of commercially available RDTs (Table 1). The eight rapid tests selected were required to have certain characteristics (1), as follows. The tests were required to (i) be commercially available or in the prerelease phase at the time of assessment; (ii) cost US$5 or less per test based on a Ministry of Health purchase price of 2,000 tests; (iii) provide results within 5 h; (iv) not require the purchase of specific/expensive equipment (e.g., enzyme-linked immunosorbent assay [ELISA] plate readers); and (v) be manufactured on site. Externally sourced assays that were repackaged were excluded.
|
View this table: [in a new window] |
TABLE 1. Characteristics of selected dengue rapid diagnostic tests
|
RDT methodologies. All assays were performed with admission and convalescent-phase specimens according to the manufacturers instructions contained in the RDT kits. As the tests were performed in a routine hospital laboratory setting with staff rotation, the RDTs were performed and read individually by trained operators who were blinded to the ELISA results, without conferring, under the direction of the study supervisor at Mahosot Hospital. If more than one operator was on duty, all operators read the results so that interrater agreement could be calculated. Admission samples were tested on the same day they arrived at the laboratory, while convalescent-phase serum samples were batched and assayed on a median of 3 (range, 0 to 35) days after arrival and storage at –80°C. The RDT results were not given to the ward doctors; however, to allow clinical service, the admission samples were tested on one occasion each week, using a commercial anti-dengue virus IgM capture ELISA (Panbio Pty. Ltd., Australia), and results were released.
Dengue reference assays.
Dengue reference assays were performed by staff at the Armed Forces Research Institute of Medical Sciences (AFRIMS), Bangkok, Thailand, who were blinded to the results of serology performed in Laos. Dengue virus infections were confirmed on an individual patient basis by the AFRIMS IgM and IgG antibody capture ELISAs (8, 14) on paired admission and convalescent-phase specimens. For paired specimens, admission samples with less than 15 U of dengue virus IgM antibodies, rising to 30 U in the convalescent-phase specimens (with dengue IgM antibody levels higher than Japanese encephalitis virus [JEV] IgM antibody levels) was considered evidence of an acute primary dengue virus infection. In the absence of an IgM antibody level of more than 40 U in the admission specimen, a twofold rise in IgG (to a value of
100 U) was indicative of secondary or later dengue virus infection (8, 15). Reverse transcriptase PCR (RT PCR) was used to determine serotype identity (1, 9). All samples were stored at –85°C until testing.
Non-dengue virus serology. JEV infection was confirmed by the presence of specific IgM antibodies, using the AFRIMS JEV IgM capture ELISAs (2). Sera were screened for the presence of antibodies to the Chikungunya virus, using the hemagglutination inhibition method (4) at a 1:10 dilution, and for antibodies to Orientia tsutsugamushi (scrub typhus) and Rickettsia typhi (murine typhus), using indirect microimmunofluorescence (11) assays in which a fourfold (or greater) rise in titer defined acute infection (5).
Analysis. Differences in clinical and hematological results between serologically proven dengue and non-dengue patients were assessed for statistical significance (P < 0.05), using either Student's t test or the Wilcoxon signed-rank test, with Stata/SE 8.0 as follows. A (Stata Corp., College Station, TX) software. Diagnostic accuracy scores were calculated by using RDT results in comparison with those of the final case diagnosis for each patient sample, was constructed 2-by-2 table in which the final case diagnostic result was cross-tabulated with the index RDT and thus to define true-positive, false-positive, false-negative, and true-negative values calculate the standard diagnostic accuracy indices of sensitivity, specificity, negative predictive values, and positive predictive values. To determine the level of interrater agreement, kappa scores were calculated, and the strength of agreement was interpreted using the Landis and Koch criteria (10) in which a score of 0 to 0.20 is slight, 0.21 to 0.40 is fair, 0.41 to 0.60 is moderate, 0.61 to 0.80 is substantial, and 0.81 to 1 is almost perfect, with a significant difference between observers at a P value of <0.05.
Assessment of diagnostic utility. In order to examine the true diagnostic utility of the RDTs in a clinical setting, we posed the following questions and performed the following comparisons.
(i) In a patient presenting with suspected acute dengue virus infection, how accurate is the RDT for the diagnosis of dengue virus infection in absolute terms? To answer this question, the RDT result for the admission sample was compared with the final reference result (based on acute- and convalescent-phase AFRIMS IgM and IgG capture ELISA results) on a per-patient basis.
(ii) In a patient who has been recently acutely ill with symptoms indicative of dengue virus infection and is now recovering (such as a traveler returning from a region where dengue virus is endemic), how accurate is the RDT for the diagnosis of dengue in absolute terms? To answer this question, the RDT result for the convalescent-phase sample was compared with final reference result (based on acute- and convalescent-phase AFRIMS IgM and IgG capture ELISA results) on a per-patient basis.
(iii) In a patient presenting with suspected acute dengue virus infection, how accurate is the RDT for the diagnosis of dengue virus infection relative to that of the best available "acute" test? To answer this question, the admission sample RDT result was compared with the AFRIMS IgM capture ELISA admission sample result using
15 units as the positivity cutoff value.
(iv) In a patient with suspected acute dengue virus infection, can the RDTs differentiate between primary and secondary dengue virus infection status? To answer this question, the infection status as classified by the manufacturer of each RDT for admission specimens was compared to the true dengue virus infection status assigned by final reference serology.
|
|
|---|
Thirty-eight of the 87 (43.7%) patients had confirmed dengue virus infection (4 had acute primary infections, 33 had acute secondary infections, and 1 had an infection of indeterminate status) as defined using AFRIMS diagnostic criteria. RT PCR was positive for 25 of 38 patients (65.8%) with confirmed dengue virus infection, and all four dengue virus serotypes were detected. The patients without serological evidence of dengue virus infection were diagnosed with scrub typhus (12/87; 13.8%), murine typhus (4/87; 4.6%), JEV (1/87; 1.2%), and Streptococcus pyogenes septicemia (1/87; 1.2%). Chikungunya virus antibodies were not detected. No diagnosis was available for 35.6% of patients (31/87). RDT reading was performed by seven individual operators, with three operators responsible for more than 96% of the results. Interpretation of kappa scores for the three primary operators (see Table 3) ranged from moderate to slight (kappa score range, 0.06 to 0.54), with significant differences in the operators results for the Diazyme and Tulip RDTs.
|
View this table: [in a new window] |
TABLE 3. Diagnostic accuracy and kappa scores for eight RDTs for the diagnosis of dengue virus infectiona
|
15 years) compared to that of children (P
0.00005). Median age and hematocrit values were higher, and rash was significantly (P < 0.05) more common in patients with serologically confirmed dengue virus infection than in those without (Table 2). The proportions of patients, as classified by the admitting physician, with suspected DF, DHF, and DSS were 70.1% (61/87), 29.9% (26/87), and 0%, respectively, and of these, 38% (23/61) of DF and 58% (15/26) of DHF patients were serologically confirmed as true dengue cases. |
View this table: [in a new window] |
TABLE 2. Demographic and admission clinical features of patients with and without serological evidence for acute dengue infection, as defined by AFRIMS criteria
|
93.9% for seven RDTs), although the Globalemed RDT demonstrated relatively low specificity (69.4%; 95% CI, 59.7 to 79.1). |
View this table: [in a new window] |
TABLE 4. Diagnostic accuracy of eight RDTs for detection of dengue virus IgM antibodiesa
|
(ii) In a patient who had been acutely ill and then recovered (such as a returning traveler), how accurate is the RDT for the diagnosis of dengue virus infection in absolute terms? RDT sensitivity results for convalescent-phase specimens were low (median of 9 days since fever onset; IQR, 8 to 11.5 days) (Table 4) but were generally higher than those for admission samples, ranging from 3.2% (95% CI, 0 to 7.6%) for the Diazyme RDT to 41.9% (95% CI, 29.9 to 54.0%) for the Globalemed RDT, although the specificity for this RDT was lower than those of the other RDTs (81.8%; 95% CI, 72.4 to 91.3%). The most accurate RDT results were those from Panbio and Core (for both tests 21.7%; 95% CI, the sensitivity was 13.1 to 30.4%).
(iii) In a patient presenting with suspected acute dengue virus infection, how accurate is the RDT result for the diagnosis of dengue infection relative to that of the best available "acute" test? The AFRIMS IgM capture ELISA was positive for 57.9% of admission samples of patients with a final diagnosis of dengue virus infection (75% [3/4] acute primary, 54.6% [18/33] acute secondary, and 100% [1/1] indeterminate). When RDT admission sample sensitivity results were compared with those of the AFRIMS IgM capture ELISA (Table 4), agreement ranged from 4.4% (Diazyme) to 26.1% (Globalemed).
(iv) Can the RDTs differentiate between primary and secondary dengue virus infection status in admission samples? Five manufacturers (Core, Panbio, Standard Diagnostics, Teco, and Tulip) claimed that their RDTs were able to differentiate between acute primary infection (IgM+/IgG–) and acute secondary infection (IgM+/IgG+ or IgM–/IgG+). The Diazyme, Globalemed, and Minerva RDTs were also assessed for their abilities to differentiate among infection status, although the manufacturer did not claim this capacity. Most RDTs demonstrated a poor predictive capacity to differentiate between primary and secondary dengue infections (Table 5).
|
View this table: [in a new window] |
TABLE 5. Proportion of positive results for each of eight RDTs used to evaluate acute primary and secondary dengue virus infection statusa
|
|
|
|---|
An increase in sensitivity between the admission and the convalescent-phase samples for most RDTs highlighted the importance of taking convalescent-phase samples when admission samples give a negative result and dengue virus infection remains clinically suspected. However, when the convalescent-phase samples from confirmed-dengue patients were tested, the RDTs showed poor sensitivity, demonstrating the limitations of such assays for the diagnosis of dengue virus infections in patients presenting relatively late, as may occur with travelers recently returned from regions of dengue virus endemicity. Although five of the eight manufacturers claimed their tests had the ability to differentiate between acute primary and secondary infections, no RDT had the capacity to reliably differentiate primary from secondary or subsequent infection, as determined by reference assays.
Limitations of the study include the relatively short time between collection of admission and convalescent-phase sera, the low proportion of primary infections, and a relatively small sample size. A reduced dengue virus IgM antibody response may occur during secondary or subsequent infections with comparatively high IgG titers, which may account for the reduced sensitivity for IgM antibody detection (3, 12). The finding that the median age of those with dengue virus infections was higher than that of those without is possibly due to recruitment bias; as pediatricians gain more experience in the management of dengue virus infections, they may request tests only for children whose symptoms make a clinical diagnosis highly uncertain, while physicians who treat adults may request tests for more patients with the clinical diagnosis of dengue virus infection.
Further independent assessment of rapid, bedside tests for dengue virus infection and other diseases is required. Selection should be based on the results of published independent assessments of diagnostic accuracy rather than solely on the performance characteristics provided by the manufacturer. For dengue virus and many other infections, the duration of fever before sampling is an important determinant of test sensitivity, as the frequency of antibody-positive results is low during the febrile phase of disease and remains so until at least 3 days postdefervescence (8, 15). Notably, the number of days of illness at the time of blood sampling was not quoted by any of the RDT manufacturers that stated sensitivity and specificity values for RDTs assessed in this study. Manufacturers should be required to state this information alongside their claims of accuracy in the product information. The findings highlight the need for further development of rapid dengue diagnostic assays using alternative biological markers such as NS1 antigen (6) to complement existing antibody-based tests.
This study was funded at the World Health Organization-Regional Office for the Western Pacific, Manila, Philippines, and the Wellcome Trust of Great Britain.
The Wellcome Trust-Mahosot Hospital-Oxford Tropical Medicine Research Collaboration and the Mahidol-Oxford Tropical Medicine Research Unit have received test kits from Panbio Pty. Ltd. for evaluation purposes. There are no personal conflicts of interest.
The opinions or assertions contained herein are the private ones of the authors and are not to be construed as official or as reflecting the view of the U.S. government.
Published ahead of print on 22 August 2007. ![]()
|
|
|---|
| |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Copyright © 2009 by the American Society for Microbiology. For an alternate route to Journals.ASM.org, visit: http://intl-journals.asm.org | More Info»