This Article
Right arrow Full Text (PDF)
Right arrow Alert me when this article is cited
Right arrow Alert me if a correction is posted
Services
Right arrow Similar articles in this journal
Right arrow Similar articles in PubMed
Right arrow Alert me to new issues of the journal
Right arrow Download to citation manager
Right arrowReprints and Permissions
Right arrow Copyright Information
Right arrow Books from ASM Press
Right arrow MicrobeWorld
Citing Articles
Right arrow Citing Articles via Google Scholar
Google Scholar
Right arrow Articles by Collins, M. T.
Right arrow Articles by Thompson, B. R.
Right arrow Search for Related Content
PubMed
Right arrow PubMed Citation
Right arrow Articles by Collins, M. T.
Right arrow Articles by Thompson, B. R.

 Previous Article  |  Next Article 

Clinical and Vaccine Immunology, December 2006, p. 1373-1374, Vol. 13, No. 12
1071-412X/06/$08.00+0     doi:10.1128/CVI.00283-06

LETTER TO THE EDITOR

Proper Estimation of Sensitivity and Specificity


arrow
LETTER
 
I write to express concern about the diagnostic accuracy statements of Eda et. al. concerning a new Johne's disease enzyme-linked immunosorbent assay (ELISA) (2).

The diagnostic sensitivity of this new ELISA for bovine paratuberculosis, reported to be 100%, was based on only 64 (according to the Materials and Methods) or 51 (according to the Results) Mycobacterium paratuberculosis fecal culture-positive cattle. Fecal culture-positive cattle with fewer than five colonies on culture were discarded from the dataset, which might explain the different numbers in the Materials and Methods and Results, but this is unclear in the paper. While the authors speculate that these very low fecal shedders of M. paratuberculosis "might arise from pass-through bacilli," there is no legitimate basis for excluding these cattle from sensitivity analysis. No other reports on ELISA sensitivity for bovine paratuberculosis have done this. There is no way to ascertain if these cattle were truly infected or if such fecal culture results represented "pass-through" (i.e., the cows were not infected and M. paratuberculosis from the environment was simply in transit through the gastrointestinal tract).

Only 38 cows from a single herd in Japan were used to estimate ELISA specificity. Moreover, the authors state in the Materials and Methods that this herd was selected because it had been annually tested and found to be ELISA negative (assay source not specified) for 5 consecutive years. It is no surprise that these cattle were again found negative for serum antibodies to M. paratuberculosis by the new ELISA. The cattle population chosen for specificity estimation was biased, i.e., preselected for absence of serum antibody, too small, and all from a single herd. Studies with statistical validity use more than 400 animals from multiple herds for ELISA specificity estimation (1, 4, 5). Prior publications document herd-specific variation in M. paratuberculosis ELISA specificity (1). For these reasons, the claimed specificity of 97.4% is tentative at best and has a large 95% confidence interval (94.78 to 99.9%), a statistic that should have been reported.

Meaningful evaluation of immunoassays requires testing large numbers of subjects representative of target populations for diagnosis. For infectious diseases such as paratuberculosis, the infected and noninfected populations must have well-defined, objective, unbiased case definitions. Furthermore, receiver-operating characteristic curves and likelihood ratios should be used to complement sensitivity and specificity estimates (1, 3).

In summary, given the small and biased cattle populations tested and methods of data interpretation, the claims of a highly sensitive and subspecies-specific ELISA for bovine paratuberculosis by Eda et al. must be considered tentative at best.


arrow
REFERENCES
 
    1
  1. Collins, M. T., S. J. Wells, K. R. Petrini, J. E. Collins, R. D. Schultz, and R. H. Whitlock. 2005. Evaluation of five antibody detection tests for bovine paratuberculosis. Clin. Diagn. Lab. Immunol. 12:685-692.
    2. 2
  2. Eda, S., J. P. Bannantine, W. R. Waters, Y. Mori, R. H. Whitlock, M. C. Scott, and C. A. Speer. 2005. New method of serological testing for Mycobacterium avium subsp. paratuberculosis (Johne's disease) by flow cytometry. Foodborne Pathogens Dis. 2:250-262.
    3. 3
  3. Gardner, I. A., and M. Greiner. 2006. Receiver-operating characteristics and likelihood ratios: improvements of traditional methods for the evaluation and application of veterinary clinical pathology tests. Vet. Clin. Pathol. 35:8-17.[Medline]
    4. 4
  4. Griffin, J. F., E. Spittle, C. R. Rodgers, S. Liggett, M. Cooper, D. Bakker, and J. P. Bannantine. 2005. Immunoglobulin G1 enzyme-linked immunosorbent assay for diagnosis of Johne's disease in red deer (Cervus elaphus). Clin. Diagn. Lab. Immunol. 12:1401-1409.
    5. 5
  5. Vannuffel, P., P. Gilo, B. Limbourg, B. Naerhuyzen, C. Dieterich, M. Coene, L. Machtelinckx, and C. Cocito. 1994. Development of species-specific enzyme-linked immunosorbent assay for diagnosis of Johne's disease in cattle. J. Clin. Microbiol. 32:1211-1216.[Abstract/Free Full Text]
Michael T. Collins
University of Wisconsin-Madison
Pathobiological Sciences
2015 Linden Ave.
Madison, Wisconsin 53706
Phone: (608) 262-8457
Fax: (608) 265-6463
E-mail: mcollin5@wisc.edu


Authors' Reply


arrow
LETTER 
 
The paper is the third in a series that should be considered proofs of concept in demonstrating that Mycobacterium avium subsp. paratuberculosis has subspecies-specific antigens on its surface which can be used to diagnose prepatent, as well patent, infections. This discovery is extremely important in light of the fact that a recent study by Sweeney et al. (4) shows that commercial enzyme-linked immunosorbent assays (ELISAs) for diagnosing Johne's disease (JD) in cattle have sensitivity rates of approximately 13.5%. Our paper was not designed or intended to be a validation of the ethanol vortex ELISA for the diagnosis of JD.

Initially, we used a flow cytometric method (FCM) to show that bacilli of M. avium subsp. paratuberculosis have subspecies-specific surface antigens and the FCM was capable of diagnosing infections 6 to 44 months earlier than the fecal culture test and 17 to 67 months earlier than a commercial ELISA (1). Information gleaned from the FCM was then used to develop ELISAs that are user friendly and less costly than flow cytometry (2, 3). Recently, we sent three subsets of serum samples from 22 JD-positive and 28 JD-negative cattle to a state JD testing laboratory and one subset to a JD testing center, all of which were tested for JD by ELISA. Six subsets were tested blindly in our laboratory. All tests detected no false positives. Of the 22 JD-positive samples, the state laboratory detected 5 JD-positive samples in one subset and 1 JD-positive sample in each of the other two subsets but none of them was a repeat positive. The JD testing center detected only one positive and two suspects; all others were negative. In contrast, our test detected all 22 positives in each of the six subsets.

We are currently conducting validation studies, which will be reported in future publications.


arrow
REFERENCES 
 
    1
  1. Eda, S., J. P. Bannantine, W. R. Waters, Y. Mori, R. H. Whitlock, M. C. Scott, and C. A. Speer. 2005. New method of serological testing for Mycobacterium avium subsp. paratuberculosis (Johne's disease) by flow cytometry. Foodborne Pathogens Dis. 2:250-262.
    2. 2
  2. Eda, S., J. P. Bannantine, W. R. Waters, Y. Mori, R. H. Whitlock, M. C. Scott, and C. A. Speer. 2006. A highly sensitive and subspecies-specific surface antigen enzyme-linked immunosorbent assay for diagnosis of Johne's disease. Clin. Vaccine Immunol. 13:837-844.[Abstract/Free Full Text]
    3. 3
  3. Speer, C. A., M. C. Scott, J. P. Bannantine, W. R. Waters, Y. Mori, R. H. Whitlock, and S. Eda. 2006. A novel enzyme-linked immunosorbent assay for diagnosis of Mycobacterium avium subsp. paratuberculosis infections (Johne's disease) in cattle. Clin. Vaccine Immunol. 13:535-540.[Abstract/Free Full Text]
    4. 4
  4. Sweeney, R. W., R. H. Whitlock, S. McAdams, and T. Fyock. 2006. Longitu-dinal study of ELISA seroreactivity to Mycobacterium avium subsp. paratuberculosis in infected cattle and culture-negative herd mates. J. Vet. Diagn. Investig. 18:2-6.[Abstract/Free Full Text]
C. A. Speer*
B. Ray Thompson
Center for Wildlife Health
373 Plant Technology Building
The University of Tennessee
Knoxville, Tennessee 37901-1071

* Phone: (865) 974-0467
Fax: (865) 946-1643
E-mail: caspeer{at}utk.edu

Clinical and Vaccine Immunology, December 2006, p. 1373-1374, Vol. 13, No. 12
1071-412X/06/$08.00+0     doi:10.1128/CVI.00283-06





This Article
Right arrow Full Text (PDF)
Right arrow Alert me when this article is cited
Right arrow Alert me if a correction is posted
Services
Right arrow Similar articles in this journal
Right arrow Similar articles in PubMed
Right arrow Alert me to new issues of the journal
Right arrow Download to citation manager
Right arrowReprints and Permissions
Right arrow Copyright Information
Right arrow Books from ASM Press
Right arrow MicrobeWorld
Citing Articles
Right arrow Citing Articles via Google Scholar
Google Scholar
Right arrow Articles by Collins, M. T.
Right arrow Articles by Thompson, B. R.
Right arrow Search for Related Content
PubMed
Right arrow PubMed Citation
Right arrow Articles by Collins, M. T.
Right arrow Articles by Thompson, B. R.

Copyright © 2009 by the American Society for Microbiology.   For an alternate route to Journals.ASM.org, visit: http://intl-journals.asm.org | More Info»