Reliability of CD4 Quantitation in Human Immunodeficiency Virus-Positive Children: Implications for Definition of Immunologic Response to Highly Active Antiretroviral Therapy

Our objective was to develop data-based algorithms for definitionof immunologic response to AIDS therapies in pediatric patients,taking account of T-cell subset measurement errors. The studydesign involved cross-protocol analysis of 2,148 enrollees insix completed Pediatric AIDS Clinical Trials Group trials. Weused standard quantitation of T-cell subsets; linear modelingwith mean-dependent measurement error variance was used to develop95% tolerance limits for change in CD4%. For individuals witha CD4% of approximately 25%, the measurement error-based 95%tolerance interval ranges from 15% to 35%, whereas for individualswith a CD4% of approximately 5%, the tolerance interval rangesfrom 3% to 7%. When pairs of CD4% measures taken within a timeinterval of less than 30 days are averaged to estimate steady-stateCD4%, tolerance interval width decreases by approximately 30%.A simple graphical tool that provides a data-based criterionfor immunologic response over and above variation ascribableto T-cell measurement error is provided. Variability in CD4%due to measurement error is substantial, increases with levelof CD4%, and complicates assessment of immunologic responseto therapy. Replicates of CD4% measures could be used to improveprecision of interpretation of CD4% measures.

INTRODUCTION

Top

Introduction

Methods of managing human immunodeficiency virus (HIV) diseaseare evolving rapidly. The most mature and well-established markersof disease progression are CD4 T-helper cell count and RNA copynumber (2, 3, 5, 7). While these parameters are straightforwardlyused for cross-sectional classification of patients with respectto clinical state, much uncertainty exists regarding evaluationand interpretation of changes in these markers over time (4).Recovery of CD4 T cells is an important criterion for immunereconstitution in patients who are given antiretroviral therapy.This report focuses on the short-term variability of repeatedmeasures of CD4% in HIV-infected children enrolled in the PediatricAIDS Clinical Trials Group (PACTG) treatment trials. Data fromsix clinical trials conducted in the past decade are combinedfor approximate estimation of measurement error variability.The impact of using replicated CD4% measures (i.e., averagingtwo measures obtained simultaneously or separated by a veryshort time intervals) to reduce effects of measurement erroris illustrated.

MATERIALS AND METHODS

Top

Materials and Methods

Participants. Data were collected from six clinical trials conducted in thePACTG: protocols 152, 190, 300, 338, 377, and 382. Of these,protocols 152, 190, and 300 were the earlier treatment trials,consisting of one- or two-nucleoside analog reverse transcriptaseinhibitor therapies, with protocols 152 and 190 completing by1996 and 300 completing in 2001. Protocols 338, 377, and 382were instituted later (ending by 2002) and evaluated triple-antiretroviraltherapies, including protease inhibitors and/or nonnucleosideanalogues.

Measurement protocols. The six trials enumerated above used the same protocol for T-cellsubset measurement. Immunophenotyping for T-cell subsets wasperformed by two-color standard flow cytometry according tothe PACTG consensus protocol (http://pactg.s-3.com/immeth.html).Laboratories that performed these assays were certified by thePACTG immunology quality assurance program. All trials used"pre-entry" and "entry" timed T-cell subset measurements tocharacterize patient baseline state. These pairs of T-cell subsetmeasurements usually occurred over a period of about one week,during which the patient is reasonably assumed to be clinicallystable and on a stable treatment regimen preparatory to startinga new trial.

Statistical methods. Standard descriptive statistics and scatterplots were derivedusing the R statistical computing environment (www.r-project.org).Variance component analysis was conducted using the linear mixedeffects models software package for R by Pinheiro and Bates(6).

The basic model for response variable y (e.g., CD4%) measuredon subjects from study s is

where i indexespatients in study s, j = 1, 2 indexes the pre-entry and entrymeasures, µ_s is the overall mean value of response fora subject aged c years, ß_s is the study-specific slopeof mean response on age at time of measurement, a_i is a subject-specificrandom effect with distribution ), and e_ijis a residual error term.

The residuals e_ij are of central interest in this study. Letm_i denote the true mean CD4% for subject i. If m_i is very small(say 5% or less), then the measurement error variability formeasures on subject i tends also to be small, in part becausenegative values of CD4% are not possible. Among subjects forwhom m_i is larger, measurement error variability tends to belarger as well. We therefore adopt a heteroskedastic measurementerror model. Conditionally on the value of a_i, e_ij has distribution). Note that as m_i tends to zero, so doesthe measurement error variance. As m_i increases, measurementerror variability increases proportionally to . The parameter ${delta}$ can be used to control the growth of measurementerror variability with magnitude of mean CD4%.

Given the generally short time elapsed between repeated measuresand the likely clinical and therapeutic stability of the patientin the pre-entry to entry interval, the e_ij are reasonably regardedas data on irreducible measurement error in the subset measurementprocess. For a specified value of m, "intraclass correlationcoefficient"

varies between 0 and 1 andis a dimensionless measure of repeatability, high values reflectinggood repeatability.

RESULTS

Top

Results

Demographics and baseline characteristics. Table 1 (columns 2 to 6) presents basic descriptive statisticsregarding the patients in the various protocols. This tableincludes information on all patients who supplied valid dataon pre-entry and entry CD4 counts, regardless of treatment assignmentor reappearance of patients in multiple protocols. Individualswhose pre-entry and entry measures were separated in time bymore than 30 days were excluded.

View this table:
[in this window]
[in a new window]
TABLE 1. Descriptive statistics and selected results of variance component modeling stratified by protocol and overall^a

Median baseline patient age varied from 2.2 to 7.1 years. Protocols152, 300, and 382 provided data on patients as young as sixmonths at baseline; minimum ages on other protocols ranged from9 months on 377 to 15 months on 338. The overall median agewas 3.35 years.

In general, HIV RNA loads were uncontrolled, with medians wellabove 10,000 copies in all protocols for which viral loads weremeasured.

Immunosuppression as measured by median CD4% varied among thecohorts, with a median of 23% for protocol 300 and a medianof 29% for protocol 382. Median CD8 counts were above 1,000for all protocols (data not shown), and CD4/CD8 ratios rangedfrom 0.53 (protocol 190) to 0.79 (protocol 382). Forty-ninepercent of patients were Centers for Disease Control and Prevention(CDC) category I (CD4% of >25), 32% were category II (15< CD4% < 24), and 19% were category III (CD4% of <15%).A total of 116 patients (5.4%) had a CD4% of <5%.

After excluding individuals whose time elapsed from pre-entryto entry T-cell counts exceeded 30 days, median lag betweenpre-entry and entry measurements was generally considerablyless than two weeks.

Graphical depiction of CD4% repeatability. Figure 1 depicts on a study-specific basis the dispersion inentry-pre-entry CD4% measures. If CD4% measurements were perfect,we would expect these scatterplots of individual-level changesto be tightly concentrated around the line y = 0, as in generalthere is no biological basis for variation in CD4% in this pretreatmentinterval. For very short lags (up to three days), variabilityis very modest, but it appears that short-term variability issubstantial at five days and that longer lags are not associatedwith greater variability. The appearance of these figures, inconjunction with the numerical analyses to be discussed, ledus to conclude that amalgamation of the data across these protocolswas a reasonable step in learning about CD4% repeatability inpediatric clinical trials.

View larger version (40K):
[in this window]
[in a new window]
FIG. 1. Scatterplots of CD4% at entry minus CD4% at pre-entry visit, versus time elapsed between the two visits. Data show considerable variability between CD4% measurements. The variability was similar across studies.

Modeling of CD4% repeatability. Table 1 (columns 7 to 11) presents results of fitting the variancecomponent model of the "Participants" section in two basic forms:to data stratified by study and to the amalgamated completedata set. The data show that study-specific mean CD4% variedfrom 24.5% to 30.4%. Despite the fact that intra- and interpatientmeasures varied considerably among studies, the parameter ${delta}$ ,which describes the association between CD4% measurement errorand CD4% mean, varied remarkably less, between 0.59 and 0.77.Furthermore the repeatability of CD4% measurements at meansof 15% and 25% was remarkably stable across studies, with valuesof 0.91 to 0.95 and 0.84 to 0.90, respectively.

Tolerance limits for CD4% change. Figure 2 depicts 95% tolerance limits for change in true meanvalue when comparing two CD4% measures. To use the figure, findthe baseline or other reference value on the x axis and drawa vertical line from that point. Note where the vertical lineintersects the solid lines and project to the y axis. The intervalthus defined on the y axis is the tolerance limit. This identifiesthe central 95% of the distribution of CD4% measures consistentwith fluctuation due to measurement error alone, with no changein underlying mean between baseline and follow-up. Follow-upmeasures that lie outside the interval are highly unlikely undera hypothesis of no change in underlying mean CD4%. Note thatthe choice of 95% limits is conventional. Other choices of thresholdcan be used.

View larger version (19K):
[in this window]
[in a new window]
FIG. 2. Ninety-five percent tolerance limits for change in CD4%. The current value is located on the x axis, the boundaries of tolerance limit for single CD4% measurements are projected to the y axis using solid curves, and the boundaries for replicated CD4% measurements are projected to the y axis using dashed curves. In the construction of the dashed curves, it is assumed that both initial and follow-up values are obtained by averaging a pair of measures at each time point.

Figure 2 also provides information on the effect of reducingmeasurement error variability by averaging pairs of CD4% measures.The error decreases by approximately 30% when the CD4% resultis derived from the average of two measurements performed withina time interval of $≤$ 30 days.

DISCUSSION

Top

Discussion

Loss of CD4 T cells is a hallmark of HIV infection. This trendis reversed in patients given antiretroviral therapy that successfullyachieve virologic suppression, but it is now well recognizedthat gains in CD4 counts may also occur in the absence of durablevirologic suppression. Clear criteria for designating an individualas an immunologic responder are lacking, and many studies haverelied upon arbitrary measures of CD4 increase from baselinevalues. The goal of this study was to establish how to interpreta change in CD4 from an initial value. By estimating measurementerror variability using repeated CD4% measures taken over ashort time interval, we provide a basis for identifying changesthat are likely to represent true immunologic response or deterioration.

Based on the variance analysis, we constructed a model depicting95% tolerance limits for change in true mean value of CD4%.As shown in Fig. 2, the model indicates that the variance isgreater at higher levels of CD4% than at lower CD4%. Thus usinga standard definition of gain in CD4 either in absolute numberor in percentages could be misleading, failing to identify clinicallymeaningful CD4 changes at lower initial CD4 percentages or overinterpretingchanges at higher initial CD4 percentages. The data supportthe idea of computing averages of two measurements performedwithin a time interval of <30 days.

This study is the first model-based analysis of pediatric patientsfor assessing CD4 measurement variability. While we assembledrecords on over 2,000 participants in PACTG trials conductedover the past ten years, we acknowledge that our cohort doesnot represent a systematic or random sample of HIV-positivechildren. The protocols selected for this study included infantsand young children <8 years of age with various degrees ofimmunosuppression, and our basic model included an adjustmentfor an age effect. The size and diversity of the cohort on whichour model is constructed contribute to, but do not guarantee,the face validity of our inferences on measurement variability.It was noted by a referee that some of the observations shownin Fig. 1 appear to be outliers. A formal test for outliersin the marginal distribution of all CD4% measures used in themodel did not reject the null hypothesis of no outliers. However,a test applied to the marginal distribution of all entry-pre-entrydifferences did identify 15 putative outliers, with magnitudeof the difference exceeding 21 percentage units. When the modelwas refit, excluding these observations, the estimate of ${delta}$ was0.56, that of was 0.35, and the impact on the analog of Fig. 2 was slight, with a slight narrowing ofthe bands. In the absence of frank evidence of data error, weprefer to employ the entire data set to fit the model of interest.Robustness to data anomalies is a concern with any applicationof sophisticated statistical modeling. A related analysis withHIV-1 RNA measures was conducted by Brambilla and colleagues(1), who obtained robust estimates of assay standard deviationby rescaling empirical quantiles.

The construct validity of the tolerance limits presented inFig. 2 can be assessed by evaluating the association betweendeclared "genuine immunologic response" (changes beyond thebounds of the tolerance limit for given initial value) and otherclinical, immunologic, or virologic events. Such analyses willbe important steps toward formation of an evidence-based criterionfor immunologic response to antiretroviral therapy.

ACKNOWLEDGMENTS

This work was supported by NIH/NIAID contract/grant number 2U01 AI41110-06.

FOOTNOTES

* Corresponding author. Mailing address: Channing Laboratory, Brigham and Women's Hospital, 181 Longwood Ave., Boston, MA 02115. Phone: (617) 525-2265. Fax: (617) 731-1541. E-mail: stvjc{at}channing.harvard.edu.

REFERENCES

Top