FACSCount CD4 and Pima CD4 T-Cell Count Systems
A total of 440 HIV-infected patients were recruited at the HIV outpatient clinic of the ITM Antwerp (Belgium) and at the Infectious Diseases Clinic HIV care and treatment center in Dar es Salam (Tanzania). The characteristics of the study population recruited from the 2 sites are summarized in Table 1.
The intra-laboratory variability of FACSCount CD4 and Pima CD4 expressed as CV is summarized in Table 2.
The FACSCount CD4 showed an instrumental (run-to-run) variability with mean CVs ranging from 3% to 6%. The intra-assay variation with mean CVs ranging from 3.2% to 6.8% is similar to the CVs of the instrumental precision.
The inter-assay (day-to-day) variation showed CVs ranging from 4% to 9% for absolute CD4 counts and from 2% to 8% for CD4%. The FACSCount CD4 provided both CD4 counts and CD4% without any significant carry over (k < 0) in both Antwerp and Dar es Salam sites.
The instrumental precision of Pima CD4 was determined using values from the control beads and resulted in a CV of less than 4%.
The Pima CD4 showed an average mean intra-assay CV >10% (CV = 13.4%). However, all blood samples with CD4 T cells ≤200 per microliter showed acceptable variability (SD) of <35 per microliter as recommended by the manufacturer. For samples with CD4 counts between 200 and 300 per microliter, the mean CV was higher than 10% but still less than 15% in both study sites except for 1 sample in Dar es Salam. In Antwerp, based on 1 sample, the Pima devices showed true intra-assay CVs ranging from 6% to 12% with a mean of 9.4%. The 5 Pima CD4 devices showed an inter-instrument CV of 4.6% calculated as the statistical mean of the intra-instrument CVs.
The Pima CD4 showed inter-assay CVs ranging from 7% to 15%.
When the Multicheck controls were run on the FACSCalibur, all values provided in the 2 study sites were within the expected ranges, with a CV <6% for normal CD4 counts and <8% for low CD4 counts.
Figures 1A–D and E–H, respectively, show the comparisons of absolute CD4 counts and CD4% obtained from FACSCount CD4 and FACSCalibur.
(Enlarge Image)
Figure 1.
Comparison between FACSCount CD4 and FACSCalibur Trucount: absolute CD4 counts obtained by FACSCount CD4 and FACSCalibur Trucount were compared by Passing–Bablok regression in Antwerp (A) and Dar es Salam (B). The corresponding graphs depicting the relative bias between the 2 instruments are represented in Pollock plots for Antwerp (C) and Dar es Salam (D). CD4% obtained by FACSCount CD4 and FACSCalibur Trucount are compared by Passing–Bablok regression in Antwerp (E) and Dar es Salam (F). The corresponding graphs depicting the absolute bias between the 2 instruments are depicted in Bland–Altman plots for Antwerp (G) and Dar es Salam (H). In Passing–Bablok regression graphs, the solid blue line represents the regression line and the dashed lines represent the 95% confidence interval for the regression line. In the Pollock and Bland–Altman graphs, the solid blue line represents the mean bias. The dashed lines represent mean bias 6 1.96 SD, which are the upper and lower LOA.
These 2 systems showed an excellent correlation with a slope of 1.02 in Antwerp and 1.06 in Dar es Salam. The agreement between the 2 methods was assessed by Bland–Altman and similarity analyses. The mean relative bias (LOA) and the mean similarity (CV) were 0.4% (-22.4 to 23.2) and 101% (5%) in Antwerp and 3.1% (-30.1 to 36.3) and 102% (9%) in Dar es Salam, respectively. Agreement within the different CD4 categories is summarized in Table 3.
At the threshold of 200 cells per microliter, the FACSCount CD4 misclassified 2% (10/435) of patients in both sites. Of these 10 misclassifications, the FACSCount CD4 would delay the ART initiation of only 2 patients. The sensitivity was 100% (34/34) in Antwerp and 95% (38/40) in Dar es Salam, and the specificity was 99% (201/204) and 97% (152/157) in Antwerp and Dar es Salam, respectively. At the threshold of 350 cells per microliter, the FACSCount CD4 misclassified 7% (30/435) of patients. Thus, sensitivity and specificity were 97% (91/94) and 96% (138/144) in Antwerp and 89% (86/97) and 98% (98/100) in Dar es Salam, respectively. At the threshold of 500 cells per microliter, 5% (23/435) of patients were misclassified by the FACSCount CD4, which showed sensitivity and specificity of 95% (150/158) and 96% (74/80) in Antwerp and 96% (136/142) and 95% (52/55) in Dar es Salam, respectively. The performances (agreement) of the FACSCount CD4 on absolute CD4 counts were different between Antwerp and Dar es Salam.
The Passing–Bablok analysis on CD4% showed excellent correlation between FACSCount CD4 and FACSCalibur with a slope of 0.99 in Antwerp and 0.96 in Dar es Salam. These 2 methods showed excellent agreement with mean bias (LOA) and mean similarity (CV) of −0.6% (-3.6 to 2.4) and 99% (3%) in Antwerp and −1.1% (-5.3 to 3.1) and 98% (7%) in Dar es Salam, respectively. Agreement within different CD4 categories is summarized in Table 3. At the threshold of 25%, the FACSCount CD4 showed sensitivity of 97% (137/141) in Antwerp and 96% (123/128) in Dar es Salam and specificity of 84% (78/93) and 97% (62/64), respectively, in Antwerp and Dar es Salam. Except for the bias with a P value of 0.0053, the FACSCount CD4 showed a similar performance between Antwerp and Dar es Salam.
The supplemental Figure (see Supplemental Digital Content 1, http://links.lww.com/QAI/A533) shows the comparison of the absolute CD4 counts obtained from FACSCount CD4 and FACSCount. The 2 absolute CD4 counts obtained on the 2 different FACSCount versions correlated well and showed excellent agreement in both Antwerp and Dar es Salam sites. The slopes were 1.02 and 0.96 in Antwerp and Dar es Salam, respectively. In addition, the mean relative bias (LOA) and similarity (CV) were 1.1% (−14.4 to 16.7) and 101% (4%) in Antwerp and 3.9% (−17.1 to 24.8) and 102% (5%) in Dar es Salam, respectively.
Figures 2A–D and E–H, respectively, illustrate the comparisons between Pima CD4 using venous blood or capillary blood and FACSCalibur.
(Enlarge Image)
Figure 2.
Comparison between Pima CD4 and FACSCalibur Trucount: absolute CD4 counts obtained by Pima CD4 using venous blood and FACSCalibur Trucount were compared by Passing–Bablok regression in Antwerp (A) and Dar es Salam (B). The corresponding graphs depicting the relative bias between the 2 instruments are represented in Pollock plots for Antwerp (C) and Dar es Salam (D). Absolute CD4 counts obtained by Pima CD4 using capillary blood and FACSCalibur Trucount were compared by Passing–Bablok regression in Antwerp (E) and Dar es Salam (F). The corresponding graphs depicting the relative bias between the 2 instruments are represented in Pollock plots for Antwerp (G) and Dar es Salam (H). In Passing–Bablok regression graphs, the solid blue line represents the regression line and the dashed lines represent the 95% confidence interval for the regression line. In the Pollock graphs, the solid blue line represents the mean bias. The dashed lines represent mean bias 6 1.96 SD, which are the upper and lower LOA.
These methods showed good correlation with a slope of 0.93 in Antwerp and 0.85 in Dar es Salam. Pima CD4 using venous blood showed a mean bias (LOA) of −4.1% (-29 to 20.8) and similarity (CV) of 98% (7%) in Antwerp and a mean bias of −9.4% (-54.4 to 35.6) and similarity (CV) of 98% (21%) in Dar es Salam. Agreement within the different CD4 categories is summarized in Table 3. The bias and correlation coefficients of Pima CD4 using venous blood were different between Antwerp and Dar es Salam.
At the threshold of 200 cells per microliter, 3% (14/440) of patients were misclassified by the Pima CD4 using venous blood in both sites. Only 2 patients would have their ART initiation delayed when relying on the Pima CD4 compared with FACSCalibur. The sensitivity and specificity were 97% (34/35) and 98% (201/205) in Antwerp and 98% (40/41) and 95% (151/159) in Dar es Salam, respectively. At the threshold of 350 cells per microliter, 9% (40/440) of HIV patients were misclassified relying on the Pima CD4 using venous blood, which would delay the ART initiation in 7 patients. The sensitivity and specificity were 96% (92/96) and 91% (131/144) in Antwerp and 97% (96/99) and 80% (81/101) in Dar es Salam, respectively. At the threshold of 500 cells per microliter, sensitivity and specificity were 99% (158/160) and 85% (68/80) in Antwerp and 99% (143/145) and 78% (43/55) in Dar es Salam, respectively.
The Passing–Bablok regression plots showed slopes of 0.89 and 0.94 in Antwerp and Dar es Salam, respectively. The Pima CD4 using capillary blood showed a mean bias (LOA) and a mean similarity (CV) of −9.5% (−46.8 to 27.9) and 96% (12%) in Antwerp and −0.9% (−57.3 to 55.6) and 102% (21%) in Dar es Salam, respectively. The Pima CD4 using capillary blood showed different performances between Antwerp and Dar es Salam.
At the threshold of 200 cells per microliter, 4% (16/410) of patients were misclassified by the Pima CD4 using capillary blood in both sites. None of the patients would have their ART initiation delayed relying on Pima CD4 using capillary blood. The sensitivity and specificity were 100% (26/26) and 96% (177/184) in Antwerp and 100% (41/41) and 94% (150/159) in Dar es Salam, respectively. At the threshold of 350 cells per microliter, 13% (55/410) of patients were misclassified by the Pima CD4 using capillary blood. Sixteen out of 55 patients would have the initiation of ART delayed. The sensitivity and specificity were 96% (75/78) and 78% (103/132) in Antwerp and 87% (86/99) and 90% (91/101) in Dar es Salam, respectively. At the threshold of 500 cells per microliter, the sensitivity was 98% (134/137) and 97% (140/145) and the specificity was 73% (53/73) and 82% (45/55) in Antwerp and Dar es Salam, respectively.
The Figure, SDC 2, http://links.lww.com/QAI/A533, compares CD4 counts in capillary and venous blood read on Pima CD4. Absolute CD4 counts measured in capillary blood correlated well with those in venous blood in both Antwerp and Dar es Salam. The Passing–Bablok regression showed a slope of 0.97 in Antwerp and 1.12 in Dar es Salam. The mean bias (LOA) and similarity (CV) were −5.5% (−48.0 to 37.0) and 98.6% (14%) in Antwerp and 8.4% (−38.2 to 55.1) and 106.3% (15%) in Dar es Salam, respectively.
A total of 33 errors were reported during the study on FACSCount CD4. Of these, 15 errors were from agreement assessment and 18 from precision assessment. Of the 33 errors reported, 3 samples failed to provide values even after repeating the tests; the 30 remaining provided CD4 counts but failed to provide CD4%. On Pima CD4 analyzers, 54 errors were reported during the study. Of these, 31 were from agreement assessment, 10 from precision assessment, and 13 from Pima standard beads. In Antwerp, capillary blood showed more errors (30/32) than venous blood, and 63.2% were attributed to 1 of the 3 nurses performing the finger-stick collections. In Dar es Salam, the venous blood showed a higher rate of errors of 59% (13/22) than the capillary blood, and 61.5% of errors were attributed to 1 of the 6 Pima CD4 operators.
Results
Study Population
A total of 440 HIV-infected patients were recruited at the HIV outpatient clinic of the ITM Antwerp (Belgium) and at the Infectious Diseases Clinic HIV care and treatment center in Dar es Salam (Tanzania). The characteristics of the study population recruited from the 2 sites are summarized in Table 1.
Precision Assessment
The intra-laboratory variability of FACSCount CD4 and Pima CD4 expressed as CV is summarized in Table 2.
FACSCount CD4
The FACSCount CD4 showed an instrumental (run-to-run) variability with mean CVs ranging from 3% to 6%. The intra-assay variation with mean CVs ranging from 3.2% to 6.8% is similar to the CVs of the instrumental precision.
The inter-assay (day-to-day) variation showed CVs ranging from 4% to 9% for absolute CD4 counts and from 2% to 8% for CD4%. The FACSCount CD4 provided both CD4 counts and CD4% without any significant carry over (k < 0) in both Antwerp and Dar es Salam sites.
Pima CD4
The instrumental precision of Pima CD4 was determined using values from the control beads and resulted in a CV of less than 4%.
The Pima CD4 showed an average mean intra-assay CV >10% (CV = 13.4%). However, all blood samples with CD4 T cells ≤200 per microliter showed acceptable variability (SD) of <35 per microliter as recommended by the manufacturer. For samples with CD4 counts between 200 and 300 per microliter, the mean CV was higher than 10% but still less than 15% in both study sites except for 1 sample in Dar es Salam. In Antwerp, based on 1 sample, the Pima devices showed true intra-assay CVs ranging from 6% to 12% with a mean of 9.4%. The 5 Pima CD4 devices showed an inter-instrument CV of 4.6% calculated as the statistical mean of the intra-instrument CVs.
The Pima CD4 showed inter-assay CVs ranging from 7% to 15%.
Agreement Between Methods
When the Multicheck controls were run on the FACSCalibur, all values provided in the 2 study sites were within the expected ranges, with a CV <6% for normal CD4 counts and <8% for low CD4 counts.
Comparison Between FACSCount CD4 and FACSCalibur (Trucount)
Figures 1A–D and E–H, respectively, show the comparisons of absolute CD4 counts and CD4% obtained from FACSCount CD4 and FACSCalibur.
(Enlarge Image)
Figure 1.
Comparison between FACSCount CD4 and FACSCalibur Trucount: absolute CD4 counts obtained by FACSCount CD4 and FACSCalibur Trucount were compared by Passing–Bablok regression in Antwerp (A) and Dar es Salam (B). The corresponding graphs depicting the relative bias between the 2 instruments are represented in Pollock plots for Antwerp (C) and Dar es Salam (D). CD4% obtained by FACSCount CD4 and FACSCalibur Trucount are compared by Passing–Bablok regression in Antwerp (E) and Dar es Salam (F). The corresponding graphs depicting the absolute bias between the 2 instruments are depicted in Bland–Altman plots for Antwerp (G) and Dar es Salam (H). In Passing–Bablok regression graphs, the solid blue line represents the regression line and the dashed lines represent the 95% confidence interval for the regression line. In the Pollock and Bland–Altman graphs, the solid blue line represents the mean bias. The dashed lines represent mean bias 6 1.96 SD, which are the upper and lower LOA.
On Absolute CD4 T-Cell Counts
These 2 systems showed an excellent correlation with a slope of 1.02 in Antwerp and 1.06 in Dar es Salam. The agreement between the 2 methods was assessed by Bland–Altman and similarity analyses. The mean relative bias (LOA) and the mean similarity (CV) were 0.4% (-22.4 to 23.2) and 101% (5%) in Antwerp and 3.1% (-30.1 to 36.3) and 102% (9%) in Dar es Salam, respectively. Agreement within the different CD4 categories is summarized in Table 3.
At the threshold of 200 cells per microliter, the FACSCount CD4 misclassified 2% (10/435) of patients in both sites. Of these 10 misclassifications, the FACSCount CD4 would delay the ART initiation of only 2 patients. The sensitivity was 100% (34/34) in Antwerp and 95% (38/40) in Dar es Salam, and the specificity was 99% (201/204) and 97% (152/157) in Antwerp and Dar es Salam, respectively. At the threshold of 350 cells per microliter, the FACSCount CD4 misclassified 7% (30/435) of patients. Thus, sensitivity and specificity were 97% (91/94) and 96% (138/144) in Antwerp and 89% (86/97) and 98% (98/100) in Dar es Salam, respectively. At the threshold of 500 cells per microliter, 5% (23/435) of patients were misclassified by the FACSCount CD4, which showed sensitivity and specificity of 95% (150/158) and 96% (74/80) in Antwerp and 96% (136/142) and 95% (52/55) in Dar es Salam, respectively. The performances (agreement) of the FACSCount CD4 on absolute CD4 counts were different between Antwerp and Dar es Salam.
On Percentage of CD4 T-Cells (CD4%)
The Passing–Bablok analysis on CD4% showed excellent correlation between FACSCount CD4 and FACSCalibur with a slope of 0.99 in Antwerp and 0.96 in Dar es Salam. These 2 methods showed excellent agreement with mean bias (LOA) and mean similarity (CV) of −0.6% (-3.6 to 2.4) and 99% (3%) in Antwerp and −1.1% (-5.3 to 3.1) and 98% (7%) in Dar es Salam, respectively. Agreement within different CD4 categories is summarized in Table 3. At the threshold of 25%, the FACSCount CD4 showed sensitivity of 97% (137/141) in Antwerp and 96% (123/128) in Dar es Salam and specificity of 84% (78/93) and 97% (62/64), respectively, in Antwerp and Dar es Salam. Except for the bias with a P value of 0.0053, the FACSCount CD4 showed a similar performance between Antwerp and Dar es Salam.
Comparison Between FACSCount CD4 and Standard FACSCount on Absolute Counts
The supplemental Figure (see Supplemental Digital Content 1, http://links.lww.com/QAI/A533) shows the comparison of the absolute CD4 counts obtained from FACSCount CD4 and FACSCount. The 2 absolute CD4 counts obtained on the 2 different FACSCount versions correlated well and showed excellent agreement in both Antwerp and Dar es Salam sites. The slopes were 1.02 and 0.96 in Antwerp and Dar es Salam, respectively. In addition, the mean relative bias (LOA) and similarity (CV) were 1.1% (−14.4 to 16.7) and 101% (4%) in Antwerp and 3.9% (−17.1 to 24.8) and 102% (5%) in Dar es Salam, respectively.
Comparison Between Pima CD4 and FACSCalibur (Trucount)
Figures 2A–D and E–H, respectively, illustrate the comparisons between Pima CD4 using venous blood or capillary blood and FACSCalibur.
(Enlarge Image)
Figure 2.
Comparison between Pima CD4 and FACSCalibur Trucount: absolute CD4 counts obtained by Pima CD4 using venous blood and FACSCalibur Trucount were compared by Passing–Bablok regression in Antwerp (A) and Dar es Salam (B). The corresponding graphs depicting the relative bias between the 2 instruments are represented in Pollock plots for Antwerp (C) and Dar es Salam (D). Absolute CD4 counts obtained by Pima CD4 using capillary blood and FACSCalibur Trucount were compared by Passing–Bablok regression in Antwerp (E) and Dar es Salam (F). The corresponding graphs depicting the relative bias between the 2 instruments are represented in Pollock plots for Antwerp (G) and Dar es Salam (H). In Passing–Bablok regression graphs, the solid blue line represents the regression line and the dashed lines represent the 95% confidence interval for the regression line. In the Pollock graphs, the solid blue line represents the mean bias. The dashed lines represent mean bias 6 1.96 SD, which are the upper and lower LOA.
On Venous Blood Samples
These methods showed good correlation with a slope of 0.93 in Antwerp and 0.85 in Dar es Salam. Pima CD4 using venous blood showed a mean bias (LOA) of −4.1% (-29 to 20.8) and similarity (CV) of 98% (7%) in Antwerp and a mean bias of −9.4% (-54.4 to 35.6) and similarity (CV) of 98% (21%) in Dar es Salam. Agreement within the different CD4 categories is summarized in Table 3. The bias and correlation coefficients of Pima CD4 using venous blood were different between Antwerp and Dar es Salam.
At the threshold of 200 cells per microliter, 3% (14/440) of patients were misclassified by the Pima CD4 using venous blood in both sites. Only 2 patients would have their ART initiation delayed when relying on the Pima CD4 compared with FACSCalibur. The sensitivity and specificity were 97% (34/35) and 98% (201/205) in Antwerp and 98% (40/41) and 95% (151/159) in Dar es Salam, respectively. At the threshold of 350 cells per microliter, 9% (40/440) of HIV patients were misclassified relying on the Pima CD4 using venous blood, which would delay the ART initiation in 7 patients. The sensitivity and specificity were 96% (92/96) and 91% (131/144) in Antwerp and 97% (96/99) and 80% (81/101) in Dar es Salam, respectively. At the threshold of 500 cells per microliter, sensitivity and specificity were 99% (158/160) and 85% (68/80) in Antwerp and 99% (143/145) and 78% (43/55) in Dar es Salam, respectively.
On Capillary Blood Samples
The Passing–Bablok regression plots showed slopes of 0.89 and 0.94 in Antwerp and Dar es Salam, respectively. The Pima CD4 using capillary blood showed a mean bias (LOA) and a mean similarity (CV) of −9.5% (−46.8 to 27.9) and 96% (12%) in Antwerp and −0.9% (−57.3 to 55.6) and 102% (21%) in Dar es Salam, respectively. The Pima CD4 using capillary blood showed different performances between Antwerp and Dar es Salam.
At the threshold of 200 cells per microliter, 4% (16/410) of patients were misclassified by the Pima CD4 using capillary blood in both sites. None of the patients would have their ART initiation delayed relying on Pima CD4 using capillary blood. The sensitivity and specificity were 100% (26/26) and 96% (177/184) in Antwerp and 100% (41/41) and 94% (150/159) in Dar es Salam, respectively. At the threshold of 350 cells per microliter, 13% (55/410) of patients were misclassified by the Pima CD4 using capillary blood. Sixteen out of 55 patients would have the initiation of ART delayed. The sensitivity and specificity were 96% (75/78) and 78% (103/132) in Antwerp and 87% (86/99) and 90% (91/101) in Dar es Salam, respectively. At the threshold of 500 cells per microliter, the sensitivity was 98% (134/137) and 97% (140/145) and the specificity was 73% (53/73) and 82% (45/55) in Antwerp and Dar es Salam, respectively.
Comparison Between Capillary Blood and Venous Blood on Pima CD4
The Figure, SDC 2, http://links.lww.com/QAI/A533, compares CD4 counts in capillary and venous blood read on Pima CD4. Absolute CD4 counts measured in capillary blood correlated well with those in venous blood in both Antwerp and Dar es Salam. The Passing–Bablok regression showed a slope of 0.97 in Antwerp and 1.12 in Dar es Salam. The mean bias (LOA) and similarity (CV) were −5.5% (−48.0 to 37.0) and 98.6% (14%) in Antwerp and 8.4% (−38.2 to 55.1) and 106.3% (15%) in Dar es Salam, respectively.
Error Rates
A total of 33 errors were reported during the study on FACSCount CD4. Of these, 15 errors were from agreement assessment and 18 from precision assessment. Of the 33 errors reported, 3 samples failed to provide values even after repeating the tests; the 30 remaining provided CD4 counts but failed to provide CD4%. On Pima CD4 analyzers, 54 errors were reported during the study. Of these, 31 were from agreement assessment, 10 from precision assessment, and 13 from Pima standard beads. In Antwerp, capillary blood showed more errors (30/32) than venous blood, and 63.2% were attributed to 1 of the 3 nurses performing the finger-stick collections. In Dar es Salam, the venous blood showed a higher rate of errors of 59% (13/22) than the capillary blood, and 61.5% of errors were attributed to 1 of the 6 Pima CD4 operators.
SHARE
