Table 1 Description of data analysis

Data format

Methods

The number (proportion) of CD4 observations in the following CD4 categories: <100 cells/μl; 100 – 350 cells/μl; 350–500 cells/μl and >500 cells/μl was determined for both Pima CD4 and reference methods. The data were further divided into the type of specimen (venous or capillary) tested on the Pima CD4

Significance (p ≤0.05) between categories was determined using the proportions test.

The Pima CD4 and reference CD4 observations were also converted to binary (0 = above the specified threshold and 1 = below the threshold). The observation pairs were also sorted by specimen type, comparator reference technology and year when observations were collected.

The false positive, false negative, sensitivity (ability to correctly identify patients requiring treatment) and specificity (ability to correctly identify patients not requiring treatment) were calculated for the three clinical thresholds of the entire dataset. The total misclassification rate (percentage) was calculated as the addition of false positive rate and false negative rate. The upward (percentage of patients requiring treatment incorrectly identified by the Pima CD4 as above the threshold) and downward (percentage of patients not requiring treatment incorrectly identified by the Pima CD4 as below the threshold) misclassification rates were calculated. The Q-statistic was calculated [35] to quantify and account for the presence of any study heterogeneity due to differences in sample size, study quality, study designs, and/or data collection methods. A bivariate and/or univariate random effects model was applied using METANDI commands in STATA 13.

Methods applied (where applicable, 95 % confidence intervals (CI) were reported)

Description

Data description.

The CD4 count paired observations were described by mean (using random effects models), median and standard deviation (SD).

The agreement between the Pima CD4 and reference technology was measured using the Bland-Altman (bias [or mean difference] and SD of the bias) [23],

The Bland-Altman measures the difference between observation pairs (a-b), where method ‘a’ is the Pima CD4. The mean paired difference (the bias or accuracy) and SD of this bias (precision) were determined. A zero mean difference implies good accuracy between reference and Pima CD4 and a small SD of the bias implies good precision (low variability). The accuracy and precision are visually represented on a modified Bland-Altman difference plot with the paired difference on the vertical axis and the absolute CD4 count of the reference on the horizontal axis.

The agreement between the Pima CD4 and reference technology was also measured using the percentage similarity (mean, SD and coefficient of variation [CV]) [24],

The percentage similarity is calculated as the average between the reference and Pima CD4 technology represented as a percentage of the reference technology: [([a + b]/2) /b] × 100, where ‘b’ is the reference method. Observation pairs with the same value will be 100 % similar (accurate) and observation pairs where the Pima CD4 is greater than the reference will be > 100 %, and conversely <100 % if Pima CD4 has a value smaller than the reference. The amount of variability (precision) is represented by the percentage similarity SD and overall agreement by the percentage similarity CV.

The agreement between the Pima CD4 and reference technology was also measured using the percent difference (bias, SD) [25]

The percentage difference is calculated as (a-b)/b (or the average between ‘a’ and ‘b’) × 100 % [25]. Observation pairs with the same value will have no difference and therefore low percent difference, as the percentage difference method is more relative than absolute difference over the range of data.

The strength of the agreement (accuracy and precision) was measured by the concordance correlation (Pc) between the Pima CD4 and reference technologies [17, 36]

The formula applied is pc (concordance correlation) = p (Pearson correlation [measure of precision]) x Cb (bias correction factor [measure of accuracy]) [17, 36]. The value of pc (strength of agreement) is suggested as: <0.9 (poor); 0.90-0.95 (moderate); 0.95 – 0.99 (substantial); >0.99 (almost perfect) [17, 36].

Description of subset

Methods applied

Sample size in method comparison: Few CD4 method comparison studies’ sample sizes are based on statistical criteria, but rather constrained by costs. This pooled meta-analysis data set afforded the ability to investigate potential impact of sample size on statistical outcomes. An analysis was therefore performed on a subset of data from the comparison between the Pima CD4 and FACSCount of venous derived specimens, as this was the largest subset of paired observations from a single reference and Pima CD4 comparison.

Once the data pairs were entered in MS Excel, random sample numbers (between 1 and 3,486) and irrespective of CD4 category were generated for each CD4 observation pair. This would ensure selection of sample sizes would be independent of the CD4 count and range of CD4 count. The misclassification and agreement analysis was then performed in STATA for sample sizes ranging from 50 to 4,000. The bias, SD of the bias, percentage similarity mean and SD, total misclassification, sensitivity and concordance correlation were all plotted against sample size to determine the impact of sample size on method comparison parameters.

Performance of the Pima CD4 compared to various reference technologies.

The data were sorted based on the reference CD4 method comparator performed in comparison to the Pima CD4, irrespective of study, region or year when the study was performed. The data selection, however, took into account the outcome of the analysis performed in (c) on sample size. Categorical and numerical statistical analyses were applied and results visualized in scatter plots and bar charts.

Performance of the Pima CD4 by different cadre of staff

A subset of 3,751 paired observations was evaluated for total misclassification rates based on different healthcare worker cadres of Pima CD4 operators. This subset was from 11 studies that provided such information with their data. Three cadres were defined: laboratory technician/technologist (includes scientists); laboratory assistant (a lower level of training than technicians) and clinical staff (includes nurses and lay counselors).