PLoS OnePLoS ONEplosplosonePLoS ONE1932-6203Public Library of ScienceSan Francisco, CA USA25790185436609410.1371/journal.pone.0115019PONE-D-14-33559Research ArticleCD4 Enumeration Technologies: A Systematic Review of Test Performance for Determining Eligibility for Antiretroviral TherapyCD4 Assay Performance: A Systematic ReviewPeelingRosanna W.1*SollisKimberly A.1GloverSarah1CroweSuzanne M.2LandayAlan L.3ChengBen4BarnettDavid5DennyThomas N.6SpiraThomas J.7StevensWendy S.8CrowleySiobhan9EssajeeShaffiq10VitoriaMarco11FordNathan11London School of Hygiene and Tropical Medicine, London, WC1E 7HT, EnglandCentre for Biomedical Research, Burnet Institute, Melbourne, 3004, Victoria, AustraliaDepartment of Immunology/Microbiology, Rush University Medical Center, Chicago, IL, 60612, United States of AmericaPangaea Global AIDS Foundation, Oakland, CA, 94607, United States of AmericaUK NEQAS for Leucocyte Immunophenotyping, Sheffield, S10 2QD, EnglandDuke Human Vaccine Institute and Center for HIV/AIDS, Immunology and Virology Quality Assessment Center, Durham, NC, 27710, United States of AmericaDivision of AIDS, STD, &TB Laboratory Research, National Center for Infectious Diseases, Centers for Disease Control and Prevention, Atlanta, GA, 30333, United States of AmericaUniversity of the Witwatersrand, Parktown, 2193, South AfricaDirector Health Programs, ELMA Philanthropies, New York, NY, United States of AmericaClinton Health Access Initiative, Boston, MA, 02127, United States of AmericaWorld Health Organization, Geneva, SwitzerlandCameronD WilliamAcademic EditorUniversity of Ottawa, CANADA

Competing Interests: S. Crowe is a co-developer of the Burnet POC CD4 test. This does not alter the authors’ adherence to PLOS ONE policies on sharing data and materials. All other authors have no conflicts of interest.

Conceived and designed the experiments: RWP SMC SC ALL BC DB TND TJS WSS. Performed the experiments: RWP KAS SG. Analyzed the data: RWP SMC SC ALL BC DB TND TJS WSS. Wrote the paper: RWP KAS SG SMC SC ALL BC DB TND TJS WSS SE MV NF.

* E-mail: rosanna.peeling@lshtm.ac.uk
19320152015103e011501968201417112014This is an open access article, free of all copyright, and may be freely reproduced, distributed, transmitted, modified, built upon, or otherwise used by anyone for any lawful purpose. The work is made available under the Creative Commons CC0 public domain dedicationBackground

Measurement of CD4+ T-lymphocytes (CD4) is a crucial parameter in the management of HIV patients, particularly in determining eligibility to initiate antiretroviral treatment (ART). A number of technologies exist for CD4 enumeration, with considerable variation in cost, complexity, and operational requirements. We conducted a systematic review of the performance of technologies for CD4 enumeration.

Methods and Findings

Studies were identified by searching electronic databases MEDLINE and EMBASE using a pre-defined search strategy. Data on test accuracy and precision included bias and limits of agreement with a reference standard, and misclassification probabilities around CD4 thresholds of 200 and 350 cells/μl over a clinically relevant range. The secondary outcome measure was test imprecision, expressed as % coefficient of variation. Thirty-two studies evaluating 15 CD4 technologies were included, of which less than half presented data on bias and misclassification compared to the same reference technology. At CD4 counts <350 cells/μl, bias ranged from -35.2 to +13.1 cells/μl while at counts >350 cells/μl, bias ranged from -70.7 to +47 cells/μl, compared to the BD FACSCount as a reference technology. Misclassification around the threshold of 350 cells/μl ranged from 1-29% for upward classification, resulting in under-treatment, and 7-68% for downward classification resulting in overtreatment. Less than half of these studies reported within laboratory precision or reproducibility of the CD4 values obtained.

Conclusions

A wide range of bias and percent misclassification around treatment thresholds were reported on the CD4 enumeration technologies included in this review, with few studies reporting assay precision. The lack of standardised methodology on test evaluation, including the use of different reference standards, is a barrier to assessing relative assay performance and could hinder the introduction of new point-of-care assays in countries where they are most needed.

The funding for this study was provided by the World Health Organization to the London School of Hygiene and Tropical Medicine. The decision to write this article was not influenced by any diagnostic or commercial company. The mention of specific companies or of certain manufacturers’ products does not imply that they are endorsed or recommended by the World Health Organization in preference to others of a similar nature that are not mentioned. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.Data AvailabilityAll relevant data are within the paper and its Supporting Information files.
Introduction

The increased availability of antiretroviral therapy (ART) has resulted in major reductions in morbidity and mortality in high HIV burden settings. Through significant global scale-up, access to ART is increasing, with around 10 million people in low- and middle-income settings receiving treatment as of the end of 2013, an estimated 65% of the of the global target of 15 million people set for 2015 [1].

CD4+ T-lymphocytes, also known as the helper T-cells, are the coordinators of the immune response that protects the body against microbial disease, a variety of autoimmune diseases and some forms of cancer. The destruction of CD4+ T-lymphocytes by HIV is the main cause of the progressive weakening of the immune system in HIV infection, and leads ultimately to acquired immune deficiency syndrome (AIDS). The CD4 count is a strong predictor of HIV progression to AIDS and death, and is considered the best laboratory marker for deciding when to initiate ART [2,3]. The use of clinical staging alone to determine the timing of ART initiation is limited by the unreliable correlation between asymptomatic or mild disease and short-term prognosis and may result in dangerous delays in treatment initiation in those without symptoms but with severe immune suppression [4].

Prior to 2013, the World Health Organization (WHO) recommended ART initiation in all HIV-infected individuals whose CD4 count has dropped to ≤350 cells/μl, irrespective of clinical symptoms. The WHO 2013 guidelines have raised the threshold for ART initiation to ≤500 cells/μl, with priority given to those with a CD4 count to ≤350 cells/μl, consistent with emerging data, indicating a clinical and public health benefit of earlier treatment and as part of a global effort to get 15 million HIV patients on ART by end of 2015 [5].

A number of technologies are available for CD4 enumeration, with considerable variation in cost, complexity, and operating requirements (Tables 15). The traditional approach to calculating absolute CD4+ T lymphocyte counts is to use the total leukocyte count (or lymphocyte count) obtained from the hematology analyzer and then use the percentage CD4+ T lymphocytes from the flow cytometric analysis to calculate the absolute values—the so-called “dual platform” (DP) approach. Quite often, however, two separate samples are used in the procedure, one to obtain the total leukocyte count using a hematology analyzer and one to undertake the flow cytometry, each having its own in-built variation. Thus, when the results from each are combined to determine the absolute CD4+ T lymphocyte count, the variation is compounded such that inter-laboratory variation between centers can be as high as 40%. Thus, the need to derive accurate and precise absolute CD4+ T lymphocyte counts has led to the development of instruments that can produce both percentage and absolute values, termed the”single platform” approach (SP).

10.1371/journal.pone.0115019.t001Operating characteristics of flow cytometric methods for CD4+ T-cell enumeration, using conventional flow cytometers.
FLOW CYTOMETRY USING CONVENTIONAL FLOW CYTOMETERa
Conventional dual platformBead-based single platform technology on conventional flow cytometerPanleucogating
Assay nameVariousTrucount tubesFlow-Count beadsPerfect-CountPLG CD4
ManufacturerVariousBecton DickinsonBeckman CoulterCytognosBeckman Coulter
Compatible with various reagent systems and flow cytometersCompatible with various reagent systems and flow cytometers
InstrumentationFlow cytometer plus haematology analyserFlow cytometerGating strategy
Assay principleAbsolute count calculated using results from flow cytometry together with the total lymphocyte count from haematology analyserAbsolute CD4 counts determined using ratio of CD4 to a known quantity of fluorescent beads—no need for haematology analyserGating strategy using total white cell count
Trucount tubes contain premeasured quantity of lyophilised beadsLiquid beads need accurate pipetting by operatorTwo different bead populations allow detection of inadequate mixing
Parameters measuredAbsolute CD4 and CD4%Absolute CD4 and CD4%Absolute CD4 and CD4%
Others depend on reagent kits/methods
Specimen500μl whole blood in EDTAUp to 100μl whole blood in EDTA100μl whole blood in EDTA
ThroughputUp to 250 samples/dayUp to 250 samples/dayUp to 250 samples/day
Compatible with independent EQA/PT programmes?YesYesYesYesYes
Is there access to compatible full process QC reagent?YesYesYesYesN/A
Maximum length of time from blood draw to testing24 hours24 hours5 days
Can the test be performed on fixed/ stabilised bloodYesYesYesYesYes
Complexity/training requiredComplex. Significant training required.Complex. Significant training requiredModerate training requirement
Environmental/energy issuesRequires uninterrupted mains electricityRequires uninterrupted mains electricityRequires uninterrupted mains electricity
Robustness to heat/humidityClimate control recommendedClimate control recommendedClimate control recommended
Reagents require refrigerationReagents require refrigerationReagents require refrigeration

a Various flow cytometers available from various manufacturers. Examples include Becton Dickinson FACSCalibur, Beckman Coulter Epics XL.

10.1371/journal.pone.0115019.t002Operating characteristics of dedicated single platform CD4+ T-cell enumeration systems.
DEDICATED SINGLE PLATFORM CD4 SYSTEMS
Assay nameFACSCountGuava PCAPartec CyFlow Counter or CyFLow SL_3Apogee Auto40
ManufacturerBecton DickinsonMillipore (formerly Guava Technologies and now a division of Merck)PartecApogee flow systems
InstrumentationFlow cytometerFlow cytometerFlow cytometerFlow cytometer
Assay principleDedicated single platform bead-based flow cytometer, two colour, no-lyse no-washSingle platform volumetric flow cytometer. Microcapillary flow cell. Previously used EasyCD4 reagents (EasyCD4 system), since replaced by Auto CD4/CD4%. Company also produces the more complex easyCyte instruments, for research use only.Volumetric SP flow cytometer. Up to 3 parameters (2 colour plus SSC). Simplest is single parameter CD4. For CD4%, 2 colour (CD45/4) + SSC. SL_3 (also called SL green) operates on the same principles but is adaptable to wider applications. Other CyFlow instruments such as SL blue have a wider range of parameters and potential applications.Volumetric flow cytometer
Parameters measuredAbsolute CD4 count and CD4%Absolute CD4 count and CD4%Absolute CD4 count and CD4%Absolute CD4 and CD4%
Specimen50μl whole blood in EDTA10μl whole blood in EDTA20μl whole blood in EDTA50μl whole blood in EDTA
Throughput≥30 samples/hour, at CD4 counts≥400100–150 samples/day250 samples/day, 400 samples/day with loader20 samples/hour
Compatible with independent EQA/PT programmes?YesYesYesNo data available
Is there access to compatible full process QC reagent?YesNo manufacturer-produced full process controlManufacturer produces dry stabilised control bloodNo manufacturer-produced full process control
Maximum length of time from blood draw to testingStain within 48 hours of draw (24 hours for CD4%) (store at 20–25°C), analyse within 48 hours of staining.48 hours48 hoursPreferably run within 6–8 hours unless stabilised
Can the test be performed on fixed/ stabilised bloodYesYesYesYes
Complexity/training requiredOne day trainingOne day trainingModerate training requirementOne day training
Environmental/energy issuesRequires uninterrupted mains electricityRequires uninterrupted mains electricity Less biohazardous waste (reduced by 100 fold)Mains electricity or car battery or solar panelsUPS with battery back up included. Optional inverter allows instrument to be powered by vehicle battery
Robustness to heat/humidity10–35°C, 5–95% non-condensing humidity. Reagents require refrigeration.Instrument: up to 35°C. 10–90% non-condensing relative humidity. Reagents require refrigeration.Solid state laser more stable than water or air cooled gas lasers at high temperatures. Liquid reagents require refrigeration, or dry reagent kits available which can be stored at up to 60°C in dark for up to 12 monthsOption of thermostable reagents with 9 month shelf life at room temperature. Instrument: 5–35°C, <90% humidity.

PCA = personal cell analysis, SSC = side scatter.

10.1371/journal.pone.0115019.t003Operating characteristics of manual technologies for CD4+ T-cell enumeration.
MANUAL METHODS
Assay nameCoulter Manual CD4 Count Kit (Cytospheres)Dynal T4 Quant Kit (Dynabeads)
ManufacturerBeckman CoulterInvitrogen/Dynal Biotech
InstrumentationLight microscope, haemocytometerFluorescent or light microscope, haemocytometer, magnet
Assay principleInert latex spheres coated with monoclonal Ab form rosettes on contact with CD4 cells—readily visible by light microscopyMagnetic polystyrene beads, coated with mouse monoclonal Ab, allow isolation of CD4+ T cells followed by counting using fluorescent (preferable) or light microscope
Parameters measuredAbsolute CD4 countAbsolute CD4 count
Specimen100μl whole blood in EDTA125μl whole blood in EDTA or ACD
Throughput10 samples/day. Operator fatigue a limitation. Suggest stop counting at 500 cells. Batch size limited to 2 samples10 samples/day. Operator fatigue a limitation. Suggest stop counting at 500 cells. Batch size limited to 6 samples
Compatible with independent EQA/PT programmes?NoNo
Is there access to compatible full process QC reagent?NoNo
Maximum length of time from blood draw to testing72 hours at 20°C72 hours at 20°C
Can the test be performed on fixed/ stabilised bloodNoCompatible with Cyto-Chex (Streck, USA) stabilised refrigerated samples up to 9 days after collection. Not compatible with TransFix (CytoMark, UK).
Complexity/training required1–3 days training1–3 days training
Environmental/energy issuesRequires power for microscopeRequires power for microscope
Robustness to heat/humidityReagents need refrigerationReagents need refrigeration
10.1371/journal.pone.0115019.t004Operating characteristics of other commercially available technologies for CD4+ T-cell enumeration.
OTHER COMMERCIALLY AVAILABLE TECHNOLOGIES
Assay namePocH-100i/KX-21N with Dynal DynabeadsCD4 Select
ManufacturerSysmexi+MED Laboratories Co. Ltd.
InstrumentationHaematology analyserHaematology analyser, magnet, rotator
Assay principleCD4 cell isolation using modified Dynal dynabeads, counting using haematology analyser with specific softwareFerrous beads coated with MT4 mAb bind to CD4 T cells, which are then removed using magnet. Whole blood and CD4-deplete blood counted on haematology analyser. CD4 count and % calculated using established formulae
Parameters measuredAbsolute CD4 count and CD4%Absolute CD4 and CD4%
Specimen125 μl whole blood in EDTA400μl whole blood in EDTA
Throughput12 samples/hourNo data available
Compatible with independent EQA/PT programmes?No EQA evaluation data availableNo EQA evaluation data available
Is there access to compatible full process QC reagent?No data availableNo data available
Maximum length of time from blood draw to testing24 hours at 4 or 20°CNo data available
Can the test be performed on fixed/ stabilised bloodNo data availableNo data available
Complexity/training requiredModerate complexityModerate complexity
Environmental/energy issuesRequires power for haematology analyserRequires power for haematology analyser and rotator
Robustness to heat/humidityNo data availableNo data available
10.1371/journal.pone.0115019.t005Operating characteristics of Point of Care (POC) technologies for CD4+ T-cell enumeration.
POINT OF CARE TECHNOLOGIES
Assay namePointCare NOWPimaCyFlow miniPOC
ManufacturerPointCare TechnologiesAlere (formerly Inverness Medical)Partec
InstrumentationPoint of care instrumentPoint of care instrumentPoint of care instrument
Assay principleCD4 and haematology contained in one unit.All test reagents sealed within disposable cartridge.Compact flow cytometric device. Uses dry reagents
Gold nanoparticles conjugated to CD4 antibodies (no fluorescence). LED light source.Portable analyser uses static image analysis and cell counting principles
Parameters measuredAbsolute CD4, CD4%, Hb, differential white cell countAbsolute CD4Absolute CD4 and CD4%
Specimen40μl whole blood in EDTA25μl capillary or venous blood20μl whole blood in EDTA
Throughput50 samples/day3 samples/hour250 samples/day
Compatible with independent EQA/PT programmes?NoUnder investigationCompatible with German EQA programme
Is there access to compatible full process QC reagent?No data availableNo data availableCompany produce dry stabilised blood samples for QC
Maximum length of time from blood draw to testing8 hours24 hoursProvisional data suggest 24 hrs
Can the test be performed on fixed/ stabilised bloodNo data availableCompatible with Transfix (CytoMark, UK)No data available
Complexity/training requiredModerateMinimalModerate
Environmental/energy issuesRequires mains or battery powerRequires mains or battery powerRequires mains or battery power
Robustness to heat/humidityReagents do not require refrigeration. Instrument: 18–34°C, <80% relative humidity, non-condensingNo refrigeration required (dried reagents)No refrigeration required (dried reagents)

Two SP approaches are in widespread use today: volumetric and bead based. The principle of the volumetric approach is that a known volume of sample is passed through the flow cell (and interrogated by the laser beam) in a known amount of time. The alternative approach is to use bead-based technologies where a known number of beads is added to a known volume of sample thus allowing calculation of the bead to cell ratio and the subsequent calculation of the absolute cell count, in this instance, CD4+ T lymphocytes. An important feature of any absolute counting system is pipetting accuracy and minimum sample manipulation. The introduction of SP technologies has had a beneficial effect and lowered inter-laboratory variation in CD4 enumeration.

It is critical that country programmes consider whether these tests can give accurate and reproducible results as well as being appropriate for the setting [6]. In particular, we focussed on how bias and misclassification probabilities of different CD4 assays may affect eligibility for ART initiation. Misclassification probabilities give clinically useful measures of test performance and should be reported in evaluations of CD4 technologies. An upward misclassification around a treatment threshold means that a patient who should be eligible for treatment would be denied treatment, while a downward misclassification would not lead to ineligibility for treatment. To date, there have been no systematic reviews of the performance of CD4 technologies. Here we provide an evaluation of the performance characteristics of CD4 technologies through a systematic review of published literature.

Methods

We performed a systematic review of studies evaluating the performance of CD4 enumeration technologies. A search of the Cochrane Library and the Centre for Reviews and Dissemination databases, including the Database of Abstracts of Reviews of Effects (DARE), the National Health Service Economic Evaluation Database (NHS EED) and the National Institute for Health Research Health Technology Assessment (NIHR HTA) database found no existing reviews addressing the review objective.

We followed standard guidance in performing the review [7]. Objectives and methods of the review were documented in a review protocol, which is included as S1 Methods.

Eligibility criteria

Eligibility criteria were defined using the PICOS (Population, Interventions, Comparisons, Outcomes, Study Design) format. Studies evaluating the accuracy and/or precision of any CD4 technology commercially available at the time of the review were considered eligible for inclusion. Currently, no “gold” standard technology or internationally recognised reference preparation exists for CD4 enumeration, and a wide range of flow cytometric technologies have been used as comparators [6].

For the purposes of this review, we included studies that used as reference technologies any flow cytometric method considered to be acceptable by the WHO HIV diagnostics working group named in the review protocol (S1 Methods).

Information Sources

Studies were identified by searching two electronic databases—MEDLINE and EMBASE, scanning the reference lists of the Nature supplement Evaluating diagnostics: the CD4 guide, and by inviting the WHO working group, whose members are authors of the Nature supplement to identify relevant studies for the review [6,8].

Search Strategy

We used the following search terms to search the electronic databases: “CD4”, “technolog*”, “methodolog*”, “techn*”, “method*”, “test”, “evaluation”, “validation”, “accuracy”, “comparison”, “efficacy”, “performance”, “reproducibility”, “precision”, “flow AND cytometry”. Subject headings included: “CD4 antigen”, “antigens, CD4”, “CD4 lymphocyte count”, “CD4-positive T-lymphocytes”, “technology”, “methodology”, “technique”, “evaluation”, “clinical evaluation”, “economic evaluation”, “evaluation research”, “evaluation and follow up”, “instrument validation”, “validation process”, “validation study”, “evaluation studies as topic”, “validation studies”, and “flow cytometry”. Full electronic search strategies and review protocols are detailed in S1 Methods.

Study selection

Articles were exported from the search database to EndNote and screened for relevance (Fig. 1). Data were extracted by two independent reviewers (SG and KS) and disagreements resolved through consensus.

10.1371/journal.pone.0115019.g001PRISMA flow diagram of study selection.

EQA: external quality assurance, FC: flow cytometry.

Data extraction

The following data were extracted: study location, index test, reference test, and population (HIV positive or HIV positive and negative). Data on accuracy and precision included bias or mean difference and limits of agreement, misclassification probabilities (when sensitivity or specificity values were given, misclassification probabilities were calculated), and coefficient of variation. Where possible, HIV positive data alone were extracted. Where this was not possible, combined HIV positive and negative data were extracted. Only data within the clinically relevant range (thresholds of 200, 350, 500 cells/μl) were extracted as per the inclusion criteria.

Studies should report not only percent misclassification around clinically important CD4 cell thresholds (e.g., 200, 350 or 500 cells/μl), but should also report the magnitude of these misclassifications.

The secondary outcome measure addressed precision or reproducibility. Precision is particularly important when following a patient’s serial measurements using the same technology. Precision can be measured within-laboratory or between-laboratories and is expressed as percent coefficient of variation (%CV).

Studies meeting inclusion criteria were also assessed for bias and quality on ten points drawn from the STARD guidelines (Fig. 2) [9]. This review has been reported following the PRISMA statement guidance for reporting of systematic reviews [10,11].

10.1371/journal.pone.0115019.g002Methodological quality of included studies.

EQA: external quality assurance, IQC: internal quality control.

Results

A summary of different commercially available CD4 technologies, including their assay principles, operational characteristics and compatibility with international external quality assurance programme reagents, is shown in Tables 15. Conventional flow cytometry based technologies include DP technologies such as the BD FACSCalibur (BD Biosciences, a division of Becton Dickinson, Franklin Lakes, NJ, USA [BD]) and the Epics XL (Beckman Coulter, Inc., Pasadena, CA, USA [BC]); bead-based SP technologies such as the BD Trucount Tube, the BC Flow Count Tubes, the Cytognos Perfect Count (Cytognos S.L., Salamanca, Spain [Cytognos]), and the BC Pan-leucogating (PLG) FlowCare CD4. Dedicated SP CD4 technologies include bead-based BD FACSCount, and three volumetric assays, Millipore Guava PCA (EMD Millipore, Darmstadt, Germany [Guava]), Partec CyFlow Counter (Partec, a division of Sysmex, Corporation, Kobe, Japan [Partec]) and Apogee Auto40 Flow Cytometer (Apogee Flow Systems, Hertfordshire, UK [Apogee]). There are two manual microscopy based counting methods, the BC Cytospheres and the Dynal T4 Quant (Dynal Biotech ASA, a division of Thermo Fisher Scientific Inc., Waltham, MA, USA [Dynal]). Point-of-care (POC) analysers include the Sysmex pocH-100i with Dynabeads from Dynal, the i+MED CD4 Select (i+ MED Laboratories, Bangkok, Thailand [i+ MED]), the Pima Analyzer (Alere Inc., Waltham, MA, USA [Alere]), and PointCare NOW (PointCare Technologies, Marlborough, MA, USA [PointCare]).

Study selection

This systematic review was first performed in July 2009. Of the 433 studies in the search, 345 were excluded as they were not performance evaluations. After further triage, 20 studies that measured bias, misclassification and/or %CV were accepted for inclusion in this review (PRISMA flow diagram, Fig. 1). A second search was conducted in April 2013 using the same search strategy and review protocol with the goal of capturing more POC CD4 enumeration technologies in the review. An additional 12 studies were included. A summary of data extracted from all eligible studies with data on bias, misclassifications and/or %CV is shown in S1 Dataset.

Study characteristics

A summary of study characteristics is shown in Table 6.

10.1371/journal.pone.0115019.t006Characteristics of included studies.
StudyLocationLaboratory or healthcare level at which the study was performedReference standardIndex test (test under evaluation)PopulationNo. of samples used in accuracy studies (n)
Balakrishnan 200412Chennai, IndiaCharitable foundation clinical/research labBD FACSCountCoulter CytospheresHIV pos122
Carella 199513USATeaching hospital / research lab?DP FCCoulter CytospheresHIV pos and neg117
Landay 199317USA and UgandaUniversity research hospitalsDP FCCoulter CytospheresHIV pos and neg382
Gernow 199515Denmark and Ivory CoastNot clearDP FCCoulter CytospheresHIV pos and neg123
Didier 200114ParisTeaching hospital labSP FC using BD Trucount tubesCoulter CytospheresHIV pos and neg55
SP FC using BD Trucount tubesDynal DynabeadsHIV pos and neg46
SP FC using BD Trucount tubesBD FACSCountHIV pos and neg45
Karcher 200616UgandaLocal lab, field conditionsDP FCCoulter CytospheresHIV pos131, from 102 donors
DP FCPartec CyFlow Counter CD4/SCC NLNWHIV pos128 from 121 donors
Lutwama 200818UgandaUniversity laboratoryBD FACSCount or FACSCaliburCoulter CytospheresHIV pos1444 (≈497 donors)
BD FACSCount or FACSCaliburDynal DynabeadsHIV pos1671
Diagbouga 200319Five countries in West AfricaLocal laboratoriesSP FC using BD Trucount tubesDynal DynabeadsHIV pos657, from 301 donors
Imade 200521NigeriaUniversity hospital laboratoryPartec CyFlowDynal DynabeadsHIV pos and neg40
Idigbe 200620NigeriaResearch laboratoryBD FACSCountDynal DynabeadsHIV pos and neg97 (46 HIV pos, 51 neg)
Balakrishnan 200623Chennai, IndiaCharitable foundation clinical/research labBD FACSCountGuava Easy CD4HIV pos and neg228
Kandathil 200524South IndiaMedical college laboratory and charitable foundation clinical/research laboratoryBD FACSCountGuava Easy CD4HIV pos and neg72 (51 HIV pos, 21 neg)
Spacek 200628United StatesUniversity laboratoryDP FCGuava Easy CD4HIV pos77
UgandaUniversity laboratoryBD FACSCountGuava Easy CD4HIV pos141
Thakar 200629Pune, IndiaResearch instituteDP FCGuava Easy CD4HIV pos79
Pattanapanyasat 200725Bangkok, ThailandTeaching hospital laboratory and ministry of public health-cdc labSP FC using BD Trucount tubesGuava Easy CD4HIV pos250
BD FACSCountGuava Easy CD4HIV pos250
Pattanapanyasat 200826ThailandTeaching hospital laboratorySP FC using BD Trucount tubesGuava Easy CD4HIV pos150
BD FACSCountGuava Easy CD4HIV pos150
SP FC using BD Trucount tubesPartecv CyFlow SL_3HIV pos150
BD FACSCountPartec CyFlow SL_3HIV pos150
Renault 201027Burkina FasoCommunity laboratoryBD FACSCountGuava Easy CD4HIV pos98
Zijenah 200636ZimbabweUniversity laboratoriesBD FACSCountPartec CyFlow CounterHIV pos150
Fryland 200632MalawiDistrict hospital laboratoryBD FACSCountPartec CyFlow CounterHIV pos and neg311 (276 HIV pos, 35 HIV neg)
Cassens 200430Africa and Asia‘Hospital laboratories’VariousPartec CyFlow SL (SCC + CD4), NLNWHIV pos and neg434
Pattanapanyasat 2005 ‘Evaluation of a new.’ 52Bangkok, ThailandTeaching hospital laboratorySP FC using BD Trucount tubesPartec CyFlow SL_3 (using single parameter CD4)HIV pos200
BD FACSCountPartec CyFlow SL_3 (using single parameter CD4)HIV pos200
Manasa 200734ZimbabweNational reference laboratoryDP FCPartec CyFlow SL_3 (SCC +CD4 +CD45) (absolute CD4 and %CD4)HIV pos229
Dieye 200531SenegalLaboratory of a University HospitalSP FC using BD Trucount tubesPartec CyFlow SL Blue CD45/CD3/CD4 + SCCLNWHIV pos and neg121 (102 HIV pos donors, 28 neg)
BD FACSCountPartec CyFlow SL Blue CD45/CD3/CD4 + SCCNLNW121 (102 HIV pos donors, 28 neg)
Lynen 200633CambodiaLaboratory of an NGO-run hospitalBD FACSCountPartec CyFlow SL_3 CD4+SSC LNW manual gating‘Likely to be HIV positive’115
Strauss 199642Belgium, UK, USA, SpainResearch laboratories, teaching hospital laboratory and manufacturerDP FCBD FACSCountHIV pos and neg
Lopez 199939Spain and Portugal7 centres: 6 hospital laboratories and one University laboratoryBD FACScanBD FACSCountHIV pos and neg49 (37 HIV pos, 12 HIV neg)
Johnson 199538USAUniversity laboratory?DP FCBD FACSCountHIV pos and neg47
Nicholson 199440USAGovernment laboratoryDP FCBD FACSCountUnknown for accuracy studies50
Lyamuya 199622TanzaniaTeaching hospital laboratory?DP FCBD FACSCountHIV pos and neg173
DP FCDynal DynabeadsHIV pos and neg189
Bergeron 201037Morocco/CanadaNational reference laboratoriesBD FACSCountBD FACSCount using Rea T Count dry reagentsHIV pos167
BD FACSCaliburBD FACSCount using Rea T Count dry reagentsHIV-pos80
Sitoe 201141MozambiqueNot statedBD FACSCount using venous bloodBD FACSCount using capillary bloodHIV pos101 adults
BD FACSCalibur using venous bloodBD FACSCalibur using capillary bloodHIV pos101 adults
Schnizlein-Bick 200046USA5 NIAID certified FC laboratoriesDP FCSP FC using BD Trucount tubesHIV pos60 common samples shipped to 5 sites, plus 14 local donors at each of 5 sites
Nicholson 199745USAUniversity laboratoryDP FCSP FC using BD Trucount tubesHIV pos and neg81
Higgins 200743USAResearch laboratoryDP FCSP FC using BD Trucount tubesHIV pos25
Jeganathan 200844AustraliaCentral laboratoryDP FCSP FC using BD Trucount tubesHIV pos60 paired samples from 18 patients
Reimann 200047USA5 clinical FC laboratoriesDP FCSP FC using BC Flow-Count fluorospheres: two colour + flow countHIV pos≥14 from each of 5 local sites = 71, plus 67 central specimens
DP FCSP FC using BC Flow-Count fluorospheres: tetraONE systemHIV pos≥14 from each of 5 sites = 71, plus 71 central specimens
Storie 200448UKTeaching hospital laboratory?SP FC using BD Trucount tubesSP FC using Cytognos Perfect-Count beadsHIV pos104
Storie 200349UKTeaching hospital laboratory?SP FC with BD Trucount tubesSP FC using flow rate calibrationHIV pos113
SP FC with BD Trucount tubesDP PLGHIV pos113
Denny 200850USA5 FC laboratoriesVaried by lab—SP or DP FCDP PLGHIV pos99 samples sent to 5 labs, plus 58 local samples
Pattanapanyasat 2005 ‘A multicentre evaluation.’ 35Thailand2xteaching hospital labs, 2xregional hospital labsDP FCDP PLGHIV pos611
Glencross, 200251Johannesburg, South AfricaTeaching hospital laboratoryDP FCDP PLGNot stated183
Volumetric SP FCDP PLGNot stated112
SP FC using BD Trucount tubesDP PLGNot stated112
Sippy-Chatrani 200853BarbadosNational HIV laboratorySP FC with BC Flow-Count fluorospheresSP PLG using BC FlowCAREHIV pos153
Briggs 200954UKlaboratorySP FCSysmex PocH-100i with Dynal DynabeadsHIV positive for accuracy studies, HIV positive/ leukaemia /lymphoma for precision115
Srithanaviboonchai 200855ThailandResearch lab?DP FCi+MED CD4 selectHIV positive100
Mtapuri-Zinyowera 201057ZimbabweVCT clinicSP FC using BD Trucount tubesAlere Pima using capillary bloodHIV pos165
Jani 201156MozambiquePrimary healthcare clinicsBD FACSCaliburAlere PIMA using capillary and venous bloodHIV pos135
Sukapirom 201158ThailandTeaching hospital laboratoryBD FACSCountAlere PIMA using venous bloodHIV pos203
Manabe 2012UgandaTertiary hospital laboratoryBD FACSCalibur with TruCount beadsAlere PIMA using capillary and venous bloodHIV pos380
Herbert 201361UKHospital laboratory (HIV service)Coulter FC 500Alere PIMA using capillary and venous bloodHIV pos283
Bergeron 201260Belgium, Canada, Mozambique, South Africa, USANational laboratoriesBD FACSCalibur, except in South Africa where the BC Epics-XL was usedPointCare NOWHIV pos and neg472
Logan 201362USAResearch laboratoryBD FACSCaliburmBio Snap Count using capillary and venous bloodHIV pos146
Mbopi-Keou 201263CameroonNational Health LaboratoryBD FACSCaliburApogee Auto40 Flow CytometerHIV pos and neg234 (adults and children)
Mbopi-Keou 201264CameroonHIV reference LaboratoryBD FACSCaliburApogee Auto40 Flow Cytometervolunteers257 (adults and children)

FC = flow cytometry.

DP = dual platform.

SP = single platform.

SCC = side scatter.

BD = Becton Dickinson.

BC = Beckman Coulter.

LNW = lyse, no-wash.

NLNW = no-lyse, no-wash.

VCT = Voluntary Counselling and Testing.

At least one published performance evaluation study was found for each of the following technologies: BC Cytospheres [1218], Dynal Dynabeads [14,1822], Guava PCA [2329], Partec CyFlow instruments [16,26,3036], BD FACSCount [14,3742]; BD Trucount tubes [4346], BC Flow-Count fluorospheres [47], Cytognos Perfect-Count microspheres [48], SP Flow Cytometry using flow rate calibration [4952]; BC PLG FlowCARE CD4 [53], Sysmex PocH-100i with Dynal Dynabeads [54], i+MED CD4 Select [55], Alere Pima Analyzer [5658,61], PointCare NOW [60], Apogee Auto40 [59,63,64] and MBio Snap Count (MBio Diagnostics, Inc., Boulder, CO, USA [MBio]) [62].

No published, peer-reviewed performance evaluations of the commercially available technology Partec miniPOC were found by our literature search, and the MBio assay is not yet commercially available.

Methodological quality of included studies

The findings of the quality assessment of included studies are summarised in Fig. 2. Most studies reported the index test (test under evaluation) and the reference standard in sufficient detail to be reproduced, but few studies reported whether staff at the evaluation sites were proficient at performing the reference standard and/or sufficiently trained on performing the index test. Few studies reported internal quality controls being performed during the evaluation period. Without these quality measures, it would be difficult to differentiate whether the bias or misclassification between the index and reference tests was due to differences in inherent test characteristics or to operator error.

Manufacturer involvement was evident in a number of studies. Seven studies declared one or more authors to be affiliated with the manufacturer of the index test [12,17,39,42,46,54]. Four studies were partially sponsored by the manufacturer [25,26,45,53]. One study stated that the manufacturer’s site was one of the study sites [42], and four studies declared donation of reagents or equipment by the manufacturer [15,29,38,40]. A further four studies could be considered to be calibration or test developers’ papers [30,49,51,55]. In the absence of definitions for a sponsored study versus an independent evaluation, it is not clear to what extent the inclusion of manufacturers as co-authors of papers influenced the study results.

Accuracy

As there is no international standard for CD4 enumeration, a variety of reference standard technologies were used for evaluating the performance of new CD4 technologies, making it difficult to pool data on bias and misclassification across all studies.

Bias

Bias (mean difference) data were collated and represented graphically but only from studies that compared the index tests mean difference to the same reference technology (Table 6 and Fig. 3a, b). FACSCount and FACSCalibur were the most common reference technologies.

10.1371/journal.pone.0115019.g003A. Bias compared to FASCount as the reference technology. B. Bias compared to FASCalibur as the reference technology.

Bias was evaluated at CD4 thresholds of 200 cells/μl and 350 cells/μl using the BD FACSCount as a reference method. For the Pima Analyser, at CD4 counts at both <350 cells/μl and >350 cells/μl, the studies found that the Pima Analyser consistently underestimated CD4 counts by-16.6 cells/μl (limits of agreement-88.4, +55.3 cells/μl at <350 cells/μl) and at-70.7 cells/μl (limits of agreement-216.5, +75 cells/μl) [58]. However, when the Guava Easy CD4 was compared to the FACSCount, the bias for CD4 counts <350 cells/μl was +13 cells/μl (limits of agreement-27, +53) and for cell counts >350 cells/μl, bias was-45.3 cells/μl [23,27].

The FACSCalibur, in comparison to the FACSCount, overestimated CD4 counts by +13.1 cells/μl at values of <350 cells/μl [41]. However, bias estimation of manual bead based methods showed underestimation of CD4 counts at <350 cells/μl of-35.2 cells/μl (limits of agreement-164.9,+ 94.6) and -0.4 cells/μl (limits of agreement-126, +125.2) for BC Cytospheres and Dynal Dynabeads respectively [18].

When the overall bias for all CD4 technologies was calculated, this ranged from-70.7 to +47 cells/μl for CD4 counts >350 cells/ μl and -35.2 to +13.1 cells/μl for CD4 counts <350 cells/μl when compared to the FASCount as a reference method.

We then progressed to study bias data using a threshold of 200 cells/μl compared to FACSCount as a reference method. A total of six publications were identified covering the following technologies, the Apogee Auto 40 [59], Guava Easy CD4 [23,2527], and Partec CyFlow Counter [26,52].

The four studies that had reported data at <200 cells/μl and compared the Guava Easy CD4 to the FACSCount, all had a positive bias that ranged from +10 to +45.5 cells/μl [23,2527]. One study had data for CD4 counts >200 cells/μl and had a bias of +44.9 cells/μl (limits of agreement-112.6 to + 212.3) [25]. It is interesting to note that all studies reporting the performance of the Guava Easy CD4 showed that this assay overestimated CD4 counts compared to the FACSCount [23,2527].

Two studies compared the Partec CyFlow Counter to the FACSCount. One showed an over-estimation by +0.8 cells/μl (limits of agreement-21.7 to +23.2) while the other showed an underestimation by-5.8 cells/μl (limits of agreement-37.6 to +25.9) [26,52].

Overall when comparing the technologies to the FACSCount as a reference method, bias ranged from +44.9 to -12.1 for CD4 counts >200 cells/ μl and +45.5 to -5.8 for CD4 counts <200 cells/ μl. Bias using FACSCalibur as a reference method showed similar results to that using FACSCount (Fig. 3a and b).

Misclassification

Fig. 4 showed the range of misclassifications at thresholds of 200 and 350 cells/μl of new CD4 assays compared to different reference standards.

10.1371/journal.pone.0115019.g004Misclassification (%), using a CD4 thresholds of 350 cells/μl and 200 cells/μl.

Nine studies provided data on misclassification probabilities using a cut-off of 350 cells/μl [16,18,27,56,6064]. Data were available on the following assays: BC Cytospheres [16,18], Dynal Dynabeads [18], Guava Easy CD4 [27], Partec CyFlow Counter [16], Pima Analyzer [56,61], PointCare NOW [60], MBio Snap Count [62], and Auto40 Flow Cytometer [63,64].

Two studies [63,64] evaluated the Auto40 Flow Cytometer compared to the FACSCalibur in Cameroon. One study [63] reported upward and downward misclassification probabilities of 8% and 2%, respectively, while another [64] found the likelihood of under-treatment (upward misclassification) to be 3% and the probability of over-treatment (downward misclassification) to be 2%.

Karcher et al. conducted a large trial in field conditions in Uganda comparing BC Cytospheres and the Partec CyFlow Counter to DP flow cytometry [16]. HIV positive patients were recruited, the majority of whom had CD4 counts within a range from 0 to1200 (median 332 cells/μl). Of samples with CD4 counts <350 cells/μl measured by DP flow cytometry, BC Cytospheres misclassified 16% of the patients as having CD4 counts of >350 cells/μl, thereby denying them of eligibility for treatment. Similarly for those with CD4 counts > 350 cells/μl measured by DP flow cytometry, BC Cytospheres misclassified 20%, indicating that 20% of patients not qualifying for treatment using the reference test would have done so if BC Cytospheres were used. Karcher et al. also compared the Partec CyFlow to the DP flow cytometer and found that of samples with counts <350 cells/μl, 29% were misclassified as having >350 cells/μl by the Partec CyFlow Counter, and 7% of samples with CD4 counts >350 cells/μl were misclassified downward as being <350 cells/μl [16].

Lutwama et al. conducted a large study of manual technologies in Uganda; they recruited only HIV positive patients, the majority of whom had CD4 counts within the clinically important range (range in study: 0–900 cells/μl) using the reference standard technology [18]. Of samples with counts of <350 cells/μl using the reference standard technology, BC Cytospheres misclassified only 1% as >350 cells/μl. However, of those with counts >350 cells/μl, 68% were misclassified as <350 cells/μl (indicating that 68% of patients not qualifying for treatment using the reference test would have done so if BC Cytospheres were used). Dynal Dynabeads had upward and downward misclassification probabilities of 6% and 30% respectively.

Renault et al. (2010) conducted a comparison study between the Guava Easy CD4 and FACSCount. Across a range of CD4 (0–1100 cells/μl), the upward and downward misclassification was calculated and found to be 6.1% and 9.4%, respectively [27].

One study reported on evaluations of the PointCare NOW assay (this instrument has since been re-marketed/rebranded as HumaCount CD4now (Human Diagnsotics Worldwide mbH, Weisbaden, Germany) in five countries (Mozambique, Belgium, Canada, USA and South Africa) [60]. Mozambique, Belgium, Canada and USA used the FACSCalibur as a reference standard while South Africa compared the PointCare NOW to the Epics XL. Upward and downward misclassification were reported by country: Mozambique, +51%, -20%; Belgium, +62%, -4%; Canada, +50%, -0%; USA, +0%, -3%; and for South Africa, +64%, -6%. Overall misclassification was also calculated, and it was found that testing with PointCare NOW would have led to 47% of patients with CD4 counts less than 350 cells/μl not eligible to receive treatment (upward misclassification) and 6% of patients with CD4 counts greater than 350 cells/μl eligible to receive treatment (downward misclassification).

Of the two studies evaluating the Pima Analyzer, Herbert et al. (2013) used the BC Cytomics FC 500 as a reference standard while Jani et al. compared the Pima Analyzer to the BD FACSCalibur [56,61]. Across the clinically relevant range (60–1200 cells/μl), Herbert et al. found the upward and downward misclassification to be 6.1% and 9.4% respectively. MBio Snap Count was evaluated by Logan et al. (2013) and compared to the FACSCalibur [62]. Of the 94 samples, 2.1% were misclassified upward and 3.2% were misclassified downward at a threshold of 350 cells/μl.

Thirteen studies presented misclassification data using a CD4 cut-off of 200 cells/μl; these are presented in Fig. 4b [12,13,15,17,18,20,23,28,36,56,60,61,63,64].

Even through misclassification probabilities can be influenced by the number of patients with CD4 counts close to the threshold in each study, Pointcare NOW showed an overall tendancy towards upward misclassification at both thresholds of 200 and 350 cells/μl. Most other technologies showed misclassification probabilities of <10%.

Precision

Forty-four percent of studies reported within-laboratory precision of absolute CD4 count measurement, using replicates of fresh whole blood. [15,17,19,21,24,29,3234,3842,4547,50,54,56,58] Fig. 5 shows the inter- and intra assay variations expressed as % CV for the Apogee Auto assay and the intra-assay variation for the mBio SnapCount [62,63].

10.1371/journal.pone.0115019.g005Intra- and inter-assay variation (% CV) for the Apogee Auto 40 and for the MBio SnapCount at thresholds <350 cells/mL and >350 cells/mL.

Five studies reported between-laboratory precision using whole blood. Two were studies evaluating BD FACSCount, [39,42] one evaluated Panleucogating, [50] and two were studies evaluating bead-based SP technology (BD Trucount tubes and BC Flow-Count fluorospheres) [46,47]. Gernow et al. studied the reproducibility of BC Cytospheres and found poor precision, with a coefficient of variation of 58% [15]. The study by Landay et al., however, found precision levels more in keeping with the other technologies [17].

Overall, in studies addressing between-laboratory precision, SP flow cytometry using BD Trucount tubes or BC Flow-Count fluorospheres showed less inter-laboratory variability than the DP comparators. [46,47] In addition, Denny et al. demonstrated improved inter-laboratory precision using DP Panleucogating compared with technologies which included DP or SP conventional flow cytometry [50]. External quality assurance data showed that the BD FACSCount, which is most commonly used a reference standard for CD4 assay evaluations, has within-laboratory and between-laboratory precision of 15% or less [38,39,42,46].

Discussion

This review highlights the difficulties of answering clinically relevant questions about CD4 test performance from the published literature. A minority of studies reported clinically useful measures of accuracy, and few POC tests were carried out under field conditions.

It can be seen that whatever technology is chosen, there is variability associated with CD4 measurement. It should be noted that there is also significant physiological variability in CD4 count, that may account for as much as, if not more than, technical variability of CD4 measurement [6567]. Performance characteristics vary between technologies and for the same technology depending on the reference technology used as a comparator. These characteristics have important implications both for individual patient management and for HIV treatment programmes. It is essential to consider assay performance as well as operating characteristics when choosing a technology. However, these data are not always available in the literature, and currently, evaluations are not sufficiently robust or comprehensive to give a clear idea of the comparative merit of different technologies.

Misclassification probabilities describe the likelihood that a test will incorrectly categorise a result as higher or lower than a given cut-off value as measured by a reference standard. They are clinically relevant measures of accuracy, as they can be used to assess the likelihood that a patient will be incorrectly classified above or below a defined CD4 threshold used in clinical decision making. Misclassification probabilities for the same assay can vary not only because the test is compared to different reference standards, but also because the probabilities are affected by the number of samples clustered around the thresholds of 200 or 350 cells/ul.

Two types of misclassification can be defined—upward misclassification and downward misclassification. Upward misclassification around a treatment threshold may be the most clinically important, leading to a delay in starting ART in some patients, with potentially harmful consequences. Downward misclassification on the other hand would be expected to lead to ART use earlier than indicated, with potential implications for cost and drug exposure. Given the trend towards earlier initiation in global and national guidelines, a degree of over-treatment is likely to be preferred over significant under-treatment [68]. Furthermore, the use of CD4 counts alone to assess ART immunological failure in the absence of viral load monitoring will, because of the biases observed, lead to some patients not receiving the appropriate clinical intervention.

Misclassification data showed that manual technologies [18], particularly the method using BC Cytospheres, were associated with substantial downward misclassification. It would therefore be expected that the implementation of these tests would lead to the decision to treat potentially large numbers of additional patients who have CD4 counts above the guideline threshold when using the reference test. Less upward misclassification was seen, suggesting that under-treatment might be less of a problem with these technologies. Upward misclassification by either manual technology is however likely to be an underestimate as the majority of counts in this study were very low (less than 25% of samples had counts >200 cells/μl); if the tests were to be used in a population with counts closer to the treatment threshold (as might be the case if used primarily in asymptomatic patients), upwards misclassification would be expected to be higher.

Limited misclassification data were available for the Partec CyFlow instruments. Of concern, one study evaluating the Partec CyFlow Counter under field conditions found 29% upward misclassification; that is, 29% of patients potentially eligible for treatment may have been denied treatment if the Partec CyFlow Counter-determined CD4 counts were the only criteria for assessing eligibility [16]. No other studies of the Partec CyFlow Counter or other CyFlow instruments reported misclassification probabilities at 350 cells/μl. Therefore, we do not know if this finding was replicated elsewhere. The Guava PCA (using EasyCD4 reagents) and the Pima Analyzer showed acceptable upward and downward misclassification rates.

There is some disparity in precision reported for the BC Cytospheres, and the reason for this disparity is not clear. The CD4 counts of the 19 samples used for replicate analysis in the study conducted by Gernow et al were not stated; however, they included 12 HIV negative samples that might be assumed to have high counts [15]. Given that several papers found BC Cytospheres to have poorer performance at higher counts, this may be in keeping with poor reproducibility in these replicates. Another study, conducted by Landay et al., found better precision on a sample with a CD4 count of 1200 (%CV 3·5%) than on a sample with a CD4 count of 200 (%CV 10·8%) [17]. Manual methods, although employing simple technology, are labor intensive and require significant user skill. Inadequate training, lack of supervision, or user fatigue may lead to poor performance of these techniques. Neither study described the training received by technicians performing the manual tests, nor reported blinding. Between-laboratory precision is likely to be superior with SP technologies (using BD Trucount tubes or BC Flow-Count fluorospheres) than with conventional DP technologies. As more point-of-care devices are introduced to lower levels of the health system, where training and supervision can be challenging, the lack of adequate training and supervision may introduce additional sources of error, contributing to decreased assay precision. This should be addressed through the development of a comprehensive training and supervision policy and implementation plan for the introduction of POC devices. In addition to the studies presented in this review, evaluations have been performed by government agencies and other national bodies that have not been published in peer-reviewed journals. An evaluation of BC Cytospheres published by the Medical Devices Agency of the UK included samples from 17 HIV positive subjects, and compared BC Cytospheres against DP flow cytometry [69]. Accuracy was reported using assessment of bias; misclassification probabilities were not reported. Imprecision was addressed using 6–7 replicates of 6 samples, and found a CV range of 3·2–17·6% (mean 8·5%). Unpublished evaluations have not been included in this review.

A recent review of external quality assurance (EQA) programmes involving 58,626 CD4 data sets from over 3,000 laboratories over a 12-year period show that SP technologies consistently give lower relative errors and confidence limits than DP technologies at clinically significant absolute CD4 counts [70].

Limitations of this review

Limitations include the fact that we only included papers published in the English language, and we may have overlooked data because of this limitation. Limiting the search to the peer-reviewed literature may have overlooked robust evaluations conducted by national reference facilities or similar institutions.

It is important to consider that the reference standard technologies themselves are not perfect. Misclassification assumes that the reference result is accurate, i.e., the closest approximation to the truth. Thus, a result considered as a misclassification may in fact be correct. The reference technology if performed once may give a result of 340cells/ul, but if performed in duplicate using the same specimen may give results of 340 and 360. It may be important for the reference technology to be performed in duplicate and only when a concordant result is obtained around a threshold of 350 can it be used as the reference standard for the new test.

What constitutes an “acceptable” margin of error and misclassification probabilities around a threshold remain undefined and may vary among sites, depending on local factors such as the distribution of CD4 counts among asymptomatic patients, how often patients undergo repeat CD4 testing, and the implications of potential overtreatment (e.g., cost, long term risk of drug toxicity). However, given the move towards earlier treatment and the use of better-tolerated, less toxic drugs, misclassification that results in overtreatment may be more acceptable than would have previously been the case. It is relatively straightforward for national programmes to decide which technology best fits their needs based purely on cost and operating characteristics; it may be harder to decide what performance characteristics are acceptable, and harder still to obtain data on test performance to inform choice.

Given the potential for testing error, laboratory participation in EQA programmes and access to quality control (QC) reagents are essential. EQA information is not mentioned in the publications included in the review. Without this information, the proficiency of the laboratory staff performing the testing may have contributed to the errors and variation in addition to the assays themselves.

Conclusions

A wide range of bias and percent misclassification around treatment thresholds over the clinically relevant range were reported on the CD4 enumeration technologies included in this review. Less than half the studies reporting assay precision or reproducibility of the CD4 values obtained. This is a rapidly evolving field with new tests under development, and with existing instruments and reagents being regularly replaced by updated versions. A systematic review of POC tests compared to laboratory-based technologies showed that POC CD4 testing can increase retention in care prior to treatment initiation and can also reduce time to eligibility assessment resulting in more eligible patients being initiated on life-saving treatment [71]. The lack of standardized methodology on test evaluation, including consensus on reference standards, is a barrier to assessing relative assay performance and could hinder the introduction of new POC assays in countries where they are most needed.

Supporting InformationSystematic review protocol.

(DOC)

Click here for additional data file.

Summary of data extracted from eligible studies.

(XLS)

Click here for additional data file.

(DOC)

Click here for additional data file.

We thank Liam Whitby for his input into the study selection process and Maurine Murtagh, Ruth McNerney, Rhosyn Tuta and Chris Sanders for technical assistance.

ReferencesWorld Health Organization, UNAIDS, UNICEF (2013) Global update on HIV treatment 2013: results, impact and opportunities. Geneva: World Health Organization.GiorgiJV (1993) Characterization of T lymphocyte subset alterations by flow cytometry in HIV disease. Annals of the New York Academy of Sciences677: 126137 84942028494202SterlingTR, ChaissonRE, MooreRD (2001) HIV-1 RNA, CD4 T-lymphocytes, and clinical response to highly active antiretroviral therapy. Aids15: 22512257 1169869811698698Tayler-SmithK, ZachariahR, MassaquoiM, ManziM, PasulaniO, et al (2010) Unacceptable attrition among WHO stages 1 and 2 patients in a hospital-based setting in rural Malawi: can we retain such patients within the general health system?Transactions of the Royal Society of Tropical Medicine and Hygiene104: 313319 doi: 10.1016/j.trstmh.2010.01.0072013832320138323VitoriaM, VellaS, FordN (2013) Scaling up antiretroviral therapy in resource-limited settings: adapting guidance to meet the challenges. Current opinion in HIV and AIDS8: 1218 doi: 10.1097/COH.0b013e32835b81232318817923188179StevensW, GelmanR, GlencrossDK, ScottLE, CroweSM, et al (2008) Evaluating new CD4 enumeration technologies for resource-constrained countries. Nature reviews Microbiology6: S2938 22745957Centre for Reviews and Dissemination (2009) Systematic Reviews: CRD’s guidance for undertaking reviews in health care. York: University of York.BarnettD, WalkerB, LandayA, DennyTN (2008) CD4 immunophenotyping in HIV infection. Nature reviews Microbiology6: S715 doi: 10.1038/nrmicro199818923413BossuytPM, ReitsmaJB, BrunsDE, GatsonisCA, GlasziouPP, et al (2003) The STARD statement for reporting studies of diagnostic accuracy: explanation and elaboration. Clinical chemistry49: 718 1250795412507954LiberatiA, AltmanDG, TetzlaffJ, MulrowC, GotzschePC, et al (2009) The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: explanation and elaboration. PLoS medicine6: e1000100 doi: 10.1371/journal.pmed.10001001962107019621070MoherD, LiberatiA, TetzlaffJ, AltmanDG, GroupP (2009) Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. PLoS medicine6: e1000097 doi: 10.1371/journal.pmed.10000971962107219621072BalakrishnanP, DunneM, KumarasamyN, CroweS, SubbulakshmiG, et al (2004) An inexpensive, simple, and manual method of CD4 T-cell quantitation in HIV-infected individuals for use in developing countries. Journal of acquired immune deficiency syndromes36: 10061010 1524755215247552CarellaAV, MossMW, ProvostV, QuinnTC (1995) A manual bead assay for the determination of absolute CD4+ and CD8+ lymphocyte counts in human immunodeficiency virus-infected individuals. Clinical and diagnostic laboratory immunology2: 623625 85485448548544DidierJM, KazatchkineMD, DemouchyC, MoatC, DiagbougaS, et al (2001) Comparative assessment of five alternative methods for CD4+ T-lymphocyte enumeration for implementation in developing countries. Journal of acquired immune deficiency syndromes26: 193195 1124219011242190GernowA, LisseIM, BottigerB, ChristensenL, BrattegaardK (1995) Determination of CD4+ and CD8+ lymphocytes with the cytosphere assay: a comparative study with flow cytometry and the immunoalkaline phosphatase method. Clinical immunology and immunopathology76: 135141 76147327614732KarcherH, BohningD, DowningR, MashateS, HarmsG (2006) Comparison of two alternative methods for CD4+ T-cell determination (Coulter manual CD4 count and CyFlow) against standard dual platform flow cytometry in Uganda. Cytometry Part B, Clinical cytometry70: 163169 16498672LandayA, HoJL, HomD, RussellT, ZwernerR, et al (1993) A rapid manual method for CD4+ T-cell quantitation for use in developing countries. Aids7: 15651568 79044497904449LutwamaF, SerwaddaR, Mayanja-KizzaH, ShihabHM, RonaldA, et al (2008) Evaluation of Dynabeads and Cytospheres compared with flow cytometry to enumerate CD4+ T cells in HIV-infected Ugandans on antiretroviral therapy. Journal of acquired immune deficiency syndromes48: 297303 doi: 10.1097/QAI.0b013e31817bbc3a1854515418545154DiagbougaS, ChazallonC, KazatchkineMD, Van de PerreP, InwoleyA, et al (2003) Successful implementation of a low-cost method for enumerating CD4+ T lymphocytes in resource-limited settings: the ANRS 12–26 study. Aids17: 22012208 1452327714523277IdigbeEO, AuduRA, OparaugoCT, OnwujekweD, OnuboguCC, et al (2006) Comparison of Dynabeads and Capcellia methods with FACScount for the estimation of CD4 T-lymphocyte levels in HIV/AIDS patients in Lagos, Nigeria. East African medical journal83: 105111 1686300616863006ImadeGE, BadungB, PamS, AgbajiO, EgahD, et al (2005) Comparison of a new, affordable flow cytometric method and the manual magnetic bead technique for CD4 T-lymphocyte counting in a northern Nigerian setting. Clinical and diagnostic laboratory immunology12: 224227 1564301215643012LyamuyaEF, KagomaC, MbenaEC, UrassaWK, PallangyoK, et al (1996) Evaluation of the FACScount, TRAx CD4 and Dynabeads methods for CD4 lymphocyte determination. Journal of immunological methods195: 103112 88143258814325BalakrishnanP, SolomonS, MohanakrishnanJ, CeceliaAJ, KumarasamyN, et al (2006) A reliable and inexpensive EasyCD4 assay for monitoring HIV-infected individuals in resource-limited settings. Journal of acquired immune deficiency syndromes43: 2326 1688578016885780KandathilAJ, KannangaiR, DavidS, NithyanandamG, SolomonS, et al (2005) Comparison of Microcapillary Cytometry Technology and Flow Cytometry for CD4+ and CD8+ T-Cell Estimation. Clinical and diagnostic laboratory immunology12: 10061009 1608592016085920PattanapanyasatK, Phuang-NgernY, LerdwanaS, WasinrapeeP, SakulployN, et al (2007) Evaluation of a single-platform microcapillary flow cytometer for enumeration of absolute CD4+ T-lymphocyte counts in HIV-1 infected Thai patients. Cytometry Part B, Clinical cytometry72: 387396 17474130PattanapanyasatK, Phuang-NgernY, SukapiromK, LerdwanaS, ThepthaiC, et al (2008) Comparison of 5 flow cytometric immunophenotyping systems for absolute CD4+ T-lymphocyte counts in HIV-1-infected patients living in resource-limited settings. Journal of acquired immune deficiency syndromes49: 339347 1918634719186347RenaultCA, TraoreA, MachekanoRN, IsraelskiDM (2010) Validation of Microcapillary Flow Cytometry for Community-Based CD4+ T Lymphocyte Enumeration in Remote Burkina Faso. The open AIDS journal4: 171175 doi: 10.2174/18746136010040101712125346321253463SpacekLA, ShihabHM, LutwamaF, SummertonJ, MayanjaH, et al (2006) Evaluation of a low-cost method, the Guava EasyCD4 assay, to enumerate CD4-positive lymphocyte counts in HIV-infected patients in the United States and Uganda. Journal of acquired immune deficiency syndromes41: 607610 1665203416652034ThakarMR, KumarBK, MahajanBA, MehendaleSM, ParanjapeRS (2006) Comparison of capillary based microflurometric assay for CD4+ T cell count estimation with dual platform Flow cytometry. AIDS research and therapy3: 26 1704293617042936CassensU, GohdeW, KulingG, GroningA, SchlenkeP, et al (2004) Simplified volumetric flow cytometry allows feasible and accurate determination of CD4 T lymphocytes in immunodeficient patients worldwide. Antiviral therapy9: 395405 1525990215259902DieyeTN, VereeckenC, DialloAA, OndoaP, DiawPA, et al (2005) Absolute CD4 T-cell counting in resource-poor settings: direct volumetric measurements versus bead-based clinical flow cytometry instruments. Journal of acquired immune deficiency syndromes39: 3237 1585191115851911FrylandM, ChailletP, ZachariahR, BarnabaA, BonteL, et al (2006) The Partec CyFlow Counter could provide an option for CD4+ T-cell monitoring in the context of scaling-up antiretroviral treatment at the district level in Malawi. Transactions of the Royal Society of Tropical Medicine and Hygiene100: 980985 1654269016542690LynenL, TeavS, VereeckenC, De MunterP, AnS, et al (2006) Validation of primary CD4 gating as an affordable strategy for absolute CD4 counting in Cambodia. Journal of acquired immune deficiency syndromes43: 179185 1694085416940854ManasaJ, MusabaikeH, MasimirembwaC, BurkeE, LuthyR, et al (2007) Evaluation of the Partec flow cytometer against the BD FACSCalibur system for monitoring immune responses of human immunodeficiency virus-infected patients in Zimbabwe. Clinical and vaccine immunology: CVI14: 293298 1726759317267593PattanapanyasatK, ShainH, NoulsriE, LerdwanaS, ThepthaiC, et al (2005) A multicenter evaluation of the PanLeucogating method and the use of generic monoclonal antibody reagents for CD4 enumeration in HIV-infected patients in Thailand. Cytometry Part B, Clinical cytometry65: 2936 15800883ZijenahLS, KadzirangeG, MadzimeS, BorokM, MudiwaC, et al (2006) Affordable flow cytometry for enumeration of absolute CD4+ T-lymphocytes to identify subtype C HIV-1 infected adults requiring antiretroviral therapy (ART) and monitoring response to ART in a resource-limited setting. Journal of translational medicine4: 33 1690797316907973BergeronM, DingT, ElhartiE, OumzilH, SoucyN, et al (2010) Evaluation of a dry format reagent alternative for CD4 T-cell enumeration for the FACSCount system: a report on a Moroccan-Canadian study. Cytometry Part B, Clinical cytometry78: 188193 doi: 10.1002/cyto.b.2050119847883JohnsonD, HirschkornD, BuschMP (1995) Evaluation of four alternative methodologies for determination of absolute CD4+ lymphocyte counts. The National Heart, Lung, and Blood Institute Retrovirus Epidemiology Donor Study. Journal of acquired immune deficiency syndromes and human retrovirology: official publication of the International Retrovirology Association10: 522530.LopezA, CaragolI, CandeiasJ, VillamorN, EchanizP, et al (1999) Enumeration of CD4(+) T-cells in the peripheral blood of HIV-infected patients: an interlaboratory study of the FACSCount system. Cytometry38: 231237 1051660910516609NicholsonJK, VellecaWM, JubertS, GreenTA, BryanL (1994) Evaluation of alternative CD4 technologies for the enumeration of CD4 lymphocytes. Journal of immunological methods177: 4354 78228377822837SitoeN, LueckeE, TembeN, MataveleR, CumbaneV, et al (2011) Absolute and percent CD4+ T-cell enumeration by flow cytometry using capillary blood. Journal of immunological methods372: 16 doi: 10.1016/j.jim.2011.07.0082178777921787779StraussK, HannetI, EngelsS, ShibaA, WardDM, et al (1996) Performance evaluation of the FACSCount System: a dedicated system for clinical cellular analysis. Cytometry26: 5259 88094818809481HigginsJ, HillV, LauK, SimpsonV, RoayaeiJ, et al (2007) Evaluation of a single-platform technology for lymphocyte immunophenotyping. Clinical and vaccine immunology: CVI14: 13421348 1776152417761524JeganathanS, BansalM, SmithDE, GoldJ (2008) Comparison of different methodologies for CD4 estimation in a clinical setting. HIV medicine9: 192195 doi: 10.1111/j.1468-1293.2008.00544.x1836644218366442NicholsonJK, SteinD, MuiT, MackR, HubbardM, et al (1997) Evaluation of a method for counting absolute numbers of cells with a flow cytometer. Clinical and diagnostic laboratory immunology4: 309313 91443699144369Schnizlein-BickCT, SpritzlerJ, WilkeningCL, NicholsonJK, O’GormanMR (2000) Evaluation of TruCount absolute-count tubes for determining CD4 and CD8 cell numbers in human immunodeficiency virus-positive adults. Site Investigators and The NIAID DAIDS New Technologies Evaluation Group. Clinical and diagnostic laboratory immunology7: 336343 1079944310799443Reimann KAO, ’GormanMR, SpritzlerJ, WilkeningCL, SabathDE, et al (2000) Multisite comparison of CD4 and CD8 T-lymphocyte counting by single- versus multiple-platform methodologies: evaluation of Beckman Coulter flow-count fluorospheres and the tetraONE system. The NIAID DAIDS New Technologies Evaluation Group. Clinical and diagnostic laboratory immunology7: 344351 1079944410799444StorieI, SawleA, GoodfellowK, WhitbyL, GrangerV, et al (2004) Perfect count: a novel approach for the single platform enumeration of absolute CD4+ T-lymphocytes. Cytometry Part B, Clinical cytometry57: 4752 14696063StorieI, SawleA, WhitbyL, GoodfellowK, GrangerV, et al (2003) Flow rate calibration II: a clinical evaluation study using PanLeucoGating as a single-platform protocol. Cytometry Part B, Clinical cytometry55: 813 12949954DennyTN, GelmanR, BergeronM, LandayA, LamL, et al (2008) A North American multilaboratory study of CD4 counts using flow cytometric panLeukogating (PLG): a NIAID-DAIDS Immunology Quality Assessment Program Study. Cytometry Part B, Clinical cytometry74Suppl 1: S5264 doi: 10.1002/cyto.b.2041718351622GlencrossD, ScottLE, JaniIV, BarnettD, JanossyG (2002) CD45-assisted PanLeucogating for accurate, cost-effective dual-platform CD4+ T-cell enumeration. Cytometry50: 6977 1211634812116348PattanapanyasatK, LerdwanaS, NoulsriE, ChaowanachanT, WasinrapeeP, et al (2005) Evaluation of a new single-parameter volumetric flow cytometer (CyFlow(green)) for enumeration of absolute CD4+ T lymphocytes in human immunodeficiency virus type 1-infected Thai patients. Clinical and diagnostic laboratory immunology12: 14161424 1633906516339065Sippy-ChatraniN, MarshallS, BranchS, Carmichael-SimmonsK, LandisRC, et al (2008) Performance of the Panleucogating protocol for CD4+ T cell enumeration in an HIV dedicated laboratory facility in Barbados. Cytometry Part B, Clinical cytometry74Suppl 1: S6568 doi: 10.1002/cyto.b.2040618228556BriggsC, MachinS, MullerM, HaaseW, HofmannK, et al (2009) Measurement of CD4+ T cells in point-of-care settings with the Sysmex pocH-100i haematological analyser. International journal of laboratory hematology31: 169179 doi: 10.1111/j.1751-553X.2007.01017.x1817743418177434SrithanaviboonchaiK, RungruengthanakitK, NouanthongP, PataS, SirisanthanaT, et al (2008) Novel low-cost assay for the monitoring of CD4 counts in HIV-infected individuals. Journal of acquired immune deficiency syndromes47: 135139 1809104618091046JaniIV, SitoeNE, ChongoPL, AlfaiER, QuevedoJI, et al (2011) Accurate CD4 T-cell enumeration and antiretroviral drug toxicity monitoring in primary healthcare clinics using point-of-care testing. Aids25: 807812 doi: 10.1097/QAD.0b013e328344f4242137853521378535Mtapuri-ZinyoweraS, ChidemeM, MangwanyaD, MugurungiO, GudukeyaS, et al (2010) Evaluation of the PIMA point-of-care CD4 analyzer in VCT clinics in Zimbabwe. Journal of acquired immune deficiency syndromes55: 17 doi: 10.1097/QAI.0b013e3181e930712062267920622679SukapiromK, OnlamoonN, ThepthaiC, PolsrilaK, TassaneetrithepB, et al (2011) Performance evaluation of the Alere PIMA CD4 test for monitoring HIV-infected individuals in resource-constrained settings. Journal of acquired immune deficiency syndromes58: 141147 doi: 10.1097/QAI.0b013e31822866a22170956821709568DieyeTN, DiawPA, DaneauG, WadeD, Sylla NiangM, et al (2011) Evaluation of a flow cytometry method for CD4 T cell enumeration based on volumetric primary CD4 gating using thermoresistant reagents. Journal of immunological methods372: 713 doi: 10.1016/j.jim.2011.07.0122183518121835181BergeronM, DaneauG, DingT, SitoeNE, WestermanLE, et al (2012) Performance of the PointCare NOW system for CD4 counting in HIV patients based on five independent evaluations. PloS one7: e41166 doi: 10.1371/journal.pone.00411662291266822912668HerbertS, EdwardsS, CarrickG, CopasA, SandfordC, et al (2013) Evaluation of PIMA point-of-care CD4 testing in a large UK HIV service. Sexually transmitted infections88: 413417.22544309LoganC, GivensM, IvesJT, DelaneyM, LochheadMJ, et al (2013) Performance evaluation of the MBio Diagnostics point-of-care CD4 counter. Journal of immunological methods387: 107113 doi: 10.1016/j.jim.2012.10.0022306369023063690Mbopi-KeouFX, MionS, SagniaB, BelecL (2012) Validation of a single-platform, volumetric, CD45-assisted PanLeucogating Auto40 flow cytometer to determine the absolute number and percentages of CD4 T cells in resource-constrained settings using Cameroonian patients’ samples. Clinical and vaccine immunology: CVI19: 609615 doi: 10.1128/CVI.00023-122233629122336291Mbopi-KeouFX, SagniaB, NgogangJ, AngwafoFF 3rd, ColizziV, et al (2012) Validation of a single-platform, volumetric, flow cytometry for CD4 T cell count monitoring in therapeutic mobile unit. Journal of translational medicine10: 22 doi: 10.1186/1479-5876-10-222230999422309994FeiDT, PaxtonH, ChenAB (1993) Difficulties in precise quantitation of CD4+ T lymphocytes for clinical trials: a review. Biologicals: journal of the International Association of Biological Standardization21: 221231 79069487906948KorenrompEL, WilliamsBG, SchmidGP, DyeC (2009) Clinical prognostic value of RNA viral load and CD4 cell counts during untreated HIV-1 infection—a quantitative review. PloS one4: e5950 doi: 10.1371/journal.pone.00059501953632919536329RaboudJM, HaleyL, MontanerJS, MurphyC, JanuszewskaM, et al (1995) Quantification of the variation due to laboratory and physiologic sources in CD4 lymphocyte counts of clinically stable HIV-infected individuals. Journal of acquired immune deficiency syndromes and human retrovirology: official publication of the International Retrovirology Association10Suppl 2: S6773.World Health Organization (2013) Consolidated guidelines on the use of antiretroviral drugs for treating and preventing HIV infection Geneva World Health Organization.WilsonGA, BarnettD, ReillyJT (1995) Evaluation of the COULTER Manual CD4 Count Kit for the enumeration of absolute CD4 T-lymphocyte countsLondon, UK: Medical Devices Agency.WhitbyL, WhitbyA, FletcherM, HelbertM, ReillyJT, et al (2013) Comparison of methodological data measurement limits in CD4(+) T lymphocyte flow cytometric enumeration and their clinical impact on HIV management. Cytometry Part B, Clinical cytometry84: 248254 doi: 10.1002/cyto.b.2109423788473WynbergE, CookeG, ShroufiA, ReidSD, FordN (2014) Impact of point-of-care CD4 testing on linkage to HIV care: a systematic review. Journal of the International AIDS Society17: 18809 doi: 10.7448/IAS.17.1.188092444759524447595