The acquisition and use of human test serum in this study were approved by the CDC Human Subjects Institutional Review Board (IRB). Sera from the CDC Anthrax Vaccine Research Program (AVRP) clinical trial participants and clinical trial site IRB approvals were obtained as described previously [1]. The preparation of the standard AVR801, positive quality controls (QCs) (AVR1749, AVR1750 and AVR1751) and negative QC (AVR811) used in the study have been described previously [2,8].

The method of quantitative anti-PA IgG ELISA has been described previously [2]. The main difference in the current assay is in the plate format: the reference standard is run in duplicate instead of in triplicate and test samples are run in a single well at one dilution instead of in duplicate and at 8 serial dilutions. The reagent control was removed to allow an additional dilution of the reference standard (see Fig. 1). Briefiy, the procedure consists is as follows: Immulon^® 2 HB microtiter plates (Thermo Labsystems, Franklin, MA) were coated with purified recombinant PA (2 μg/mL) (BEI Resources, Manassas, VA) in 0.01 M phosphate buffered saline (PBS) pH 7.4 (Life Technologies, Gaithersburg, MD) and incubated overnight (16–24 h) at 2–8 °C. Plates were washed three times with PBS containing 0.1% Tween-20, pH 7.4 (ELISA wash buffer). Master Plate Diluent (PBS containing 5% skim milk and 0.5% Tween-20, pH 7.4) was added to the first two columns of the plate (100 μl per well). The standard reference serum AVR801 was diluted 1:25 in Master Plate Diluent and 100 μl was loaded into the first two wells of the plate and serially transferred in 2-fold dilutions down the plate to make an 8-point dilution series. The test sera were diluted 1/50 in Master Plate Diluent and 100 μl of each test serum was loaded into 72 wells of the plate (Fig. 1). The last column on the plate was designated for positive and negative QC samples. Plates were incubated for 60 ± 5 min at 37°±2 °C and washed three times with ELISA wash buffer. Horseradish peroxidase conjugated mouse monoclonal anti-human IgG Fc PAN clone HP6043 (lot #061010 at 1:27,000 dilution, Hybridoma Reagent Laboratory, Baldwin, MD) was added to all wells (100 μl/well) and incubated at 37°±2 °C for 60 ± 5 min. Plates were washed three times and 100 μl of ABTS Microwell Peroxidase Substrate System (Kirkegaard and Perry Laboratories, Gaithersburg, MD) was added to all wells. After 30 ± 5 min incubation at 37°±2 °C 100 μl of ABTS Peroxidase Stop Solution (Kirkegaard and Perry Laboratories, Gaithersburg, MD) was added to each well and plates were read within 30 min using the BioTek spectrophotometer at a wavelength of 405 nm with a 490 nm reference utilizing the Gen5 software (Bio-Tek Instruments, Inc., Winooski, VT). Assay endpoints were reported as concentrations (μg/mL) of anti-PA IgG for quality control sera and test samples using the ELISA for SAS program (ELISA HT 3.sas) version 9.3. The reference standard is fit to a 4-parameter logistic (4-PL) curve, and sample concentrations are calculated by interpolating to the reference standard, which has a pre-established concentration of 109.4 μg/mL. Samples with ODs below the lowest standard OD are reported as 0, and samples with ODs above the highest reference standard OD are reported as >109.4. High samples will be retested in serial dilution in the original assay format if quantification of high results is required. All values for validation parameters were obtained from at least three experiments performed by four operators on non-consecutive days.

A set of five acceptance criteria for assay performance in the high throughput screening format were similar to the acceptance criteria for the CDC validated anti-PA IgG ELISA [2,9]. These five assay acceptance criteria were: 1) the mean Optical Density (OD) value of the negative control was required to be less than 0.200 OD units; 2) the standard reference serum was required to have a weighted coefficient of determination (r²) value of ≥0.990 to the 4-Parameter Logistic (4-PL) model; 3) the mean anti-PA IgG concentration for each of three positive quality control sera were required to have coefficients of variation (CV) <20%; 4) at least 2 of 3 positive control sera were required to have anti-PA IgG concentrations within 2 standard deviations (SD) of their expected values and 5) no positive control sera anti-PA IgG concentrations were allowed to be >3SD from the expected value. All five acceptance criteria were required to be met, otherwise all test samples on the plate were rejected and sample testing repeated. During normal testing, all serum specimens are required to have at least two passing results generated by two independent operators and expressed as a mean of those two passing results. Inter-operator precision is required to have a %CV of ≤30%. Samples with >30% CV between operators would be repeated up to a total of 4 times, and the median result would be reported. Note that during validation, samples were tested up to 12 times for accuracy, precision, lower limits of detection and quantification and dilutional linearity of the assay. Concordance with the historical ELISA, diagnostic sensitivity (DSN) and diagnostic specificity (DSP) were tested according to the normal 2 operator test method described here.

Validation was done in accordance with the Food and Drug Administration guidance [10]. Acceptance criteria were derived from the assay development data (Table 1), and are consistent with the criteria used for the original assay. The following validation parameters were tested during validation experiments: accuracy, precision, goodness of fit, Lower Limit of Detection (LLOD), Lower Limit of Quantification (LLOQ), dilutional linearity, and range.

Serum samples from AVA vaccinated volunteers and control samples (n = 320) were used for calculation of DSP and DSN on sample and on patient levels. At least two experiments were performed by two different operators. A panel of sera from human adult volunteers from the CDC Anthrax Vaccine Research Program (AVRP) clinical trial were chosen to simulate positive and negative samples and patients. Samples from week 0 (pre-vaccination) were chosen to simulate an acute sample, and samples from week 8 were chosen to simulate convalescent samples. There were 80 participants from the control (unvaccinated) group to simulate true negative patients (expected to have no anti-PA IgG at either week 0 or week 8), and 80 participants from the treatment (vaccinated) group to simulate true positive patients (expected to have no anti-PA IgG at week 0, but detectable anti-PA IgG at week 8 after vaccination). At the sample level, all samples from the control group and the week 0 samples from treatment group were defined as negative, and the week 8 samples from the treatment group were defined as positive. At the patient level, the control group were defined as negative and treatment group were defined as positive.

A sample is considered positive if its concentration is greater than or equal to the reactivity threshold (RT). A range of reactivity threshold values were evaluated from 1.7 μg/mL to 13.4 μg/mL (the highest concentration of anti-PA IgG in a false positive sample). The historical RT was 3.7 μg/mL [2]. Diagnostic Specificity (DSP) measures the ability of the assay to identify a true negative result for a serum sample DSP = TN/(TN + FP) × 100% and Diagnostic Sensitivity (DSN) measures the ability of the assay to identify a true positive result for a serum sample DSN = TP/(TP + FN) × 100%. TN = true negative, negative result on a negative sample. FP = false positive, positive result on a negative sample. TP = true positive, positive result on a positive sample. FN = false negative, negative result on a positive sample.

Since reactive samples can be caused by vaccination as well as infection, patients are diagnosed as positive if there is a 4-fold or greater increase in concentration between the acute and convalescent samples [11]. Thus, DSP and DSN at the patient level measure the ability of the assay to identify a true negative result and a true positive result for a patient using the fold change over time. Concentrations below the LLOQ (2.6 μg/mL) were replaced by ½ LLOQ (1.3 μg/mL) for purposes of calculating fold increase. The starting dilution of 1/50 of the reference standard serum AVR801 allows detection of a maximum concentration of 109.4 μg/mL. Therefore the values above 109.4 μg/mL were replaced with 110.0 μg/mL as the next whole number above 109.4. In cases where quantification of concentration above 110 μg/mL would be required to determine if the fold increase is >4 (i.e. for patients who had >27.4 μg/mL in their acute sample), high concentration samples can be retested either at higher starting dilution in the HT format or in dilution series in the original assay format. No such samples were observed in this sample set.

Comparative analysis of anti-PA IgG concentrations (μg/mL) obtained by the High Throughput anti-PA IgG by Enzyme-Linked Immunosorbent Assay (ELISA) for Screening of Human Sera as an Emergency Response to an Anthrax Incident and the validated CDC ELISA for Detection of B. anthracis PA-Specific IgG in Human Sera [2] was performed using Deming regression [12]. Selected samples consisted of a panel of 95 human sera from AVA vaccinated volunteers. The samples encompassed a range of anti-PA IgG concentrations from the non-vaccinated volunteers (n = 19, week 0) and vaccinated volunteers (n = 76, week 8). Samples with concentration of anti-PA IgG < LLOD and >109.4 μg/mL were omitted from the analyses. The paired sets of data for each sample were evaluated by determining the Concordance Correlation Coefficient (CCC) and goodness of fit r² value was calculated from the Deming regression of the log₁₀ transformed median anti-PA IgG concentrations of AVR801. A CCC value of ≥0.95 was considered an acceptable level of agreement between the data sets. Median concentration of anti-PA IgG concentration obtained by the original CDC validated ELISA was plotted on the X axis and median anti-PA IgG concentration by the High Throughput anti-PA IgG ELISA (HTE) is plotted on the Y-axis. The accuracy component of the CCC is calculated from the distances of the Deming regression best-fit line from the line of unity, where slope = 1 and intercept = 0. Concentrations below LLOD (0.85 μg/mL) could neither be measured nor replaced by 0.0 μg/mL because a non-zero value was required to calculate the ratio of fold-increase. Therefore, all such values were replaced with 0.425 μg/mL (1/2 of LLOD) rounded up to 0.43. The values above 109.4 μg/mL were replaced with 110.0 μg/mL as the next whole number above 109.4. Samples with results < LLOD and >109.4 μg/mL in both sets were omitted from the analyses. Graphs of the Deming regression and the CCC calculations were done with SAS^® (SAS Institute Inc., Cary, NC). At least two experiments were performed by two different operators over nonconsecutive days.

The raw data were imported into and analyzed by an ELISA analysis program ELISA for SAS (ELISA HT 3.sas) generated in the laboratory for calculation of the antibody concentration for standard, positive and negative quality control sera, and the validation test samples. Mean anti-PA IgG concentration, standard deviation and coefficient of variation (% CV) s were used to calculate the validation parameters except CCC analysis where the median anti-PA IgG concentration was used. All calculations for regression analyses, DSN/DSP analysis and CCC analysis were performed with log₁₀ transformed data using SAS^® (SAS Institute Inc., Cary, NC) version 9.3. The calculation procedure for each validation parameter is described in its respective designated section.

The mean r² of the standard’s ODs were fitted to a 4-PL model by SAS program ELISA HT 3.sas. The standard’s data were obtained from all validation experiments performed by four operators. The mean r² was 0.999 with the range from 0.995 to 1.00. The assay demonstrated high accuracy and precision for this assay. The %E ranged from −4.6% to 14.4% and inter-assay (intermediate) precision expressed as %CV of the reportable value ranged from 9.0% to 21.2%. Each validation sample had a ≤25% E and ≤25% CV. These data are indicative of a high level of accuracy and reproducibility of this assay between four independent operators and met the predetermined assay acceptance criteria (Tables 1 and 2).

LLOD of the standard AVR801 has been determined as 1.7 μg/mL [2] at the 1/1600 dilution. The current format of the plate allowed us to use an additional 1/6400 dilution of the standard, so concentrations 0.85 μg/mL was accepted as LLOD of the High Throughput Screening assay. Similarly, the values above the concentration of the standard 109.4 μg/mL could not be measured and were replaced with 110.0 μg/mL (110 is the next whole number after 109.4, used as a marker to indicate >109.4). If quantitation is necessary for high samples they can be repeated at higher starting dilution or run in the multi-dilution original assay format.

LLOQ was determined experimentally by testing 14 serum samples. The sample AVR2253 had the lowest expected concentration 2.6 μg/mL that met the validation acceptance criteria and therefore is LLOQ for this assay (≤50% error and CV ≤ 25% for each validation sample) (Tables 1 and 2).

The log₁₀ transformed concentrations of anti-PA IgG (μg/mL) for 23 validation serum samples with the range of anti-PA IgG concentrations 0.3–72.9 μg/mL were used for determination of dilutional linearity. The log₁₀ transformed observed concentrations were plotted on the Y-axis and the log₁₀ transformed expected mean value of those serum samples on the X-axis. A line of best fitness was generated to represent the trend of the data points and the slope and intercept of this line was calculated. Observations of 0 were masked as ½ LLOD (0.43 μg/mL). Regression analysis showed the fit of the data with r² ≥ 0.952, slope = 1.029 and intercept = −0.03 that met the acceptance criteria (r² ≥ 0.850 with a slope between 0.9 and 1.1 and an intercept between −0.1 and 0.1) (Table 1, Fig. 2).

The assay range was calculated as the interval of anti-PA IgG concentrations that can be interpolated from the standard curve with acceptable accuracy, precision and linearity. The lower limit of the range was established at LLOQ = 2.6 μg and the upper limit was established at the maximum detectable concentration 109.4 μg/mL.

DSP and DSN on a sample level measures the ability of the assay to identify a true negative and a true positive result for a serum sample, respectively. A sample is considered positive if its concentration is greater than or equal to the reactivity threshold of 3.7 μg/mL [2]. DSP and DSN was calculated for a range of values from 1.7 μl/mL (the LLOD for the Standard Reference serum AVR801) to 13.4 μg/mL (the highest concentration of anti-PA IgG in a false positive sample).

Analysis of serum anti-PA IgG responses from unvaccinated volunteers at week 0 indicated 236 non-reactive samples (True Negative) and four reactive sera (False Positive). Based on these data the assay DSP on a sample level was 98.3%. While increasing the RT above the historical levels could increase the DSP in this set, the historical level of 3.7 μg/mL was retained to maintain high sensitivity. Analysis of serum anti-PA IgG responses of vaccinated volunteers indicated 100% DSN (80 of 80 sera > LLOQ) (Fig. 3).

DSP and DSN at the patient level measures the ability of the assay to identify a true negative and a true positive result for a patient, respectively. A patient is diagnosed as positive if there is a 4-fold or greater increase in concentration between the simulated acute (non-vaccinated volunteers, time point 0) and simulated convalescent samples (vaccinated volunteers) [11]. All 80sera pairs from vaccinated volunteers indicated a 4-fold or greater increase in concentration between simulated acute and simulated convalescent samples (100% DSN). There were also no False Positive patients (100% DSP).

Comparative analysis of anti-PA IgG concentrations (μg/mL) for 95 serum samples obtained by two assays showed that CCC = 0.974. A CCC value of ≥0.95 was considered an acceptable level of agreement between the data sets [12]. The measure of accuracy was 0.997 and measure of precision was 0.977 (Fig. 4).