Post-licensure vaccine safety surveillance is an essential component of any vaccination program and has important implications for immunization policy. Although pre-licensure human clinical trials evaluate vaccine safety, these usually have limited sample size, only assess otherwise healthy individuals. Therefore, rare adverse events (AEs), and AEs only occurring in a special sub-population may not be detected until a vaccine is widely used in the general population. For example, in 1999, post-marketing surveillance detected an excess risk of intussusception following the Rotashield^® vaccine, which led to the withdrawal of the vaccine from the market shortly after it was licensed and recommended for public use.¹

Current vaccine safety surveillance systems include both passive and active surveillance activities. The Vaccine Adverse Event Reporting System (VAERS) is a passive surveillance system which routinely provides analysis and data mining reports.^2–4 However, VAERS has important limitations, including under-reporting bias and incomplete information inherent to all passive reporting systems.^5,6 Active surveillance using rapid cycle analysis has also been regularly performed within the CDC’s Vaccine Safety Datalink (VSD) project to sequentially monitor the safety of new vaccines or the safety of established vaccines when changes are made to Advisory Committee on Immunization Practices recommendations for that vaccine.^7–9 Although rapid cycle analysis has some advantage for near real-time detection of an AE signal, the standard practice has involved pre-specifying a relatively small number of AEs in a fixed risk interval before beginning the surveillance; thus, potential AEs that are not pre-specified for screening or are outside of the predefined risk interval will not be detected.

The tree-temporal scan method was recently developed for vaccine and drug safety surveillance in situations where a wide range of potential AEs defined with different levels of specificity can be actively monitored simultaneously.^10,11 For example, using this method, we can find out whether a vaccine causes a very specific reaction such as febrile convulsion (International Classification of Diseases, Ninth Revision (ICD-9) code 780.31) or a broader group of related AEs such as convulsions (ICD-9 code 780.3). Pilot studies of the tree-temporal scan method have been conducted by the Food and Drug Administration Mini-Sentinel investigators to monitor potential AEs following HPV vaccines,¹² and they found 2 expected statistical signals (cellulitis and abscess of arm, other complication of surgical and medical procedures).

Meningococcal conjugate vaccine (Menactra^®, MENACWY-D) was approved for public use by Food and Drug Administration in January 2005. Subsequently, the Advisory Committee on Immunization Practices recommended that this vaccine be given routinely to 11 to 12-year-olds with a booster dose at age 16. In addition to local reactions, there have been reports of GBS occurring within 6 weeks after vaccination with MENACWY-D.¹³ One other study found an increased risk for Bell’s palsy in subjects receiving concomitant vaccines including MENACWY- CRM.¹⁴ No other potential AEs following Meningococcal conjugate vaccines have been identified.

The objective of our study was to conduct a data mining analysis to identify potential AEs following MENACWY-D using the tree-temporal scan statistic for pre-adolescents and adolescents in the VSD population and demonstrate the feasibility of this method in a large distributed safety data setting like VSD.

A total of 1 253 403 first doses of MENACWY-D were included in the analysis. Results for AEs following MENACWY-D are presented in Table 1. Out of 2 209 116 (4046 AEs × 546 risk intervals) AE-risk interval combinations, 38 were significant with P-values <0.05. Based on the MLCCS tree structure, those 38 significant outcomes belong to these broad groups: (1) disease of the nervous system and sense organs (Figure 2; branch 6); (2) disease of the skin and subcutaneous tissue (Figure 3; branch 12); (3) signs and ill-defined conditions and factors influencing health status (Figure 4; branch 17); (4) diseases of the respiratory system (Figure 5; branch 8); and others (including infectious and parasitic disease, diseases of digestive system, disorders of lipid metabolism, and other abnormal clinical findings). Among the 38 significant outcomes, a majority had a ratio of observed to expected (O/E) AE outcome occurrences between 1 and 3, except for unspecified erythematous condition (ICD-9 code 695.9) with an O/E ratio of 23.62, cellulitis, and abscess of upper arm and forearm (ICD-9 code 682.3 and its corresponding upper level MLCCS code 12.1.1.3) with an O/E ratio of 19.42, and other ill-defined and unknown causes of morbidity and mortality (ICD-9 code 799.8) with an O/E ratio of 8.69.

A subset of Table 1 results designated with a star sign (*) beside the MLCCS or ICD-9 code represents signals following MENACWY-D vaccination but not following well-care visits (without MENACWY-D vaccination). Only 10 outcomes were significant with P-values <0.05. According to the MLCCS hierarchical tree, those 10 outcomes belong to 1 of these categories: diseases of the skin and subcutaneous tissue (Figure 3; branch 12), signs and ill-defined conditions and factors influencing health status (Figure 4; branch 17), and diseases of respiratory system (Figure 5; branch 8). More specific descriptions of the signals in these 3 groups are listed below.

In this study, we demonstrated that the tree-temporal scan data mining technique can be successfully used in a large electronic database, such as the VSD, to monitor vaccine safety. We examined more than 1.2 million recipients of first dose of MENACWY-D and identified 3 groups of AE signals occurring within 42 days of MENACWY-D vaccination. The first 2 AE groups (“skin and subcutaneous tissue infections”, “fever and unspecified urticarial”) are expected outcomes following vaccination.¹⁷ For the third AE group which signaled (“pleurisy”), we hypothesized that there may be patients with these diagnostic codes who presented following a traumatic injury to the chest (eg, motor vehicle accident) or other pleurisy-causing conditions such as pneumonia. Therefore, we performed post-hoc exploration and searched for any diagnosis codes 60 days prior to the diagnosis of pleurisy. Among 48 cases, we identified 29 (60%) patients who had ICD-9 diagnosis codes for conditions that could cause pleurisy, including accident-related trauma, pneumonia, asthma, cardiac disease, bronchitis, abdominal injury, appendicitis, and surgery. It seems plausible that pleurisy is a false signal, considering that there is a lack of biological plausibility for pleurisy to result following this vaccination. However, it may need further investigation in future studies.

The present study has several strengths. First, we identified 2 major groups of signals, and they both were previously identified AE associations with MENACWY-D.¹⁷ The detection of previously observed associations provides support that the conditional tree-temporal scan method potentially can be reliably used to detect specific vaccine-AE associations. Second, whereas previous tree-temporal scan data mining studies only included inpatient data,¹⁸ we included data from inpatient, emergency department, and outpatient files, which improved study power. Third, the VSD distributed data model¹⁹ and the EMR (electronic medical records) database allowed us to extract detailed individual level data to follow up on detected associations. Importantly in the VSD, access to the patient’s complete EMR is available for review if we need to validate the patient’s diagnosis and to establish that there is an incident outcome. Fourth, we analyzed data for AEs following both vaccination and well-care visits without vaccination, allowing us to understand the baseline rate of the outcomes diagnosed following well-care visits without vaccination and eliminate them from consideration as true signals (eg, eye-related disorders/illnesses).

There are some limitations of this study. First, we only evaluated AEs following the first dose of MENACWY-D vaccine and during the first 6 weeks following vaccination. If there are adverse reactions that follow the booster dose(s) of MENACWY-D or manifest much later, we could not detect that. Second, this method lacks the capability to adjust for all time-varying confounders automatically.¹² However, we consider this data mining method is an approach to generate hypothesis(ses). Further investigation that takes time-varying confounders into account can be performed through rigorous traditional epidemiological studies. Third, patients with and without MENACWY-D vaccination after well-care visit may differ. If these differences are associated with AEs, it may be problematic to use patients without MENACWY-D vaccination as a control cohort for eliminating signals in the main analyses. Fourth, although our study shows the application of the tree temporal scan method as a hypothesis generating tool for MENACWY-D vaccine safety surveillance, applying this method to additional vaccines would provide strong evidence of the reliability of this method. Fifth, the rule we defined for determining incidence diagnosis (no previous diagnosis code on the third level within the past 183 days) may apply poorly to some AEs. Eg, if a patient had a fever 2 days following MENACWY-D, but also had a fever 5 months before this incidence, this AE of fever would be excluded from analysis because it was not considered as a new episode. Therefore, some truly incident cases were not captured using this rule. On the other hand, for certain chronic or long-lasting conditions, there is no guarantee that a prevalent condition will have been mentioned at a visit in the previous 6 months. A potential refinement of this design for the future studies is to tailor the incidence determination rules to each class of outcomes.

In summary, we identified 2 expected safety signals within 42 days of MENACWY-D vaccination using the tree-temporal scan data mining approach. We suspect that the third significant association was not a true signal. Overall, our findings provide reassurance that there is no evidence to suggest unexpected early onset AEs following MENACWY-D vaccine in the age group we studied. Because this data mining approach mainly serves for hypothesis generation, rigorous epidemiological studies may be needed for further validation of any signals. We also plan to develop and implement routine tree-temporal data mining applications for other vaccines in the VSD.