Estimates of the incidence of youth concussion have recently increased and health care utilization patterns indicate that families are increasingly using primary care providers (PCPs) for both initial and follow-up concussion care.^1–7 The exact cause of the higher incidence is unclear, but may in part be due to heightened public awareness, state legislation requiring medical clearance before returning to play, and greater availability of timely appointments in primary care than specialty care.^8,9 Regardless of the cause, clinical management of the majority of youth concussion currently rests with PCPs.⁶

This scenario presents a challenge to PCPs, who may have insufficient time to systematically diagnose and manage concussion patients and may lack concussion-related continuing medical education.^10–13 Furthermore, PCPs have demonstrated difficulty in translating important concussion management concepts such as “cognitive rest” and “return-to-learn” into clinical practice.¹⁴ Insufficient time and training may limit adoption of best practices, including the implementation of emerging assessment techniques. This, in turn, can lead to over-referral to specialists, unnecessarily burdening the health care system.

To address challenges in PCPs’ management of concussion, we implemented a primary-care based intervention within the Children’s Hospital of Philadelphia’s (CHOP) large pediatric health care network with the primary goal of increasing CHOP PCPs’ (including nurse practitioners and physicians) adoption and systematic documentation of recommended best practice concussion management guidelines.¹⁵ The intervention included 2 key activities: (1) in-person training of PCPs that defined and recommended best practices for managing concussion and (2) development and integration of a primary care concussion-focused clinical decision support tool within CHOP’s existing electronic health record (EHR) system. Although such tools have challenges,^16–18 when integrated into an EHR they can provide valuable infrastructure in a busy primary care practice to systematically convert best practice management guidelines into clinical practice, facilitate the use of structured screening and diagnostic assessments, and enable systematic documentation across a broad health care network.^19–23 The primary objective of this study was to determine whether the intervention was effective in changing provider behavior surrounding 2 concussion management strategies: (1) performance of a vestibular oculomotor examination, an emerging technique in concussion assessment^24,25 and (2) provision of return-to-play (RTP) and return-to-learn (RTL) guidelines to patients at the time of care. We also sought to identify relevant patient-, visit-, and provider-level predictors of these behaviors in the post-intervention period. Finally, we determined whether the introduction of an EHR-based clinical decision support tool enabled providers to systematically and consistently document these 2 practices.

The CHOP primary care health care network includes over 30 locations in southeastern Pennsylvania and southern New Jersey, serves a socioeconomically and racially diverse patient population and accepts most insurance plans, including Medicaid. CHOP PCPs function as the pediatric medical home for their patients, managing all aspects of clinical care—including all initial visits, follow-up care and subspecialty referrals—using a linked EHR system (EpicCare, Epic Systems, Inc, Madison, WI).

The development of EHR-based clinical decision support tools represents a unique opportunity to provide clinical guidance to a geographically widely distributed network of providers and to change provider behavior by promoting systematic implementation and documentation of emerging recommendations and practices. This study evaluated the effectiveness of an EHR-based clinical decision support tool, coupled with in-person training, in facilitating adoption and systematic documentation of 2 youth concussion diagnosis and management strategies—the vestibular oculomotor examination and provision of RTL/RTP guidance—by a large health care network’s PCPs. Changing provider behavior around these practices is important in that it aligns with contemporary strategies of health care quality and process improvement including the use of structured screening and diagnostic assessments, the systematic and consistent documentation of care across a broad health care network, and the conversion of best practice management guidelines into clinical practice.

This effort was motivated by increased numbers of youth seeking concussion care from PCPs and the demonstrated need for additional PCP training and support.^6,10,12,13 Results highlight that in the 2 years leading up to the intervention, neither management strategy was being implemented consistently or systematically. On implementation of the intervention, however, performance of the vestibular oculomotor examination and documentation of discussion of RTL/RTP guidelines increased dramatically. Furthermore, for the vast majority of visits, the SmartSet template was used to document the oculomotor examination. Since only approximately 50% of the PCPs attended the training, this suggests that PCPs found value in the SmartSet template itself and that it had utility in guiding and/or structuring clinical practice and documentation.

Systematic reviews indicate that clinical decision support tools have successfully improved health care processes, but there have been fewer assessments of their impact on clinical, economic, and efficiency outcomes.^29–31 Tools integrated into EHRs have improved clinical management, including screening and diagnosis, of other pediatric conditions in the primary care setting^{20–23,32,33} as well as concussion in the emergency department setting.^34–36 Our study is the first to demonstrate improved provider-specific behaviors when caring for pediatric concussion patients in the primary care setting. This aligns with the “5 rights” of clinical decision support, by providing: (1) the right information (evidence-based guidelines for concussion assessment and management), (2) to the right person (PCPs), (3) in the right intervention format (SmartSet coupled with training), (4) through the right channel (EHR), (5) at the right time in the workflow (when caring for concussion patients).³⁷

Variation existed across patient-, visit-, and provider-level variables. Performance of the vestibular oculomotor examination and documentation of RTL/RTP was less common in visits with patients younger than 5 years, with male and Hispanic patients, during follow-up visits and when another injury was present. However, for all subgroups, when the examination was performed, it was done almost exclusively through the SmartSet template, suggesting that template use does encourage the provider to implement and document best practice for concussion management. The high frequency of performance of the examination and provision of guidelines on the first visit compared to follow-up visits further highlights the value of this systematic clinical decision support in helping the provider make the initial diagnosis. Administration of the vestibular oculomotor examination and provision of RTL/RTP guidelines was also 3 times more common among nurse practitioners than physicians. There is a much larger cohort of physicians in the network; it is possible that the intervention was able to more thoroughly penetrate through the nurse practitioner group versus the physician group. Collectively, these observations suggest a continued need to demonstrate value and encourage consistent use of the template for all concussion visits regardless of patient characteristics or visit type.

The intervention was less effective in changing PCP behavior when caring for children younger than 5 years. Children in this age group are not typically engaged in organized sports or formal schooling and thus there may be less of a perceived need to provide RTL/RTP guidelines. Additionally, concussion in the very young is challenging to assess as these patients cannot always describe their symptoms.³⁸ The SmartSet template and an objective test like the vestibular oculomotor examination might be particularly advantageous in this age group given that recognition and verbalization of symptoms is challenging. In a quality improvement survey that we conducted among our network’s PCPs in May 2015, respondents revealed concerns about the reliability and applicability of the SmartSet template questions for this age group (K. Arbogast, unpublished data). Age-specific improvements to the template and strategies to increase provider usage in this age group are needed, and may include instructions related to managing a patient’s return to developmentally appropriate activities, such as preschool and playground activities.

Historically, integrating new evidence into provider behavior can take many years.³⁹ However, in this study we demonstrated a rather robust and quick uptake of several recommended management strategies—in particular the vestibular oculomotor examination, which was a new concept to many of the network’s PCPs. We hypothesize several key characteristics that drove this success. First, this effort was driven by a clinical need. We observed that PCPs in our network did not already have an existing or systematic approach to concussion assessment and management; thus, the intervention appears to have filled a specific void. Second, our results corroborate the importance of engaging the clinical team in the design of clinical decision support tools.¹⁶ Prior to development, we broadly surveyed CHOP PCPs to understand their needs and actively engaged several PCPs in the design of the tool. Third, we believe that it was important to couple the clinical decision support tool with hands-on training, which is consistent with adult learning theory where experiential learning constitutes an essential component of acquiring new skills.⁴⁰

There are several limitations of this study. First, there were other secular changes associated with concussion, including legislation and evolving societal attitudes, which occurred during the study period and may influence the findings. For example, a proportion of the observed increase in RTL/RTP documentation among Pennsylvania practices may be attributable to concussion legislation, effective July 2012, that required an injured athlete to receive medical clearance before returning to sport.⁹ However, a similar law that went into effect in New Jersey in December 2010 did not seem to appreciably increase documentation in the months following in those CHOP practices located in New Jersey.⁴¹ Furthermore, the observed increase in performance of the vestibular oculomotor examination would not be expected to be influenced by these laws. Although these and other societal changes may influence the number of concussion visits seen by a given provider, they are likely not the primary driver of the provider behaviors on the visit-level observed herein. The independent contribution of each factor is difficult to determine in this retrospective analysis. Second, we studied providers who are part of a single large health care network. Our network uses a single EHR system, which promotes standardized practice similar to other large health care systems. As such, the behaviors of the providers and characteristics of the patients represented in this study likely reflect those in other network that utilize a single EHR to standardize practice. Third, we chose 2 specific key indicators of best practice concussion care; these are not the only aspects of concussion management but were chosen for this analysis because they represented emerging concepts as well as existing guidelines that were important to reinforce. Finally, this analysis relied on EHR documentation of both key practices. We did not evaluate the quality of the examination or the details of the RTL/RTP guideline discussion as these data are only available via direct observation. Additionally, an examination may have been performed despite lack of documentation in the EHR. However, clear and systematic EHR documentation of all aspects of clinical management is best practice and thus represents an appropriate target to strive for in this setting. It is important to note that our analysis evaluated providers’ performance and documentation of the vestibular oculomotor examination in the context of the SmartSet template for head injuries that were ultimately diagnosed as a concussion. We did not evaluate whether initiation of the SmartSet template increased the accuracy of the concussion diagnosis, the likelihood that a diagnosis would be made, or treatment decisions and clinical outcomes. These are complex questions to answer. Although concussion severity likely influences provider behavior, it is not a concept for which there is a consensus on how to measure. Similarly, clinical outcomes are not always clearly defined; for example, assessment of time to recovery may be hampered by lingering symptoms and lack of clear recovery indicators in existing electronic health record data. These important questions should be the focus of future work.

In summary, our findings suggest that an intervention utilizing an electronic clinical decision support tool, coupled with in-person training, can effectively and quickly change provider behaviors leading to the early adoption of existing and emerging guidelines for concussion management and consistent and systematic documentation of those practices. Primary care represents an ideal target for such interventions given the increasing rates of concussion related visits to these settings. Such tools can increase PCP proficiency in concussion assessment, accelerate uptake of emerging knowledge, and promote practice consistency throughout an entire health care network. Future efforts linking these provider behaviors to improvements in clinical outcomes are necessary advancements in this line of research.