This study follows STrengthening the Reporting of OBservational studies in Epidemiology (STROBE) guidelines [12]. EHR data was extracted under a protocol approved by Institutional Review Board under a waiver of informed consent. Data were analyzed for 17040 adults ( ≥ age 18) who visited an outpatient neurology clinic (the Sleep Laboratory) at MGH from January 1^st, 2010 to March 31^st, 2021. These patients are generally referred by general practitioners but can also be referred by other doctors (e.g. a neurologist). Most are referred to evaluate possible sleep apnea. This cohort was selected because it is large and had a well-characterized set of neurologic and psychiatric outcomes from prior studies [13]. Patients with a first encounter less than 24 hours prior to outcome events were excluded. Deceased patients for whom date of death was not available were excluded.

Study outcomes included intracranial hemorrhage, ischemic stroke, depression, death and a composite of dementia, Alzheimer disease and MCI (“composite of dementia”). Outcomes and comorbidities were ascertained based on ICD codes and medications (Tables A.1, A.2), with additional information extracted from medical notes. Criteria to identify dementia and MCI were published previously [13]. Patients had to have at least one relevant ICD code to be assigned outcomes of intracranial hemorrhage or ischemic stroke. Patients had to have at least one relevant ICD code and one medication to be assigned outcomes of depression, diabetes or hypertension.

A patient diagnosed with any of the study outcomes, except death was assigned the corresponding comorbidity for all subsequent encounters. When predicting a given outcome (e.g. ischemic stroke), existing medical conditions were considered as possible predictors.

Additional covariates were outpatient vital signs and laboratory values, including systolic and diastolic blood pressure, temperature, heart rate and respiration rate; hemoglobin A1C, alanine transaminase (ALT), aspartate aminotransferase (AST), high-density lipoprotein (HDL), low-density lipoprotein (LDL) and albumin. We created binary indicators for absence of vital sign and laboratory values. Body mass index (BMI), age, sex and employment status were included as covariates. Employment was coded as “Active = 1” for professional/student full/part-time, self-employed, homemaker, or active military duty; and “Active = 0” for disabled, retired, not employed, or unknown status. Social covariates pertaining to marital, tobacco and alcohol status were included. Marital status was coded as “Union = 1” for those married/in civil union; “Union = 0” otherwise. Tobacco status was coded “Smoker = 1” with active smoking status, otherwise as 0. Alcohol was recorded as number of drinks per week.

Time-to event was calculated as time between the encounter and outcome, loss to follow-up or death. Encounters were assigned one of the following status: non-censored (‘1’ ) experienced the event of interest; (right) censored (‘0’) if lost to follow up or did not experience the event of interest during the study period; and, for competing risk analysis, patients who died without experiencing the event of interest were assigned a status of dead (‘2’). The common encounters for all study outcomes problems were used as training data for the missing data imputation task. Covariate measurements in this dataset were similar to those in the full dataset with all encounters (Fig. A.1).

Values outside physiological ranges were removed, followed by normalization (A.1. Data Preprocessing). Missing data was imputed using Multiple Imputation by Chained Equations (MICE) [14], where an estimator was developed for each covariate with missing data based on remaining covariates. Mean square error (MSE) was estimated on the training dataset using 5-fold cross validation. The estimator showing lowest MSE in the imputation task was selected for imputation of both train and test sets. The ExtraTreesRegressor [15] an estimator that fits a number of randomized decision trees on various subsamples of the dataset and uses averaging to improve the predictive accuracy, consistently achieved lowest MSE (Fig. A.2)¹.

Stratified random sampling was performed to create train (70%) and test (30%) sets. Patients were split into age strata by quantiles: 25% (18 to 47 years), 50% (48 to 60), 75% (61 to 71), and 99% (> 71 years) based on maximum age during the study. After obtaining train and test splits, outliers were removed, data was normalized [16], and missing values were imputed.

For model development, one random encounter was selected per patient. We trained a regularized Cox regression model with elastic net penalty [17] using 5-fold cross validation [18, 19] using Harrell’s C-index [20] to evaluate model performance. Cox regression [21] was used because it is an interpretable model, designed for predicting time to an outcome while handling right censoring in a natural way. The full training set was used to train a final model using the hyperparameter values that yielded best training performance. With the covariates selected by the elastic net, we trained a Cox proportional hazards model [21] with competing risk. For prediction of death, we applied the same methodology with no competing risk.

We assessed risk ratios (RR) for higher and lower risk groups, by adding and subtracting the standard deviation (SD) to all patients’ average risk probability, respectively, and dividing by average risk probability in the test set. RR of 1 signifies that two groups have the same risk, while results not equal to 1 indicate that one group is at more risk. Harrell’s C-index [20] was used as complementary performance metric, as well as the cumulative dynamic area under the receiver operating curve (AUC) [22–24], true positive rate (TPR) and false positive rate (FPR). We performed 1000 bootstrapping iterations with a random selection of patient encounters in the hold-out test (30%) set, which was not used to make decisions about model training/validation or hyperparameter tuning, to calculate 95% confidence intervals (CI). We plotted both empirical cumulative incidence risk curves (Aalen-Johansen estimator [25]), and risk curves from the Cox proportional hazards model. Finally, we evaluated covariate importance for each outcome model by assessing Cox models coefficients.

Our cohort comprised 17040 patients after applying inclusion and exclusion criteria (A.3. Supplementary results). Patients were predominantly male (57%), White (78%) non-smokers (87%) with average baseline age of 49 years (Table 1). The baseline average age for patients with depression (47 years) was below the cohort average, followed by diabetes, hypertension, ischemic stroke and intracranial hemorrhage, with 54, 56, 57 and 58 years, respectively. For patients with the composite of dementia and death outcomes, the average age at baseline was 64 years. the distributions of age at baseline according to positive or negative outcome is in Table A.3. Approximately equal numbers of patients were actively and non-actively employed (43% vs 40%). 57% were missing alcohol status; among those with data, 27% consumed alcohol with an average of 2.4 drinks per week (Table A.4). Most patients (63%) did not experience any study outcome events. Characteristics of training and test sets were comparable (Table A.5.).

We randomly selected one encounter per patient for train (N=11928) and test (N=5112) sets. Survival times for each outcome are shown in Fig. A.3. The modeling average concordance index and risk ratios for lower and higher risk groups are presented in Table 2. Yearly risk ratios in 95% CI are shown in Table A.6.

The average C-index in train and test sets were comparable and CIs were narrow in the 3% range for the unbalanced outcome events, indicating the absence of any significant overfitting (see Table 2). The difference between lower and higher risk patients was more accentuated for the composite outcome of dementia and death. The difference between groups was more attenuated for depression. With selection of balanced outcome events in test we observed wider CIs overall. Since the model was trained with an unbalanced distribution of the data, the model is less confident when evaluated in a balanced set of outcome events. To balance outcomes, the number of patients from each class had to be the same, and one random encounter was selected in each bootstrapping from each class. For each outcome (number of patients in test for positive class): intracranial hemorrhage (n = 35), composite of dementia (n = 251), ischemic stroke (n = 604), depression (n = 774) and death (n = 272). For intracranial hemorrhage we observed a decrease of 18% in C-index. There was 16% and 14% decrease in C-index for the composite of dementia and death, respectively.

The RR were similar between balanced and unbalanced events, except for death, where the difference between groups was more accentuated for balanced events, however coupled with wider CIs.

We trained and evaluated a baseline model with only age and sex as covariates, presented in Table A.7., where we observed a 3% decrease in C-index for intracranial hemorrhage, composite of dementia and death, a 2% decrease for depression, and a 1% decrease for ischemic stroke, in the testing set.

We also assessed the cumulative dynamic AUC in Table 3, with the competing risk coded as “no event”, since this dynamic AUC metric receives outcome events as binary. We performed 200 bootstrapping iterations to calculate 95% CI. We observed that the dynamic AUC exhibited slight oscillations in the range 2% to 5%, mean TPR from 2% to 4% and mean FPR from 2% to 4%, with the exception of the composite of dementia outcome, where there was an 8% increase of mean FPR from the 1^st to the 10^th year.

The cumulative risk for both the Cox proportional hazards and the Aalen-Johansen models is presented in Figs. 1 and 2 for patients in the test set. The cumulative incidence risk for patients in the lower and higher risk groups was well defined and separated for all outcomes. The empirical (Aalen-Johansen) risk curves were similar to those of the Cox proportional hazards model, and remained parallel, indicating acceptable fits and calibration of the Cox models.

We assessed the distribution of the cumulative incidence risk scores for each outcome (Fig. A.4). Patients who did not experience the outcome of interest tended to have overall lower risk scores compared to those who experienced the event or those who died.

Covariate importance for each study outcome is presented in Figs. 3 and 4 and Table A.8. With the exception of depression, age was a positive risk factor for all study outcomes.. Female sex conferred higher risk for the composite of dementia, ischemic stroke (p<0.1) and depression (p<0.001), while male sex conferred higher risk for intracranial hemorrhage (p<0.001). Being active (positive employment status) was a consistent predictor for decreased risk across all outcomes, except depression.

For intracranial hemorrhage, risk factors included higher age (p<0.01) and respiratory rate, and not being active (p<0.05).

For the composite of dementia, Alzheimer’s disease and MCI, risk factors included higher systolic blood pressure (p<0.001), A1C (p<0.001) and heart rate, older age (p<0.001), lower HDL (p<0.01) and ALT (p<0.001), and not being active (p<0.001).

For ischemic stroke, risk factors included older age (p<0.001) and higher A1C (p<0.05), not being active (p<0.001) and lower temperature (p<0.001).

For depression, risk factors included prior ischemic stroke, composite of dementia, Alzheimer’s disease, MCI, female sex, younger age, and non-union. All covariates showed statistical significance with a p<0.001.

For death, risk factors included older age (p<0.001), higher heart rate (p<0.001), ischemic stroke (p<0.001), hypertension (p<0.05), intracranial hemorrhage, dementia (p<0.05), diabetes, lower HDL (p<0.01) and inactivity (p<0.001).

We found that the brain health risk scores obtained from routinely acquired data and easily calculated from existing electronic health records data were associated with neurological outcomes: higher risk scores corresponded with higher risk of developing brain disease.

Age and active employment played a significant role in prediction of nearly all outcomes. Older age was related with higher risk for intracranial hemorrhage, ischemic stroke, composite outcome of dementia, Alzheimer disease and MCI and death in our cohort. This is in keeping with previous studies [26–28] analyzing the global burden of major neurological disorders, including stroke, Alzheimer’s disease and other dementias, which have shown that neurological disorders have been increasing in recent years, largely because of the aging of the population. Being employed usually requires learning new skills, maintaining a routine, and engaging socially, which may promote cognitive health, especially as one ages [29, 30]. Employment is also an activity where people are often faced with complex tasks. It is well established that engaging in complex activities in one’s environment may help prevent age-associated cognitive decline and dementia [31, 32], by facilitating brain health and optimal cognitive functioning [33].

The identification of these risk indicators is vital to alert the population of important factors that might help keep their brains healthy and fit. Patients can keep track of their progress at each subsequent encounter in modifying their lifestyle choices and risk of brain disease. The brain health scores developed in this work are similar to the American Heart Association’s (AHA) “Life’s Simple 7” [34] premise, where patients are encouraged to adjust their lifestyle choices so that they can monitor their brain health scores and thus improve their brain health over time. “Life’s Simple 7” [34] defines ideal cardiovascular health based on physical activity, healthy diet, smoking status, body mass index, total cholesterol, blood pressure and fasting blood glucose.

The brain health scores developed in this work can be calculated using the covariates which showed the most importance in the 11-year period survival prediction problems for each of the neurological outcomes. Those covariates were patients’ age and sex, employment status, marital status, vital signs, including respiration rate, systolic blood pressure, heart rate, and temperature, laboratory values, including A1C, ALT and HDL, body mass index, diabetes and hypertension. The majority of these measures have been robustly associated with cognitive decline in epidemiological studies [35].

Previous work has been done to create a brain care score, the McCance Brain Care Score (BCS) (see Fig. A.5) which can be calculated for an in-person encounter. The McCance BCS is based on a 17-point system where points are assigned to each of the measures in the physical, lifestyle and social and emotional categories, that sum up to give the total BCS for the patient. The McCance BCS was derived from evidence-supported interventions associated with reduced risk of brain diseases, building upon and expanding AHA “Life’s Simple 7” and recommendations from the Alzheimer Association. The score has been piloted by the Henry and Allison McCance Center for Brain Health and iteratively refined and improved in outpatient practice. Our model performance is not directly comparable to the McCance BCS, because the EHR does not have information on several of the variables included in the score, such as social relationships or meaning of life.

This study was performed for a cohort of patients who visited the Sleep Laboratory at MGH and may not be representative of other US and non-US populations, limiting the generalizability of the models across populations and hospital settings. The majority of patients were identified as White, and minorities are misrepresented on our dataset. This cohort was selected because it is large and had a well-characterized set of neurologic and psychiatric outcomes from prior studies by the authors [13], however there may be differences in insurance for patients who visit the Sleep Laboratory and those without insurance who visit other neurology clinics. This cohort may also have specific characteristics, such as low yearly incidence of hypertension, considering as reference the US population. Thus, we lack a study of generalizability to understand the importance of tailoring the models to different populations so that these may benefit from brain health scores assessment. Another limitation consisted of high missingness of laboratory values, which we tackled with a robust approach for missing data imputation. Also, our model only considers social covariates last registered in the system, thus a model that considers changes in social covariates each year should be considered in future studies.

A simple risk score derived from routinely collected data, easily acquired in a patient encounter, is associated with risk of neurological outcomes and death. This approach automatically uses EHR data, which makes it suitable for large-scale population screening. This enables the implementation of a very low-cost neurological screening tool for prevention of brain disease across healthcare systems. By adopting the approach of the chronic disease management model [36], the risk scores may empower especially younger to mid-life patients to make different lifestyle choices to improve their brain health and prepare for aging.