Diagnosis of obstructive sleep apnea (OSA) is increasingly being made with limited channel monitoring (level III and IV devices) in both clinical and epidemiological settings. However, the devices being used vary from single channel oximeters to devices that monitor multiple signals such as airflow, breathing efforts, oxygen saturation and surrogates for arousal. When disease is moderate to severe, both Level IV and Level III devices have been shown to have value in screening as well as in establishing the diagnosis of OSA [1, 2]. The American Academy of Sleep Medicine (AASM) has suggested only level III devices should be used to obtain OSA severity indices comparable to that obtained with full polysomnography (PSG) [3–7]. However as noted by the recent AASM guidelines document for diagnostic testing for OSA [8], there are limited data in the literature assessing the impact of the number of parameters or technologies being used, particularly in different clinical settings. We were interested in assessing the impact of using maximally different diagnostic devices on derivation of indices of OSA severity and the agreement between devices for OSA diagnosis.

One potentially important difference between devices is the technology used for oximetry as well as the measurement site which may lead to true physiologic differences in oxygen saturation. Pulse oximetry can be measured using either transmission or reflectance techniques. Transmission oximetry detects light after it passes through relatively translucent placement sites, such as the fingertips or earlobes, whereas reflectance oximetry detects reflected light from tissue that is opaque (eg. forehead sensors). There are known differences in the measurement of oxygen saturation (O2Sat) using these two methods [9], but, even by a single technique, we and others have shown a significant effect of pulse oximeter brand and sample averaging duration on OSA diagnosis [10–12]. No study has specifically addressed the impact on OSA diagnosis of using reflectance vs transmission oximetry technique with currently used devices.

Additionally, identifying sleep related respiratory events (apneas /hypopneas) using only their consequences (eg O2 desaturation) may differ from detecting events as by reduction in airflow and assessing their significance from the oximetry signal. Finally, some devices also use surrogates for EEG arousal and use these to maximize similarity to polysomnography detection of hypopnea. Given all these differences the purpose of the present study was to compare the effect on diagnosis of OSA using two devices that are maximally different in their approach to measure OSA severity. The present study compares data from a widely used stand-alone level IV device the Nonin WristOx₂™ (Model 3150, Plymouth, MN) and a level III device , the Apnea Risk Evaluation System (ARES™, Watermark Medical, Boca Raton, FL) that uses reflectance oximetry combined with other signals (Fig 1). We have previously shown that the ARES™ has good agreement and adequate sensitivity and specificity with in-laboratory polysomnography for diagnosis of OSA [4]. For the present study, we compared these devices in a non-sleep clinic population which, in contrast to sleep clinic populations, does not have the high pre-test probability of OSA. Specifically, we compared (i) the amount of data loss with each device and type of oximetry; (ii) baseline awake O2 saturation and %time below 90% O2 saturation and (iii) agreement between the values of OSA severity, and agreement for diagnosis of OSA from these devices, compared to the physiologic night-to-night variability of these metrics. We hypothesized that there would be significant differences in sleep disordered breathing (SDB) indices and diagnostic agreement for OSA when using finger pulse oximetry alone versus a device that measures multiple parameters including airflow, forehead oxygen saturation and surrogates for the EEG arousal.

Data from the first 178 subjects who enrolled in an ongoing study in the World Trade Center Responder population (Clinical trials # NCT01753999) were analyzed. The parent study collects exclusively ambulatory overnight home studies to evaluate the relationship between new onset OSA following the WTC disaster on 9/11/2001 and nasal pathology. Subjects were recruited without regard to OSA symptoms, but were not eligible if prior to 9/11/2001 they had documented evidence of OSA or significant snoring or if they were currently on treatment for OSA. For the present substudy, subjects were instructed to wear the WristOx₂™, and ARES™ simultaneously for 2 consecutive nights. Both devices were initialized on the same computer to synchronize the internal clocks. Devices were given to the patients during an in person visit. A research coordinator provided verbal instructions that took <5 minutes and a 1 page pamphlet with written instructions was also provided to the subjects. No additional interaction with the patient was required. Both devices are easy to self-apply and we have used the ARES™ successfully with just the written instructions in previous studies [4]. The ARES™ measures oximetry on the forehead, and the subjects wore the WristOx₂™ on whichever hand they were more comfortable. Data were excluded from analysis if both devices were not used on the same night or if the duration of recording on either device was less than 2 hours (tabulated as data loss).

WristOx₂™: Oximetry by transmission was sampled at 1 Hz with an averaging time of 4 beats. Automated analysis of the WristOx₂ data, using Nonin nVision data management software version 6.3, provided an index of OSA severity, the oxygen desaturation index (ODI_ox2) and % time below 90% O2 saturation (%time<90%O2Sat) for each night. ODI_Ox2 was defined as the number of drops in saturation by at least 4% lasting a minimum of 10 seconds per hour of valid recording time as defined by the Nonin software. We inferred the awake baseline O2 saturation from a period at the beginning of the study within the first 10 minutes from the start of the recording. Poor signal quality was identified and excluded in the oximetry data using the automated algorithm on the WristOx₂. The signal was manually inspected to ensure validity of the algorithm and no further editing was required.

ARES™: Oximetry by reflectance was sampled at 100 Hz with an averaging time of 3–5 beats depending on signal quality, and displayed at 1 Hz. Automated analysis of SDB events was followed by manual inspection and editing of events by investigator as per device use instructions. ODI is not provided by the ARES™ automated algorithm, so a direct comparison of transmittance and reflectance saturation derived SDB was not possible in our study. The closest measure to ODI provided by the ARES™ is the AHI4 which counts apneas and those hypopneas with a ≥4% desaturation. The Apnea Hypopnea Index 4% from ARES™ (AHI4_ARES) is calculated as the sum of apneas and hypopneas 4% divided by total sleep time (TST); apneas were defined as a reduction in flow amplitude of >90% of baseline, hypopneas 4% were defined as a reduction in flow amplitude >30% followed by ≥4% oxygen desaturation, or a visible reduction in flow amplitude along with a change in shape suggesting inspiratory flow limitation (IFL) followed by ≥4% oxygen desaturation. The ARES™ provides an estimate of total sleep time obtained from a combination of actigraphy and automated analysis of single channel forehead EEG recording [13].

The ARES™ also provides a Respiratory Disturbance Index (RDI_ARES): sum of apneas, hypopneas 4% and hypopneaArousal divided by TST; hypopneaArousal was defined by a visible reduction (usually >30%) in flow amplitude along with a change in shape suggesting inspiratory flow limitation and followed by arousal surrogates that included an abrupt change in head position or an increase in flow amplitude to >2 times the amplitude during the event along with a normalization of shape.

For ARES™oximetry data, automated algorithms were first applied to exclude areas with poor quality oximetry. This was followed by additional manual editing to exclude areas with sustained drops in O2 saturation following body position changes but not associated with SDB events.

For normally distributed variables data are presented as mean±SD and groups compared using paired t-test. Data are presented as median (IQR) and Wilcoxon Rank Sum tests were used to compare groups when data was not normally distributed. Concordance of SDB indices was assessed by performing Intraclass Correlations (ICC) between ODI_Ox2 and AHI4_ARES, and both devices’ baseline O2 saturation level and %time<90%O2Sat [14]. Bland Altman plots were used to measure bias and differences between these measures and Pearson correlation coefficients ( r) were used to assess relationships between other related variables [15]. Agreement, sensitivity and specificity for diagnosis of OSA were examined using standard cutoffs for OSA (i.e.≥5/hr,>15/hr). Analyses were performed between devices using all nights in all subjects and between pairs of data for comparisons across nights using the same device.

351 nights of data were analyzed from 178 subjects (85.3% Male, 14.6% Female, BMI=30.3±5.6 kg/m2, age=52.5±8.9 years, Epworth Sleepiness Scale (ESS)=8.5±5, 37% with ESS>10). The reported prevalence of congestive heart failure= 2.3%; hypertension =25%; stroke=0.6%; MI=2%; diabetes=10%; gastro-esophageal reflux disease=40%, chronic-rhinosinusitis=42% and mental-health conditions (depression, post traumatic stress disorder, panic disorder)=15%. 8% were current smokers. The prevalence of OSA in this dataset was 28% using a cutoff AHI4≥15/hr (data from multiple nights collected for each subject combined) and 62% for AHI4≥5/hr. The prevalence values were 10% and 22% when the coexistence of excessive daytime somnolence (ESS>10) was included in the definition of OSA.

Majority of the subjects (73%) used both devices simultaneously for 2 nights, 4% for three nights and the remaining 23% of subjects used both devices for only one night. 11.7% (n=41/351 nights) of data were excluded for having <2hrs of data on either device (10.8% Wrist Ox₂; 2.3% ARES™) leaving 310 nights with simultaneous ARES™ and Wrist Ox₂ data, and 130 studies with 2 nights of data with each of the devices.

This is the first study to our knowledge that examines the differences in SDB metrics obtained from divergent home monitoring strategies, using different types of oximetry and contrasting a Level III forehead device and a Level IV finger device. As both the ARES™ and WristOx₂™ devices are widely used our results have implications for comparing clinical and epidemiological datasets. In the simultaneously recorded home data of this large dataset of non-sleep clinic subjects, there were significant differences in the baseline awake O2Sat and %time<90%O2Sat obtained from the two devices. Despite this, the correlation between ODI_Ox2 and AHI4_ARES was high. Agreement and rate of false negatives identified for diagnosis of OSA were similar to those seen due to night-to-night variability in SDB.

We and others have compared limited channel monitoring devices (level III and IV) and shown good sensitivity (80–89%) and specificity (86–94%) for diagnosis of OSA compared to full laboratory PSG [1, 2, 20–23]. A recent study compared a level III device (ApneaLink Plus, which uses transmittance finger pulse oximetry) and a stand-alone finger pulse oximeter (Pulsox 300i) used at home against an in-lab NPSG in a sleep clinic population with suspected OSA [24]. This report showed good agreement between indices of OSA that required ≥4% oxygen desaturation, calculated from either a single pulse oximetry or multi- channel PSG recording. We have also previously found that significant differences occur in AHI4 obtained when different brands of finger pulse oximeters were used in the same patient [10]: the biases found in AHI4 in that study ranged from 0.3±1.7/hr to 7.1±9.6/hr and were in the same order of magnitude for biases in AHI4/ODI (0.7±6.1) seen in the present study between finger transmission and forehead reflectance oximetry. These data suggest differences in measurement may be device dependent in addition to varying with the method of oximetry used [10].

In contrast to the results for the SDB index, we found significant and systematic device-dependent differences in the percent time<90%O2Sat. As neither device is the gold standard for oxygen saturation and we did not obtain a blood gas, it is impossible to know which of the two measures better reflected actual O2 saturation levels. We believe a significant contributor to the systematic difference in oximetry observed in our study is the oximeter type. However, differences in averaging or sampling rates may have contributed [10]. It is also likely that calibration of the oximetry signal will account for differences in percent time below 90%; in support of this, our data showed differences in the simultaneously measured baseline awake saturation across the two devices. In addition, physiologic O2 saturation may be different at the forehead and finger as temperature and blood flow have been shown to influence O2 saturation and may differ [25]. The Wrist Ox₂™ O2 saturation measurement at the beginning of the night was lower than the ARES device, consistent with the overall greater percent time<90%O2Sat reported by the Wrist Ox₂. In addition, the manual editing of the ARES data may have contributed by being more aggressive during periods of low O2 saturation/poor signal quality. This likely will result in removal of more periods of low O2 saturation (“poor signal”) with the ARES algorithm than with the fully automated Wrist Ox₂ algorithm. Irrespective of the cause, the significant observed differences in reported %time<90%O2Sat suggest care must be taken whenever comparing these data across studies using portable monitors with different oximeter types or algorithms.

Although data loss from oximetry was <10% for both devices, the ARES™ device had significantly greater data loss than the Wrist Ox₂. This may be due to a lower signal to noise ratio in forehead reflectance oximetry compared to transmission. Review of oximetry tracings suggests that the forehead signal is particularly susceptible to data loss during movement (as during position changes) possibly due to positional changes in venous blood flow to the forehead. On the other hand, our data showed that in the WristOx₂ studies we had more nights with insufficient duration of data recorded for analysis (nights with <2 hours) than in the ARES™ studies. Possible reasons include the more precarious placement of the Wrist Ox₂ on the fingertip, allowing for the device to fall off the patient more easily than with the ARES™, which is secured around the forehead by an elastic strap and a nasal cannula.

The impact of the ability to score any SDB events without 4% O2 desaturation (e.g. those with arousal surrogates) using the ARES™ is reflected in the additional 20 (6.4%) diagnoses that were made when using RDI data from this device. This suggests that mild OSA captured by the RDI may be missed by the oximeter alone. Based on our prior work, we used a cut-off of ≥15/hr for OSA diagnosis when events with O2 desaturation and/or arousal were included in the index (i.e. RDI) as opposed to the AASM guideline cutoff≥5/hr for diagnosis of OSA when using an SDB index that includes hypopneas defined by O2 desaturation alone [26]. We and others have demonstrated the validity of this higher cutoff for OSA whenever using the more inclusive definition of RDI [4, 23, 27]. If one is interested in more than just the overall SDB and oximetry indices the ARES™ device provides additional information, such as sleep position and indirect confirmation of obstruction from snoring and from the shape of the airflow signal (inspiratory flow limitation), but this comes at a cost of additional time required for review and manual editing of data. A recent study by Chai-Coetzer et al showed excellent agreement of indices for diagnosis for moderate to severe OSA of both Level III and Level IV devices compared to PSG. However, this study also showed slightly worse functional outcomes when Level IV (oximetry alone) was used for OSA management, but similar outcomes between PSG and level III analysis [2]. The authors suggested that this was due to reduced physician confidence when using only a single channel, which is consistent with our previously published work [28].

Although our population composition was closer to an epidemiologic population (subjects were recruited without regard to symptoms of OSA and do not have significant co-morbid cardiovascular and metabolic conditions), the majority of our subjects were overweight, middle aged men. This likely contributed to the high prevalence of OSA. It remains to be tested whether our results can be generalized to a population with comorbid conditions showing significant hypoxemia.

Limitations of our study include the lack of a definitive reference measure of oximetry and not having comparison to in-laboratory PSG. Also, AHI4 by design includes apneas irrespective of any desaturation, and thus differs from the ODI, which only counts ≥4% desaturations. It has been estimated at least 20% of apneas may not have accompanying desaturation [29]. This may explain why the AHI4_ARES was slightly higher than the ODI_Ox2 (i.e., due to inclusion of apnea events without desaturation).

In our study population the level IV device (Wrist_Ox2) showed good agreement for diagnosis of OSA compared to the level III device (ARES™) despite different approaches to signals monitored. The disagreement between the devices for the same index and between OSA diagnoses based on AHI4≥5/hr and RDI≥15/hr with a single device were comparable to night-night variability in our study and reported in previous publications. This suggests using oximetry alone with the ODI≥5/hr as a criterion to diagnose OSA may detect OSA with a clinically acceptable success in populations composed of middle aged overweight males with few comorbidities. Whether the number of patients missed by ODI vs AHI4 or RDI will increase above 10% in other populations, such as pediatrics or patients with minimally desaturating events, will need to be tested separately in future studies.