About 24% of American children aged 0 to 4 years are enrolled in center-based child care and another 14% are cared for by a nonrelative adult (1). Thus, nearly 40% of young children spend most of their day being cared for by nonparent adults and are exposed to food and physical environments that are determined by their caregivers. The child care environment, including healthy nutrition and adequate physical activity, is an important factor in the health, well-being, and weight of young children. Many researchers and policy makers have suggested that health behavior patterns are established early in life, making the child care environment an essential element in national efforts to reduce childhood obesity and promote healthy behaviors even among the very young (2–4).

Policy can play a key role in regulating healthy child care environments; although research in this area is growing, increased evaluation of the effectiveness of such policies is needed (4,5). Once a policy is passed, there remain many ways in which the intended benefits of the policy may be compromised when it is implemented in real-world settings. It is critical to understand whether child care centers are able to comply with such policies.

This study examined group child care centers in New York City to assess whether and how well these centers comply with regulations, in place since 2007, intended to improve child nutrition, increase physical activity, and reduce noneducational screen time. New York City monitors compliance with regulations for licensed group child care centers using periodic (generally annual) site visits by city sanitarians; the assessment of compliance is basic, and the implications of poor compliance is an initial citation and a possible loss of license. To date, no center has lost its license because of the beverage, physical activity, and screen time regulations. Thus, our study sought to use a more comprehensive approach to assess compliance. Our approach, which measured compliance in different ways and in an applied research context, offered unique opportunities to investigate variability in compliance. Although recent research has shown relative validity of self-report and observational measures of child care nutrition and physical activity environments, additional evaluation is needed to identify reliable methods of assessing implementation of policies and regulations in real-world contexts (6).

This evaluation used a multimethod cross-sectional approach and data collected from licensed group child care centers in New York City at 2 time points. This approach was designed to measure the level of compliance with regulations in centers and classrooms, the level of agreement in compliance between centers and classrooms, the factors that affect compliance, and the behavioral outcomes associated with different levels of compliance (eg, physical activity). Details on sampling and data collection are available in Breck et al (7), but they are explained briefly below.

In the center component, most child care centers were classified as compliant with the regulations on the type of milk to be served to children (80.0%), type of juice to be served (69.1%), the restriction on sugar-sweetened beverages (78.9%), water availability (89.1%), provision of at least 30 minutes of structured physical activity time (78.5%), total physical activity time of at least 60 minutes in a full day (87.2%), amount of television time permitted (100%), and provision of educational-only screen time (84.4%) (Table 2).

Most centers in the classroom component were classified as compliant with type of milk served (90.0%), type of juice served (84.5%), restriction on sugar-sweetened beverages (84.4%), amount of television time permitted (86.1%), and provision of educational-only television time (89.0%). However, a smaller percentage of centers was classified as compliant in the classroom component with the amount of juice given to children (67.3%), water availability (55.5%), amount of structured physical activity offered (30.0%), and total amount of physical activity offered (34.9%).

Compliance varied between components (Table 2). The percentage of centers that were classified as compliant in both the center and classroom components ranged from 21.5% (for structured physical activity) to 86.1% (for television time). We found a high percentage of agreement between center and classroom component compliance for milk (82.7%), sugar-sweetened beverages (70.6%), television time (86.1%), and provision of educational-only television programming (77.1%). In contrast, we found a low percentage of agreement between center and classroom component compliance for total physical activity time offered (38.5%) and structured physical activity time offered (34.6%).

Using different methods of assessment at 2 time points, we found high levels of reported and observed compliance with most regulation requirements. The percentage of centers that were classified as compliant in both components ranged from 21.5% (for structured physical activity) to 86.1% (for television time). Especially for television and milk, consistency between components was fairly high; centers that were classified as compliant in the center component were likely to be compliant in the classroom component about 4 to 6 months later. We hypothesize that the difference in compliance is because implementation of these requirements are easy to control at the center level, whereas the provision of physical activity is more sensitive to daily fluctuations, individual classroom factors (eg, variations among teachers or children), or other factors that make consistency of compliance more difficult to achieve. Kakietek et al (9) examined the factors that contributed to center compliance in this evaluation.

Each component of this evaluation used different data collection methods, and data were collected at 2 different times (about 4 to 6 months apart). The center component used self-reports and site observations, whereas the classroom component used a 2-day classroom observation in 1 randomly selected classroom. Although this evaluation was not intended to validate methods for assessing compliance with the regulations, it does shed light on the strengths and weaknesses of various methods of assessing regulatory compliance. For some policies, levels of compliance may vary across time; each day, a center and its staff must act to achieve compliance, and compliance with all regulation requirements may not always be achieved on any given day. Understanding compliance as a daily event might help to explain some of the variations found day to day and between center and classroom components.

For several regulation requirements, we found inconsistent compliance between the center and classroom components. One possible explanation for the inconsistency is that compliance with the regulations may have changed between the first and second data collections. Center component site visits were conducted in fall 2009, and the classroom component was conducted in spring 2010. Although the center staff members were not informed of the results of the center component, during the intervening time, changes in staff, facility, or other factors may have influenced the center’s implementation of the regulations.

A second possible explanation for the inconsistency in compliance is that data collected in the center component from teachers, directors, and food service staff members were self-reported inaccurately. This inaccuracy could have been caused by social desirability bias, inconsistent implementation within each center (eg, certain classrooms were compliant and others were not), or lack of knowledge among respondents about the practices or the policy or both. For example, although directors may have believed they served only 100% juice, they may have been unclear about the definition of 100% juice and unknowingly served juice drinks. Although the city provides training for center staff on the 100% juice policy, we do not know and did not assess in this study the extent to which the training results in comprehensive knowledge among center staff. The greatest difference in compliance between components was related to the provision of physical activity opportunities. Respondents in the center component may have wanted to provide the most socially desirable response: that they typically provide the required amount of physical activity opportunities. This explanation would account for the higher proportion of teachers who reported compliance in the center component than the proportion of teachers observed in the classroom component.

A third possible explanation for the inconsistency in compliance is measurement error. Although data collection tools were adapted from other studies where possible, our interview questions or observation tools may have incorrectly classified the center environment or staff behavior. Finally, differences in compliance between the center and classroom components may have resulted from atypical events on the days that data collectors visited the centers. In this scenario, data collected on an atypical day could result in higher or lower levels of compliance compared with data collected on a typical day.

The centers in our study were in low-income, urban neighborhoods; thus, generalizing our findings to other settings is cautioned. Given the resources involved in complying with these and similar regulations, higher-income neighborhoods could demonstrate higher levels of compliance than the centers participating in our study. However, comparisons of compliance assessed by the 2 different components used in our study may have applications for those monitoring compliance and those conducting applied research.

Another limitation of the study sample in both components is the potential for nonrandom bias in the rate of refusal to participate. Although centers included in the center component sample were selected randomly from a sampling frame, about 25% refused to participate in the center component, and about 38% of the centers who participated in the center component opted not to participate in the classroom component. Centers that took part in the classroom component were significantly more likely than centers that took part only in the center component to participate in the Child and Adult Care Food Program, be part of a larger parent agency, have dedicated food service staff, be in DPHO areas, and participate in training programs provided by the health department. Our data on refusal rates and characteristics of centers that refused to participate suggest that centers with poor compliance with the regulations were less likely to participate in the classroom component than centers with better compliance.

Although our evaluation was not designed to inform the enforcement of regulations, our methods may shed light on one of the practical difficulties of enforcing these regulations through the traditional means of an annual site visit by an inspector: the inability of inspectors to monitor compliance every day. Our study demonstrated on 2 separate occasions (1 additional observation per year than an inspector would make to each facility) that compliance with all components of the regulations varies over time and the method of assessing compliance may be especially important for physical activity requirements. We found a much lower level of compliance with the physical activity requirements when we used observation rather than director and teacher self-report. This lower level of compliance suggests that the policy’s intended benefit is less likely to be achieved. Policy makers may want to consider not only the content of such policies but also the mechanisms for ensuring compliance over time. For some policies, changes to the methods and frequency of compliance checks, penalties for noncompliance, and training and support may be needed to strengthen implementation.

Recent literature (6) suggests that self-reported survey responses, interviews, and observation are highly correlated for similar studies. Our study builds on that idea by combining multiple data sources to represent not only compliance for a single classroom at a single point in time but also center-wide compliance. Although our evaluation was not designed to validate measures of compliance, our methods can inform future research and evaluation. First, numerous measures could be used to assess compliance (eg, logs, other observational tools), although all methods have strengths and weaknesses. The measures used in our study were designed to capture data on classroom-level and center-level compliance at 2 time points to examine relationships between compliance and child-level outcomes (8). Second, researchers who are interested in validating compliance metrics would want to design studies explicitly focused on validation using multiple methods of assessment over time to draw more robust conclusions about the validity of individual methods. Although we could not verify which method was most reliable or valid, our findings suggest that observational methods may be a more conservative estimate of daily compliance with physical activity regulations. A study contrasting the use of logs to collect teacher-reported data on physical activity and the use of a third party to observe physical activity offerings would aid in identifying the most reliable, valid, and cost-efficient means of assessing compliance in group child care facilities.