Collaborative approaches to change are ubiquitous in the United States (US) and around the world. Councils, task forces, and coalitions have formed across the country to respond to complex social issues including substance use prevention (Florin, Mitchell, Stevenson, & Klein, 2000; Florin et al., 2006), health promotion (Butterfoss, Goodman, & Wandersman, 1993; Butterfoss & Kegler, 2009), and family violence (Allen, Javdani, Lehrner, & Walden, 2012; Allen et al., 2013). In recent years, collaborative models have been applied in response to growing concerns about obesity (Huang et al., 2015), access to affordable healthy food (Walsh, Taggart, Freedman, Trapl, & Borawski, 2015), and food system sustainability (Hassanein, 2003). The aim of the current study was to apply existing models regarding the effectiveness of collaborative settings to the specific context of Food Policy Councils (FPCs). This creates a two-fold purpose. The first is to examine the applicability of the extant literature on collaboration to the efforts of FPCs. The second is to extend the current knowledge on collaborative settings by testing and refining a parsimonious model of FPC effectiveness adapted from an empirically supported model developed for Family Violence Councils (see Allen, Javdani, Lehrner, & Walden, 2012).

FPCs are organizations that bring together stakeholders from across the food system to identify food system issues and solutions, coordinate programs, and inform policy (Harper, Shattuck, Holt-Giménez, Alkon, & Lambrick, 2009). The number of FPCs in the US, Canada, and Tribal & First Nations increased from 43 in 2005 to 282 in 2015 (Food Policy Network, 2015a). Currently, 41% of FPCs consider themselves to be grassroots organizations, 16% are non-profits, and 18% are directly affiliated with government and may operate at local (county, municipal), state, province, tribe, and regional levels (Food Policy Network, 2015a). While the missions of FPCs vary, councils generally aim to increase access to healthy foods, promote sustainable, local agriculture and economic growth, and encourage equity within food systems (Scherb, Palmer, Frattaroli, & Pollack, 2012; Schiff, 2008). In doing so, FPCs have the potential to improve the context in which individuals make decisions related to their health and socioeconomic factors described in the Public Health Impact Pyramid (Frieden, 2010). The Centers for Disease Control and Prevention recommend establishing FPCs to improve the local and state food environments, specifically through increasing access to fruits and vegetables (Centers for Disease Control and Prevention, 2011).

Despite increased interest in FPCs as a strategy for improving food environments in communities, there is little evidence about what factors make FPCs effective. Numerous reports and case studies that describe FPCs provide valuable information about FPCs, but they do not offer a framework for FPC function that is testable and generalizable across councils (Borron, 2003; Dahlberg, 1994; Schiff, 2008). Without such a framework, it is challenging to evaluate what internal council processes are working and what could be improved across councils (Harper et al., 2009; Zakocs & Edwards, 2006). FPC members could waste time, energy, and limited resources if they do not have guidance about best practices within FPCs. Moreover, a common vocabulary that is used to describe abstract concepts related to FPC function could help FPC members efficiently communicate with fellow council members, as well as with funders and decision-makers. The growth of FPCs shows that councils are an appealing strategy for coordinating responses to complex food system issues, yet empirical evaluation of FPCs is limited.

Evaluating community councils (such as FPCs), coalitions, collaboratives, and partnerships is challenging (Berkowitz, 2001; Webb, Pelletier, Maretzki, & Wilkins, 1998). These initiatives are complicated because they require groups of individuals who may have divergent goals and levels of commitment to work together towards a common vision (Himmelman, 2001). The processes governing FPC function can strengthen council members’ satisfaction in the council, or lead to dissatisfaction and erode members’ investment in the council (Weiner, Alexander, & Shortell, 2002). Therefore evaluating the internal processes by which councils operate is important in order to explain why some councils are effective in achieving community-level outcomes while others are not (Allen, Watt, & Hess, 2008; Zakocs & Edwards, 2006). In addition to councils being internally complicated, they are working to affect change in complex systems (Roussos & Fawcett, 2000; Snowden & Boone, 2007). Specifically, it is difficult to link FPC activities to change within the food system because change in a complex system takes time, often requires acting on windows of opportunity, involves many actors, and may be counteracted by action in another part of the system (Hammond & Dubé, 2012; Lich, Ginexi, Osgood, & Mabry, 2013). As the Toronto FPC coordinator explained, “Because much of our work is indirect, facilitative, and collaborative, it’s difficult to isolate the impacts of our specific efforts” (MacRae, 1994, p.17). Without evidence about what internal council factors are associated with council effectiveness, however, FPCs may fail to reach their potential to influence the food systems in their communities.

In this study, we empirically tested the FPC Framework, which is a model of the key concepts and relationships between them that we hypothesize explain the internal functioning of a FPC. The framework was adapted from a parsimonious community collaborative model that was developed and empirically tested by Allen and colleagues in the context of the system and community response to family violence (Allen, Javdani, Lehrner, & Walden, 2012).

Building on the measures Allen and colleagues employed in the study of Family Violence Councils (Allen, Javdani, Lehrner, & Walden, 2012), a Food Policy Council Self-Assessment Tool (FPC-SAT) that asks FPC members’ about their perceptions of their council was created and validated (Calancie et. al, 2017). The FPC-SAT was used to collect data in this study.

To our knowledge, this is the first study to test a framework depicting members’ perceptions of how FPCs function. The FPC framework can help guide capacity building for FPCs, and the researchers and practitioners who work with them. It also provides insight into the mechanism by which FPCs may achieve distal, community-level outcomes. Our results show that interventions aiming to strengthen FPCs should be directed toward increasing Organizational Capacity components (leadership, breadth of active membership, inclusivity of council climate, and formality of council structure) since they may increase Council Effectiveness more than efforts directed at increasing Social Capital. Moreover, efforts to increase Organizational Capacity are likely to increase Social Capital. This study provides support for the generalizability of Allen and colleagues’ parsimonious model explaining how community collaborations can achieve institutionalized change (Allen et al., 2012).

The concepts that were measured in this study are modifiable and thus could be targeted through capacity building interventions. For example, Florin and colleagues conducted a capacity building intervention that improved the confidence of community groups, including coalitions, to address tobacco control efforts across Rhode Island (Florin et al., 2006). In another study, coalitions randomly assigned to receive an internal capacity building intervention demonstrated more effective prevention plans, plan implementation, and meetings than those that did not receive the intervention (Riggs, Nakawatase, & Pentz, 2008). A recent case study found that a technical assistance and training intervention with community coalitions improved internal coalition processes and strategic planning, and was associated with increased community-level changes at the end of the intervention (Keene Woods, Watson-Thompson, Schober, Markt, & Fawcett, 2014). The Interactive Systems Framework is an approach to identify and implement innovations that could improve organizational capacity within FPCs and other collaboratives (Wandersman et al., 2008). The revised FPC framework and FPC-SAT can help FPCs identify what internal components are important targets for capacity building in their specific council.

We conducted a multisite, theory-driven evaluation of FPC members’ perceptions of their councils with a large sample of FPC members from across the US, Canada, and Tribal & First Nations. This study addresses the numerous reports calling for more evaluation of FPCs (Harper et al., 2009; Scherb et al., 2012; Schiff, 2008). Evaluation informs best practices and can also make FPCs more attractive to funders. A study examining challenges related to FPC evaluation found the following barriers: a negative perception of evaluation held by council members; a lack of consensus about how to evaluate FPCs; insufficient evaluation capacity within FPCs; and an “inadequate appreciation of increasing accountability pressures” (Webb et al., 1998, p.65). We addressed several of those barriers in this study. Those who hold a negative perception of evaluation may not have participated in our study. However, hundreds of FPC members completed the FPC-SAT, and eight or more members from 20 councils valued the evaluation process enough to complete the FPC-SAT in order to receive a tailored feedback summary for their council. To help build consensus about how to evaluate FPCs, we developed the evaluation questions on the FPC-SAT with significant input from FPC members, leaders, researchers, and practitioners who work with councils (Calancie et al., 2017). Moreover, the evaluation questions were adapted from empirically tested measures (Allen et al., 2012) and informed by a review of the community coalition literature. The evaluation was led by academics with evaluation expertise; internal FPC evaluation capacity was not required. The resulting framework can guide evaluation activities in FPCs that may have limited capacity to develop an evaluation process from scratch. Finally, although we did not address council members’ appreciation for accountability from various stakeholders, our study did provide tools and guidance to help those who are striving to elucidate the connections between their councils’ internal processes and potential impact on their communities.

Additional research is needed about the potential drawbacks of FPCs. Some studies suggest that while community coalitions typically aim to include a broad group of community organizations and representatives, they can may actually consolidate power within a small group of participants and reinforce existing power dynamics within communities (Chavis, 2001; Himmelman, 2001). In this study, we found that fewer participants completed questions about fair resolution of conflicts than other questions about leadership. This could be an indication that some council members do not feel comfortable voicing leadership style concerns, perhaps due to power relations within the group or within the community. Furthermore, a network analysis examining changes within a network of organizations that address food security in their community found significant centralization within the network over time as an FPC formed (Freedman & Bess, 2011). While the number of relationships between organizational partners within the networks increased, the relationships did not develop evenly across partners; they increased more among a few key partners (Freedman & Bess, 2011). Increased centralization within a network of organizations could lead to greater efficiency in completing tasks, but it could also “challenge shared decision-making and accountability, and put at risk the commitment and participation of peripheral members” (Freedman & Bess, 2011, p. 407). Researchers should examine the extent to which FPCs foster collaborative empowerment, which emphasizes “community organizing, grassroots leadership development, and increasing the ownership and power of those primarily affected by the coalition’s activities” (Himmelman, 2001, p.282). Research is needed to understand whether FPCs are reinforcing existing conflicts within communities that stem from unequal distribution of power, or if councils are serving as arenas to transform those conflicts to enable community capacity buildings by empowering otherwise marginalized groups (Chavis, 2001). Indeed, studies have shown that community councils have the potential to influence the organizational contexts of participating organizations (Allen et al., 2013), which could in-turn influence community power dynamics (Himmelman, 2001). This research should be conducted to avoid unintended negative consequences of promoting FPCs in communities.

Our use of SEM is a strength of this study. SEM and path analysis are rarely used to test proposed mechanisms for coalition effectiveness; multivariate analysis and covariance descriptions are more common (Zakocs & Edwards, 2006). Studies that use these methods group the coalition factors in different arrangements, making it challenging to compare mechanisms across studies. However, some studies have tested similar mechanisms explaining community coalition effectiveness (Alexander, Christianson, Hearld, Hurley, & Scanlon, 2010; Crowley, Yu, & Kaftarian, 2000). Crowley and colleagues used structural equation modeling to test relationships between collaboratives’ organizational practices and impacts and found that their hypothesized models were a good fit for the data over three years (Crowley et al., 2000). They also found evidence that the relationships between coalition activities and impact may not be linear overtime, and factors such as organizational capacity and partnerships will not necessarily result in healthier communities. Contextual factors and a limited ability to quantify coalition impacts might explain this finding, as well as the potential delay between coalition activities and their impact. Our moderation analysis showed that community context influenced the relationship between Social Capital and Council Effectiveness such that the relationship between those variables was not significant among those who did not regularly experience barriers when trying to influence policy. Perhaps having a council in place is sufficient to influence policy in those communities, independent of the amount of Social Capital generated within those councils. Among members who reported that their councils regularly experience barriers, Social Capital is more strongly associated with Council Effectiveness. We also hope to examine the relationship between concepts displayed in the FPC framework, community context, and distal, community-level outcomes over time in future research.

As with all research, there were limitations to this study. First, there may have been selection bias because participants choose whether or not to complete the FPC-SAT. It is likely that council members who have a positive view of their council, or of evaluation and research in general, would be more motivated to complete the FPC-SAT than those who hold negative views of their council because they knew that council coordinators will receive de-identified feedback summaries. The feedback report incentive is likely to attract councils that are cohesive and are interested in receiving feedback about their council. Another limitation in this study is that we asked individual participants for their perceptions of their council. We did not collect objective measures of council characteristics or function at the council level, such as observations of FPC meetings or document review (Hawe & Stickney, 1997). Also, while we had sufficient power to test model fit with our full sample, our moderation analysis should be interpreted with caution due to the relatively small sample size in each of the two groups. Finally, though data-informed model respecification and post hoc analysis are common practices in SEM, ideally each respecified model should be tested with new data (Bollen, 1998; Kline, 2011). Future research should validate the revised FPC Framework with an independent sample of FPCs or other similar community coalitions.

Several important lessons emerged from this study. We found that members from 32% of FPCs listed in the 2015 Online Directory were willing to complete the FPC-SAT. Additional recruitment activities might have improved our response rate. A shorter survey may have increased the response rate as well. Our results indicate that FPCs can be evaluated similarly to other community coalitions; factors that are associated with community coalition effectiveness apply to FPCs. Future research should consider how to best measure and evaluate the impact that FPCs have on community-level outcomes, such as access to healthy foods, economic growth, and natural resource stewardship.

FPCs have enormous potential to bring together a variety of viewpoints and skillsets to influence food systems. From production to distribution, consumption, and disposal, food influences the health, economic viability, political stability, and ecological vitality of communities to nations. The framework tested in this study can help FPCs realize their potential in promoting food system change that equitably improves lives today and in the future. Our results indicate that increasing FPC members’ perceptions of Organizational Capacity is associated with increasing Social Capital, synergy, and perceived impact on the food system. Councils may consider using the FPC-SAT and the revised FPC framework to identify specific areas that could be improved in their FPCs. Once specific areas for improvements are identified, councils can seek technical assistance, resources, or other approaches to increase their capacity to affect change in their food systems and improve community-level public health outcomes.

Elements of the food system and their effects on the public’s health are often considered in isolation. The Institute of Medicine suggests examining the food system and its health implications using a systems approach (Nesheim, Oria, & Yih, 2015). Their approach encourages collaboration-building and integrating information from a variety of sectors. Effective FPCs offer a platform to engage and empower representatives from across the food system to better understand their food system, and take coordinated action to promote health, natural resource stewardship, economic vitality, and equity in communities.