The Center for Tobacco Policy Research is conducting a multiyear process evaluation on the status of 10 state tobacco control programs. A cross-sectional study design was used to evaluate the process of organizing and conducting statewide tobacco prevention activities. One of the specific objectives of the study was to examine the interorganizational relationships of the tobacco control network.

To obtain a diverse sample of states, states were selected based on 1) geographic location; 2) level of program capacity (e.g., funding level, age of program)To obtain a diverse sample of states, states were selected based on 1) geographic location; 2) level of program capacity (e.g., funding level, age of program); 3) presence of tobacco farming; and 4) type of lead agency (state health departments or independent organization). The 10 selected states thus represented a variety of tobacco control programs from across the country. This paper presents the network analysis results from the first set of states evaluated: Washington (evaluated June 2002), Indiana (July 2002), Wyoming (October 2002), New York (December 2002), and Michigan (February 2003).

A modified fixed-list sampling method was implemented to identify the key partner agencies of each state tobacco control program (27). The tobacco control manager from the lead agency was first contacted and asked to compile a list of partners that contributed substantially to the program or had a unique role. The research team and the tobacco control manager then discussed the list to finalize the number of agencies and individuals who would be invited to participate in an interview. The statewide coalition director verified that all major agencies were represented. During the interviews, each agency was asked questions about the other agencies in the network. Individuals interviewed also had an opportunity to suggest additional participants. The average number of agencies interviewed within each state was 14.

For each agency in the tobacco control program, a key informant was identified and asked to participate in an interview. This key informant was the staff member most familiar with the agency's tobacco control activities. The semi-structured in-depth interview collected quantitative and qualitative data on network characteristics, political support for tobacco control, the financial climate of the state, use of the CDC's Best Practices for Comprehensive Tobacco Control Programs, and organizational capacity. Trained interview teams conducted the interviews either inperson or over the telephone. Approximately 54% of the interviews were conducted in person. The average interview lasted 73 minutes.

Throughout the interviews, we collected data about each agency's interaction with other agencies within the network. The quantitative relational constructs measured were the frequency of contact (through meetings, phone calls, or e-mails) among agencies, the flow of money among agencies, and the perceived productivity of agency relationships. Appendix A shows an example of the network data collection form for one state. Each participant was asked questions about each partner agency in the state's tobacco control network. We followed typical social network analysis procedures to prepare data for analysis (28). Missing values were indicated when participants felt they were unable to answer the question because of limited contact or knowledge.

For contact frequency, if multiple respondents were interviewed from one agency (which usually only occurred with the lead agency), we averaged respondent values to produce one final value. When one response was missing for a pair of agencies, we used the response given by the other agency. Although not ideal, this type of imputation of missing network data is common. Basic network measures such as betweenness have been shown to be reliable with as much as 25% missing data (29). Answers were averaged from each pair of agencies to determine one level of contact. This is important because although individual reports of contact level may vary, it was assumed that interagency contact was symmetric. That is, if Agency A had a meeting with Agency B, this naturally means that Agency B met with Agency A. We then dichotomized the average score into "at least monthly contact" and "less than monthly contact."

For money flow, if multiple respondents were interviewed from one agency, we used the responses given by the most senior staff member interviewed. When disagreement arose on the perception of money exchange between a pair of agencies, we contacted each agency to determine the correct response. If one response was missing for a pair of agencies, we used the response given by the other agency in the pair. The money construct was dichotomized into "send money" or "do not send money."

For relationship productivity, if multiple respondents were interviewed from one agency, we averaged respondent values to produce one final value. When one response was missing for a pair of agencies, we used the response given by the other agency. When both responses were missing, we chose "neutral" as the response. This decision was made so that we would not lose the entire node in the network, nor make any assumptions on the direction of the response, given no data from either partner. We dichotomized the scores into "very productive" relationships, the highest possible productivity response, and "not very productive" relationships, which includes "counter-productive," "very unproductive," "somewhat unproductive," "neutral," and "somewhat productive."

We performed the social network analysis using graphic and statistical methods. Graphs based on each of the three constructs described above (contact frequency, money flow, and relationship productivity) were created to visually depict the relationships in each network. Statistical analyses provided measures both at the agency and network levels. Graph construction and social network analyses were conducted using UCINET Social Network Analysis Software Version 6 (Analytic Technologies, Inc, Harvard, Mass) Pajek Program for Large Network Analysis (Vlado, Ljubljana, Slovenia), and NetDraw Network Visualization (Analytic Technologies, Inc, Harvard, Mass).

Qualitative network constructs were collected via open-ended questions about the perceived effectiveness of the state's tobacco control network and suggestions for methods to improve the network's effectiveness. Each interview was transcribed verbatim and imported into the qualitative data management software NUD*IST (QSR International Pty Ltd, Melbourne, Australia). Each transcript was then coded using a detailed codebook developed during the pilot test by two trained staff members. Inter-rater reliability for coding was 83.7%. The coded text units were entered into NUD*IST, and a report was generated for each construct (e.g., network, financial climate). Analysis teams consisting of two trained staff members independently analyzed the reports to identify major themes or ideas. The team then met to discuss the results and arrive at consensus on major themes.

The five state tobacco control programs represented a variety of funding levels, network sizes, and geographic locations (Table 1). Annual funding levels for entire state programs ranged from $4.2 million (Wyoming, fiscal year 2003) to $52.3 million (New York, fiscal year 2003). According to the CDC's minimum recommendations for state tobacco control funding (1), Michigan had low funding (10% of CDC's recommendation), New York, Wyoming and Washington had middle funding levels (55%, 57%, and 62%), and Indiana had a high funding level (97%). Three of the statewide coalitions received a Smokeless States grant from the RWJF at the time of the evaluation. The average number of partner agencies per state network was 14, and the number of cities where partner agencies were located ranged from a few (two in Indiana) to as many as 11 (New York).

Four of the tobacco control networks were led by the tobacco control program within the state's department of health. The exception was Indiana's program, which was led by an independent tobacco control agency. Lead agencies usually included as key partners in the network were voluntary agencies, advocacy agencies, statewide and regional coalitions, and contractors with the lead agency. Partner agencies unique to some states included government agencies, political figures, trust fund agencies, media firms, and funding agencies. (Appendix C lists agencies, along with their abbreviations, included in each state network.)

Figure 1 depicts the monthly contact network for each of the five state programs. The amount of control over communication flow each agency had relative to other agencies in the network is based on centrality betweenness scores presented in Table 2 (the higher the score, the more control over communication flow in the network).

Two of the states presented contrasting patterns of network communication structure: New York and Michigan. New York is an example of a very hierarchical structure because of its high centralization index (40.8%), high mean betweenness score (5.1), and standard deviation (11.0). Most agencies in the New York network had frequent contact with one central agency (the lead agency New York State Department of Health Tobacco Control Program), which also had the most control over communication flow within the network, indicated by its high betweenness score (43.1). Betweenness scores dropped substantially for other partner agencies. The density of New York's network was the lowest of the five states (0.39). Figure 1 shows that many agencies had very little contact with other agencies. An agency would likely have to go through the lead agency to communicate with another agency. From qualitative analysis, we found that network partners believed the network was improving and had potential, but thought it still needed to improve its coordination and communication.

Michigan, on the other hand, had a much flatter, or nonhierarchical, communication structure. This structure is reflected by its low centralization index (10.4%), and the low mean betweenness score (3.5) and standard deviation (5.2). Partner agencies had frequent contact with many other agencies in the network. Three agencies, including the lead agency Michigan Department of Community Health Tobacco Section, the statewide coalition Tobacco Free Michigan Action Coalition (TFMAC), and the contractor Gerontology, had higher control over communication flow than other agencies in the network. Although these three agencies had higher relative betweennness scores within their own network, their scores were not very high compared to agencies in other state networks. Michigan's density (0.58) was higher than that of the other four states. In contrast to New York's contact graph (Figure 1), Michigan's figure shows a much more connected communication network. There were multiple routes for information flow from agency to agency. Michigan's partner agencies felt their network was very effective and had improved over the last few years. They believed TFMAC and a Smokefree Regulations Task Force, a group of partners focusing on clean indoor air efforts, were important network components that facilitated the process of bringing stakeholders together. Partners felt this helped strengthen the network's ability to work more efficiently. Yet partners thought communication could be improved by openly sharing agencies' priorities and activities.

In examining all five states, a few patterns emerged. The lead agency usually had the highest control over communication flow, which intuitively makes sense. However, statewide coalitions, which are known to bring stakeholders together on tobacco issues, usually did not have the highest communication control, except in Michigan. Most network communication structures were relatively hierarchical, except Michigan, which had a much flatter structure. The average density of the contact networks was 0.47. The densities ranged from 0.39 in New York to 0.58 in Michigan.

Figure 2 shows the money flow network for each state. Table 3 presents the financial influence scores for each agency in each network. In most states, the lead agency had the highest financial influence over the network, which corresponds with their duties as fiscal oversight agencies. The exception was Wyoming, where the legislature was included as a partner in the network. Due to the legislature's role in appropriating funding, it had the highest financial influence. Statewide coalitions were usually highly influenced by others in the network, except in Indiana. The statewide coalition Tobacco Smart Indiana did not send or receive money from anyone in the network at the time of the study. At the time of the evaluation, the coalition was in transition because of a loss of funding.

Many of the states with greater funding levels had more complex money flow networks, such as Indiana, New York, and Washington. In these states, money flowed not only from the lead agency to other agencies but also flowed among contractors and coalitions. Wyoming, which had a low funding level, had the simplest money flow network, with money mainly flowing from the legislature to the lead agency and then to contractors or coalitions. Some money was also given by other agencies to the statewide coalition. Network density related to money flow varied little among the states. The average density was 0.09.

Figure 3 depicts the productivity of relationships in the five state networks.  Table 4 presents the prestige score (normalized indegree) for each agency in each network.

In Washington, the lead agency, Washington State Department of Health Tobacco Prevention and Control Program (WA DOH), had the highest prestige (normalized indegree 69.2), followed by the American Lung Association-Washington State Branch and the Washington Office of the Attorney General (both normalized indegree 46.2). The WA DOH was highly regarded by partner agencies. Furthermore, partner agencies were pleased that the state secretary of health had made tobacco control her highest priority. The state attorney general had been one of the lead negotiators in the Master Settlement Agreement and was overwhelmingly identified as a tobacco control champion. The statewide coalition Washington Alliance for Tobacco Control and Children's Health (WATCH) ranked in the middle of other Washington agencies in prestige (normalized indegree 30.8). At the time of the evaluation, WATCH was going through a transition because of a recent loss of funding and was reevaluating its role in the tobacco control network. Many partners were uncertain of the coalition's existence or future plans. Washington's productive relationships network had a low density compared to the other states (0.27).

Wyoming's statewide coalition, Wyoming Tobacco Use Prevention (WY TUP), had the greatest prestige score in its state (normalized indegree 81.8). The lead agency, Wyoming Department of Health Substance Abuse Division, Tobacco Prevention and Control Program (DOH TPCP), on the other hand, had a much lower score (normalized indegree 45.5). Again, these results were supported qualitatively. Partners were very pleased with WY TUP and its accomplishments. However, they believed the DOH TPCP, which houses the tobacco control program, did not provide enough support for tobacco control. In Wyoming, the Legislature had the lowest prestige score (normalized indegree 9.1) due to sentiment that the legislature as a whole was unsupportive of tobacco control. Partners believed that only a few legislators made tobacco control a priority. The Department of Education and Department of Maternal and Child Health in Wyoming also received relatively low scores in prestige. Partners believed those agencies were not as engaged in tobacco control as they could be.

Although problems may be encountered with the lead agency in some states, it usually had a very high number of productive relationships (if not the highest) across states. Statewide coalitions funded by the RWJF (New York, Wyoming, and Michigan) had higher prestige scores than coalitions that did not receive RWJF funding and were in transition (Washington and Indiana). The average prestige score for RWJF-funded coalitions was 69.9, compared to 29.7 for non-RWJF–funded coalitions. Agencies with narrowly defined roles, such as contractors or agencies with a local focus, had fewer highly productive relationships, while state level agencies usually had higher numbers of productive relationships. The average density of the productivity networks was 0.34. The results suggested a relationship between contact network density and productivity network density. Michigan had the highest productivity density (0.41) and also had the highest contact density (0.58). New York (0.25) and Washington (0.27) had lower productivity densities and also had lower contact densities (New York, 0.39 and Washington, 0.44).