System mapping is a tool originating from systems thinking in which a given domain is represented as a set of components (nodes) and relationships (edges). While components always have a name (i.e., node labels), the type of edges depends on the specific representation of a system used in a given project. One such representation is a causal map, in which categorized edges labeled positive (+) or negative (−) link various quantifiable factors to create a directed network (Firmansyah et al., 2019). A positive edge from one factor to another implies positive causation (an increase in the first factor will cause an increase in the second), while a negative edge implies negative causation (an increase in the first factor will cause a decrease in the second). For example, a positive edge from the node representing suicide ideation to the node representing suicide attempts means that an increase in suicide ideation leads to an increase in suicide attempts. Other representations of a system include stock and flow diagrams used for System Dynamics, where an edge may have a numerical value that is encoded within a modeling software rather than displayed on a schema. Ontologies are a related representation, in which the meaning and characteristics of a system can be conveyed in more details by equipping concepts with properties or associating them with entries in a glossary.

Several features present in complex problems such as feedback loops¹ (when a change in one factor ultimately affects itself) and factors with a high number of outgoing relationships can be represented in systems maps (Giabbanelli et al., 2021). As a result, these maps are a powerful tool to analyze complex problems.

Mapping a preexisting system is now one of the key stages in identifying and evaluating interventions for public health programs (Luna Pinzon et al., 2022). Creating and visualizing a systems map can exhibit the intricacies of the issue at hand and highlight the interdependence of factors through feedback loops (Finegood et al., 2010). Depicting feedback loops is critical as it helps decision-makers to comprehend the relationships between variables and begin to weigh the ramifications of policies and interventions. Interventions are actions taken to change variables in a system to achieve a specific outcome—e.g., reducing access to lethal means among persons at risk of suicide (Stone et al., 2017). Identifying feedback loops also empowers decision-makers to utilize naturally occurring system dynamics in their interventions (Firmansyah et al., 2019). For example, implementing social-emotional learning programs in schools may increase coping and problem-solving skills, which may, in turn, reduce suicide attempts (Stone et al., 2017).

System maps also allow users to identify a group of factors, the variables affected by them (both directly and indirectly), and the nature of these effects (e.g., by monitoring their type in a causal map) (Firmansyah et al., 2019). Without a systems map, tracing the effects of variables throughout the system would likely be more difficult. The process of creating a systems map can have widespread benefits even if the map is not widely used. Bringing together a broad range of stakeholders to form the map can help “forge multisector, multidisciplinary relationships that support future decision-making based on evidence” (Finegood et al., 2010). Moreover, stakeholder participation in creating a map can increase the legitimacy of decisions (Firmansyah et al., 2019). Combining both factors can help boost the quality and support of decision-making.

The general process of constructing systems maps with participants has recently been covered in several articles, with respect to either knowledge representation (Giabbanelli and MacEwan, 2024) or the interactions between facilitators and participants (Knox et al., 2024). We thus provide a more succinct overview here, based on the diagram in Figure 1. One approach to create a map is to engage in participatory modeling. In this case, modelers perform community-based participatory research (CBPR) by building a collaborative partnership with community members through participatory activities (McGlashan et al., 2019). A model is created in CBPR by identifying and recruiting eligible participants, then obtaining informed consent. Participants are selected based on the problem boundaries; for instance, in the case of the model on adolescent suicide, individuals whose experience lies entirely on suicide in adulthood would not be engaged (Giabbanelli et al., 2021). Although research areas such as educational technology tend to use software so that each individual directly creates their map (e.g., cMap, jMap, Coggle), CBPR has fewer platforms (e.g., STICKE (Hayward et al., 2020)) and tends to employ trained facilitators who will support participants in developing maps. Data elicitation techniques can either be performed with a group, which will automatically result in a group-level model or with individuals, whose models are later combined into a group-level model (Giabbanelli et al., 2021).

When working with individuals, information is elicited using unstructured techniques, which rely on asking open-ended questions to externalize the participants” perspectives without imposing the bias of the interviewer on the interviewee. Transcribed interviews can be analyzed to identify variable names and the nature of their relationships, represented as nodes and edges, respectively. When building a causal map, relationships encode the direction and type of impact. For example, “traumatic experiences increase the risk for suicide ideation” and “having meaningful relationships with positive peers protects against suicide attempts.” Consequently, causal maps use directed, typed edges. In the two examples, the statements could be represented as traumatic experiences →+ suicide ideation and connectedness →¯ suicide attempts, respectively. In the case of ontologies or stock and flow diagrams, relationships would have attributes or numerical values, respectively.

Even when they share the same interest, modeling projects may differ in some of the sub-steps (Figure 1) based on considerations such as experience and availability of the facilitator (e.g., do they prefer one-on-one sessions or group workshops?), access to participants (e.g., can they all come to the same room?), or time constraints (e.g., do we need to produce a map by the end of the workshop?). The NCIPC model employed one-on-one interviews with a trained facilitator and an observer, then a causal map was produced from each interview (including typed and directed edges), and maps were aggregated with the assistance of SMEs to identify semantically equivalent constructs (Giabbanelli et al., 2022). The NCIPC model thus only reflects the views of the participants involved. The stock and flow diagram produced by Page and colleagues involved workshops and a wide representation “from health and social policy agencies, local councils, non-government organizations, emergency services, primary care providers, program planners, research institutions, community groups and those with lived experience of suicide” (Page et al., 2018). The model-building method thus differs from the NCIPC case, which relied solely on experts (e.g., epidemiologists, psychologists). As is commonly the case, the model by Page et al. was developed using the same protocols that the team applied for CBPR research in other domains, such as diabetes management (Freebairn et al., 2016) and alcohol-related harms (Atkinson et al., 2017). The model by Page et al. proceeds in a series of steps, from the general population to vulnerable individuals, who can become distressed and attempt suicide, leading either to fatality or post-attempt treatment.

CBPR is not the only way to create a map. For example, participants may be unavailable for a research team, or the modelers may consider that the evidence-base is sufficient to derive a map. The stock and flow model created by Chung thus indirectly builds on the work of experts, by expanding on the Interpersonal Theory of Suicide (Chu et al., 2017). This theory posits that suicide ideation happens when individuals feel that they are a burden and do not belong (i.e., the core constructs of belongingness and burdensomeness). In addition, the theory considers that attempts are enabled when individuals have a desire for suicide and no longer fear death (i.e., capability) (Vélez–Grau et al., 2023; Kelley and DeShong, 2023). Chung created the model by building on theory and identified specific variables using data from the In-Home surveys administered as part of AddHealth, a widely studied dataset created by experts. As a result, the model includes constructs such as depression (characterized by e.g., average duration and depression level), perceived self-worth, desire to die by suicide, fear of death (which is reduced by attempts and exposure to painful experiences), hopelessness, burdensomeness, and cultural stigma associated with suicide. The ontology by Brenas et al. (Brenas et al., 2019) also built on prior work, as it (i) reused the seminal study of Felitti and colleagues to identify relationships between Adverse Childhood Experiences (Felitti et al., 1998), (ii) reused medical ontologies to cover concepts such as abuse or mental illnesses, and (iii) connected the concepts as needed for the purpose of their study. The constructs thus cover mental health, physical harm (detailed as being hit, push, from thrown object, etc.), substance use and abuse, housing problems (e.g., low-quality housing or homelessness), and their effects on health (detailed via e.g., exposure to bug infestation, lead-based paint, mold, water leaks), triggering events (e.g., death, incarceration, parental separation or divorce), and access to care (e.g., transportation).

In a comparative network analysis, we need to assess the models based on shared features. Regardless of whether they originated as ontologies, causal maps, or stock and flow diagrams, all four models share a representation as directed, labeled networks. However, edges have different information, such as types (+ and − in causal maps), numerical weights or rates per time unit (stock and flow diagrams), or properties (ontologies). We can thus analyze all four models by accounting for the direction of their edges, but not in terms of edge weights or edge types.

A network can be analyzed at different levels of granularity. McGlashan and colleagues have summarized how several network analyses can be used on a systems map, defining each analysis (e.g., density, degree) along its interpretation in the context of a systems map and the takeaway message for interventions (c.f. Table 1 in McGlashan et al. (2016)). The most granular analysis is performed at the level of individual nodes, using centrality measures to capture the “importance” of a node with respect to a desired property (e.g., degree, flow). Although node centrality is a well-known analytical method in network science, the interpretation of results is field-dependent. Table 1 lists the typical centrality measures used in systems maps together with their interpretation in the context of intervention planning for public health. Figure 2 exemplifies how centralities might be calculated on a fragment of the NCIPC model. These centrality measures have been used in numerous other studies on network science for public health (Power et al., 2022; Brauer et al., 2022; Smith et al., 2022), as well as in other comparative studies on systems maps (Krabbe, 2014; Kim and McCarthy, 2021; Wang and Giabbanelli, 2023).

At an intermediate level of granularity, we can analyze groups of nodes. This can be achieved by examining very local groups, through the clustering coefficient. For a given node, this measure quantifies the density of connections among the nodes’ neighbors. For example, if a node has four neighbors and they have three edges (out of 4 × 3 = 12 possible edges among four nodes), then the node has a clustering coefficient of 312=0.25. The clustering coefficient of a network is obtained by averaging the clustering coefficient of individual nodes. While the clustering coefficient is a familiar construct in the literature on suicide, it is more commonly employed in neurophysiology studies through brain networks (Kim et al., 2024; Qin et al., 2024) or in computational social science via online social networks (Masuda et al., 2013; Colombo et al., 2016). In this paper, we use the clustering coefficient to study a network model of suicide. Larger groups of nodes can be studied through communities. Communities exist in certain subgroups where connections within the group are denser than connections to other groups. Communities can be useful for revealing hidden relationships among nodes (Lee and Lee, 2013), identifying themes (Chunaev, 2020), and visualization purposes (Fan, 2020). For example, communities can simplify a large system map into five or six themes, which become priority areas for prevention. Communities can serve to create “derivative maps” which are reduced to their groups, as illustrated by analyses of the Foresight Obesity Map which was reduced into a series of such derivative maps to guide policies (Finegood et al., 2010).

The goal of community detection via algorithms is to divide nodes in such a way that there are many edges in subgroups and few edges between them (Newman, 2018). Multiple community detection algorithms have been developed (Clauset et al., 2004; Lancichinetti and Fortunato, 2009; Newman and Girvan, 2004), and we use two algorithms to support the triangulation of results (i.e., ensure that findings are not an artifact of one specific method). The Louvain Algorithm for community detection greedily maximizes modularity (Newman, 2018, p. 511–512) by repeatedly applying two steps: joining individual nodes into groups to find the arrangement with the highest modularity (which measures the density of links within a group as compared to links going outside) and restructuring the network by aggregating communities into “super nodes”. This iterative process is performed until no increase of modularity is possible. Figure 3 exemplifies the application of this method onto the same section of the NCIPC model as shown in the previous figure for centrality. In the worst case, communities detected using the Louvain Algorithm can be badly connected or even disconnected. The Leiden Algorithm extends the Louvain Algorithm to guarantee well-connected communities by using partition refinement. This algorithm consists of three steps: 1) local moving of nodes, 2) refinement of the partition, and 3) aggregation of the network based on the refined partition (Traag et al., 2019). By refining communities, the algorithm has more room to identify high-quality partitions. The use of several community detection algorithms on a map helps to establish the stability of the communities, hence ensuring that they reflect fundamental properties of the empirical data instead of being an artifact of one specific method.

At the level of the whole network, four measures are commonly included (McGlashan et al., 2016; McGlashan, 2018). The density refers to the portion of relationships that exist compared to how many relationships are possible. Table 2 shows examples of several policy application areas and their corresponding map densities to show some common densities for these types of maps. We note that the density in this application area is generally very low (i.e., we have sparse graphs). This property can be important for other aspects of the analysis, such as guiding the choice of centrality measures (Freund and Giabbanelli, 2022). Second, distributions for the in- and out-degrees of all nodes provide some insight into the general structure of a model. Unlike other fields, the intent in systems maps for policy-making is not typically to fit a certain distribution (e.g., test for a power-law). Rather, the examination tends to focus on three categories of degrees (Eden et al., 1992; Alomia-Hinojosa et al., 2022; Konti and Damigos, 2018; Giles et al., 2007): nodes without incoming edges (i.e., sources or “transmitters” with in-degree 0), which often constitute parameters of the model; nodes without outgoing edges (i.e., sinks or “receivers” with out-degree 0), which can serve as outputs; and hubs (large in- or out-degree), which can be prime targets for intervention planning due to their high connectivity. The Average Path Length is defined as the average of the shortest paths between each pair of nodes. It relates to the amount of effort that is needed to spread change, on average. Finally, the diameter measures the furthest distance between two nodes of the graph. Intuitively, a larger diameter indicates that a systems map has a wide “span” by covering far-away topics. The diameter has been extensively studied across various types of systems maps Strautmane (2012); Maksimenkova et al. (2018); Barbrook-Johnson and Carrick (2022).

A network model can be built for different objectives. Small-scale models often serve to estimate the (direct and indirect) effects of a small set of variables of interest onto a suicide-related construct. For example, Kranzler et al. (2016) used an 11-node model to examine how self-injury and emotion dysregulation contribute to suicide attempt and Tang et al. (2010) used a six-node model to estimate how mental health constructs (e.g., post-traumatic stress disorder) ultimately shape the risk of suicide. In contrast, detailed models intended for intervention planning and evaluation have a much larger number of constructs and associated edges. Owing to their level of details and the significant investment required to create them via participatory modeling efforts (Section 2.3), there are fewer such models. For each of the four models examined in this paper, we obtained a directed network either directly from a file provided by the authors in a repository or supplementary materials (NCIPC and Brenas et al. (2019)), or by manually re-creating all edges and nodes from the authors’ figures (Chung, 2016; Page et al., 2018). Using the Python library NetworkX (version 2.6.2), we extracted characteristics for each of these four models at the node- and network-level (Section 2.4). Network-level characteristics include the number of nodes, number of edges, density, average path length, and degree distribution. Node-level characteristics include five node centrality measures: degree, Katz, Betweenness, Load, and Closeness. For each network, we focused on the top centralities, hence nodes were ranked by centrality.

The SEM is a common framework for ACEs and suicide prevention (Centers for Disease Control and Prevention’s National Center for Injury Prevention and Control, 2021; Cramer and Kapusta, 2017). This approach considers that suicide-related factors belong to four different levels: individual, relationship, community, or societal. Taking the example of (Shagle and Barber, 1995), suicidal ideation is shaped by personal coping strategies (individual), family and peers (relationship), the school environment (community), and religion (societal). The CDC offers a complementary perspective through its suicide prevention technical package of policy, programs, and practices (Stone et al., 2017). In the technical package, concepts are classified into seven prevention strategies and approaches (Stone et al., 2017, p. 12–44), with an associated “shorthand notation” introduced here: strengthen economic supports (“economic”), strengthen access and delivery of suicide care (“care”), create protective environments (“environments”), promote connectedness (“connectedness”), teach coping and problem-solving skills (“coping”), identify and support people at risk (“support”), and lessen harms and prevent future risk (“prevent”).

In this study, we categorized the 361 nodes in the NCIPC Model per the SEM framework. This alignment is a qualitative endeavor: given the content of a model (i.e., the nodes’ names/labels), we seek to identify which aspect of a prevention strategy is covered and which level of the SEM framework is mobilized. Different individuals may interpret a model differently, which is a classic problem in reusing models (Freund and Giabbanelli, 2021). This creates the risk that findings reflect the particular interpretation of a group rather than the inherent characteristics of a model. To mediate this risk, we employ multiple coders from different backgrounds. By analyzing where their classifications concur, we can identify reliable categories. Specifically, we assigned nodes to four coders comprising two SMEs (in either suicide and/or ACEs) and two lay persons. Note that the non-SMEs and SMEs were not included in the same categorization groups and were instead split because of the differences in experience and knowledge in the specific areas of suicide and ACEs. Each coder assigned categories to each of their assigned nodes. They could disagree on a category; for example, a person may have seen “culture of secrecy” as a matter of environments and prevention strategies, while another may have seen it as environments and coping strategies. Coding conflicts were identified and resolved with a discussion between the coders, who explained the reasons for their classification and arrived at a consensus. In this example, one possible outcome could be the final categories of environments and coping. It was also possible that a consensus led to adding a category that was used by neither coder initially. This process has been documented in our online supplementary materials, with a “Nodes Additions” table devoted to categories that emerged as a result of inter-coder discussions. For a given node, the inter-coder agreement is defined as the fraction of categories assigned to the node that were endorsed by both coders. In our guiding example, since coders only shared the “environments” category out of three categories used in total, the inter-coder agreement is 13.

Complex problems such as adolescent suicide are addressed by myriad of actors, from the school (e.g., role models and after-school programs) to the community (e.g., psychologists and psychiatrists). Systems maps in other fields such as obesity have shown that the responsibilities of each actor may not fully align with the characteristics of the problem, thus leading to aspects for which nobody was clearly responsible, or aspects with shared responsibilities without clear communication lines. Cross-sector partnerships and whole-of-society models have thus emerged as multi-stakeholder approaches to policy-making in such problems (Dubé et al., 2014; Bekker et al., 2017), thus shifting from an actor- and problem-centric perspective (e.g., nutritionists and general practitioners for obesity) to a solution-centered approach (e.g., on themes such as body image and mental well-being). The decomposition of a map into themes is thus a key step for this paradigm shift, enabled by communication detection algorithms. Using the Python Library CDlib (version 0.2.5), we performed community detection on the NCIPC model using both the Louvain algorithm and the Leiden algorithm and compared their results to minimize the risk that communities are an artifact of one specific method. Each community was then named by manual inspection of its content and compared to key categories for suicide prevention from both the CDC suicide prevention technical package of policy, programs, and practices (Stone et al., 2017) and SEM framework (Centers for Disease Control and Prevention’s National Center for Injury Prevention and Control, 2021) (Section 3.2). The identification of communities helps find themes and coordinate multiple stakeholders to take and monitor interventions. The next step in using a system map is to evaluate the potential effect of such interventions (Mkhitaryan et al., 2020). Using the Actionable Systems tool (Giabbanelli and Baniukiewicz, 2018) to navigate networks, we show that the NCIPC model can support three types of interventions:

Identifying multiple paths, feedback loops, and rippling effects is essential to comprehensively assess the cost-effectiveness of an intervention and mitigate unintended consequences. These types of measurements are a systems science approach to network-level analysis, thus complementing classical network science metrics such an examination of the system’s dynamics.

Network statistics for the four models are summarized in Table 3. We note that they are all sparse both globally (low density) and locally (low clustering), indicating that the SMEs were highly selective in identifying relationships between nodes. The relatively slow increase in the diameter and average path length compared with the increase in the number of nodes as the models get larger (from left to right in Table 3) suggests that experts increased granularity pertaining to certain facets of suicide (e.g., ACEs) instead of expanding onto new domains. For example, the NCIPC model has six times as many nodes as Chung (2016), but about the same average path length and only a small increase in diameter (from 11 to 13). For the two largest models, we see that the average path length is very short relative to the network size³ so it is relatively easy for one factor of the model to affect another. From a participatory modeling viewpoint, nodes are thus generally related, and participants did not embark on long tangents with aspects that had little to do with other factors of the model. From a policy implication viewpoint, it also means that the existing system surrounding suicide is tightly interconnected, which likely necessitates the joint use of several intervention points to deliver a system-wide change. We also note that clustering is almost twice as large in the NCIPC model compared to the previous sizeable models. This higher clustering coefficient suggests that there are more tightly connected neighborhoods of nodes, which represent areas of related concepts. The clustering coefficient is thus further evidence that SMEs have delved further into suicide-related notions instead of expanding into previously uncovered themes. Distributions for in- and out-degree (Figure 4) are highly skewed in all models but Page et al. (2018), which is the smallest one with 18 nodes. Once models become sizeable, they clearly have a few “hub nodes” that attract a lot of attention and a large number of nodes with few connections. The identification of these hub nodes is thus of particular interest to examine the focal points of the models and their policy implications.

Centrality measures help identify and compare important nodes across models. We identified important nodes for each model by observing the highly ranked nodes across centrality measures. For example, in the NCIPC Model, ACEs were ranked first according to degree, load, and betweenness centrality and third according to Katz and closeness centrality. This process led us to identify ACEs, mental health disorders, suicide ideation, suicide attempts, and substance abuse as driving nodes in the NCIPC Model. Table 4 summarizes the top five nodes for all models with respect to five centrality measures (degree, betweenness, load, closeness, and Katz centrality); our online supplementary material includes tables listing up to the top ten nodes. In other models, prominent factors are sleep and alcohol disorders, capability and desire for suicide as well as attempts, distress, and past attempts (Brenas et al., 2019; Chung, 2016; Page et al., 2018).

At a high level, the top five nodes suggest that central factors across models are proximal to the individual, rather than community or societal constructs. This can be expected for network models, as their objective is to elucidate suicide of individuals, hence all contributing factors eventually point to the individual level. At a detailed level, three of the models have similar key constructs (NCIPC, Page et al. (2018); Chung (2016)): mental health (e.g., through disorders or distress) and suicide (re-attempts). Brenas et al. (2019) have more unique focal points due to emphasis on ACEs rather than suicide. Note that key concepts in Brenas et al. (2019) are not absent in larger maps such as the NCIPC map; rather, they are relatively less important in the wider scope of the model. For example, substance abuse is among the top-10 nodes for closeness centrality in the NCIPC model, hence reflecting the notion of substance use and misuse in Brenas et al. (2019).

When seeking to use highly central nodes for intervention design, it is important to note that such nodes may be the outcomes (i.e., cannot be directly changed) of the intervention or the leverage point (i.e., the action through which we change the system). For instance, suicide ideation or attempts are the outcomes that we wish to change, but we cannot directly change them. Leverage points that will ultimately affect these target outcomes would include firearm restriction, as it will lower access to lethal means and eventually prevent attempts. In the NCIPC model, the top five most central nodes per degree centrality are ACEs, suicide ideation, suicide attempt, mental health disorders, and poverty. The first three are target outcomes, while the latter two are potential intervention points. Since mental health disorders are also among the top five constructs per betweenness, we see that mental health is heavily involved throughout the journey of an individual from trauma to ideation and attempts. The complementary measures of centrality thus suggest that intervening on mental health disorders could have a high immediate effect (per the degree) and even more rippling effects (per betweenness). Ultimately, a network analysis alone cannot determine whether a highly central node is an outcome or a leverage point: this interpretation must be made by a SME. Other examples of leverage points suggested by the network analysis and domain expertise include addressing poverty (which has a high degree in the NCIPC model) and other social determinants of health (high degree in Brenas et al. (2019)) or improving housing and treatment for mental health disorders (which have high load centrality in Brenas et al. (2019) and the NCIPC model).

Figure 5 summarizes how the nodes of the NCIPC map were categorized by lay individuals (left) and experts (right) with respect to both the SEM framework and the CDC suicide prevention technical package of policy, programs, and practices. Each group had two coders. Inter-coder agreement among lay individuals was 73.6% on the social-ecological level and 48.3% on the prevention strategy. During resolution, the lay coders realized that the CDC suicide prevention efforts were not mutually exclusive, but highly overlapping approaches that examine the problem from all sides.

Both lay coders and SMEs agreed that a large proportion of nodes fell into two major prevention strategies (teaching coping skills and identifying and supporting people at risk) and two social-ecological levels (individual and relationship). The intersection of relationship-level issues being solved by learning coping skills makes up 23.55% of all concepts covered in the NCIPC model. In both the SME and lay heatmaps (Figure 5), there is a significant decrease in the number of nodes affected by societal-level frameworks, making up at most 7% of all concepts in the NCIPC model. This suggests that many of the interventions that would impact adolescent suicide come from family, friends, and the community. We find the inclusion of the lay coders to be important, as it helps display gaps in public knowledge and can suggest tailored communication initiatives to further educate community members with tools for suicide prevention.