Coders transcribed interview responses from the interview guides into an Excel spreadsheet. To ensure accuracy, Spanish interview responses were translated to English by two bilingual CDC staff. We used a conventional content analysis approach to first review all responses to gain familiarity with the data and then inductively establish an initial list of codes for each interview question (Hsieh & Shannon, 2005). Where participants expressed multiple meanings for the images, each interpretation was coded separately. Additional comments made by participants were coded if they were considered an interpretation of or a recommendation to improve the image. Group interview responses were documented as group comments rather than individual responses, and therefore were coded once.

Intercoder reliability was assessed using Cohen’s kappa to measure the agreement between two independent coders. The Cohen's kappa coefficient values ranged from 0.72 to 1, with an average score of 0.84 indicating an almost perfect agreement (McHugh, 2012). Coders collaboratively discussed differences in coding decisions and reached a consensus to finalize the set of coded responses.

Codes were examined based on participant characteristics such as race and ethnicity (including non-Hispanic/Latino African American or Black [referred to as African American], non-Hispanic/Latino White [referred to as White], non-Hispanic/Latino Asian, Hispanic or Latino of any race, and persons of other race/ethnicity), age, gender, (including male, female, and persons of other genders), language spoken at home, and education level (including preprimary/kindergarten and below, primary/1st-6th grade, lower secondary/7^th-9^th grades, upper secondary/10th-12th grades, associate degree or vocational technical institute, bachelor’s degree, master or doctoral degree) with the aim of identifying any potential relationships or trends with the coded responses. By combining related codes, we were able to categorize participant responses into themes, which informed the development of features to consider for future educational material development.

A total of 74 adults in San Diego County, California (n = 37), the City of Miami, Florida (n = 21) and three groups of farmworkers in the City of Hidalgo, Texas (n = 5, 6, and 5) participated in this assessment. Fifty-one individual interviews were conducted in English and seven in Spanish; all three group interviews were conducted in Spanish.

Of the 58 participants interviewed individually, 34% self-identified as Hispanic/Latino: 16% as Hispanic/Latino White, 3% as Hispanic/Latino African American, and 3% as Hispanic/Latino Asian (see Table 1). Thirty-two percent of non-Hispanic/Latinos self-identified as African American, 16% as Asian, 12% as White, and 2% as American Indian or Alaska Native. Fifteen percent of participants self-identified as another race or preferred not to answer. Fifty-three percent of participants were born in the United States, 10% in Mexico, and 10% in Haiti; the remaining participants were born in nine other countries. Over half of participants reported speaking only English at home, 16% Spanish only, 10% English and Spanish, and 13% English and another language. Less frequent were Haitian Creole (7%) and Vietnamese (2%) only speakers. More than half of participants (60%) were female. The largest group of participants (36%) had a bachelor’s degree and 14% of participants had a 10^th–12^th grade education level. Participants ranged in age from 18-80 years, with a mean of 40.

All group interview participants were born in Mexico, self-identified as Latino, and spoke Spanish at home. Other languages mentioned were English, Tzeltal, and Q’anjob’a (indigenous languages spoken in Latin America). Most participants self-identified their race as Indigenous [to Latin America] (50%) or Moreno (brown-skinned) (38%), a term used by some Mexican people to define Black ethnic identity (De La Torre, 2013). Group participants were aged 18-45 years, with a mean of 30; 15 participants were male, and one was female. Sixty-nine percent of participants had an education level of kindergarten or below, 19% had a 1^st-6^th grade education level, and 13% had a 7^th-9^th grade education level.

Each of the four image groupings were assessed by 22 individual interview participants and by one of the three interview groups, except for images S4-S6, which was not assessed in a group interview. Only three group interviews were conducted, using three questionnaires, leaving the fourth set of images unused in this setting. The meanings of most (63%) of the images and symbols were understood by at least 70% of participants; however, there were seven images that frequently caused misinterpretation. There were no apparent relationships or patterns found between participant characteristics (education, age, race and ethnicity, language spoken at home, or country of birth) and interpretation accuracy. Tables 2 and 3 summarize participants’ interpretations of each image and symbol assessed. Missing values were excluded, and number of responses varied by image.

When presented with the infographic containing images they assessed with corresponding captions, participants were asked for recommendations to improve the images. Forty-nine participants provided recommendations, even when they understood the intended meaning. Recommendations to improve the clarity of images from participants were coded and combined to create seven themes, listed in order of frequency. Representative quotes for each of the themes were selected and included below; image, gender and/or the race or country of birth are indicated in brackets.

Our findings have potential implications for future research and theory in visual messaging and health communication. These results emphasize the importance of detailed visuals with appropriate and relevant context to ensure image clarity and effectiveness, as underscored by the realism theory (Scheltema et al., 2018). The nuanced response to symbols and icons calls for further investigations into confirming consistent meaning across diverse populations (Houts et al., 2006).

Testing images with diverse groups, particularly images in materials for the general population, is important to increasing comprehension and actionability of important public health messaging (CDC, 2021). Comprehensive and collaborative methodologies that include interviews or focus groups of people with varying ages, genders, and cultural and linguistic backgrounds can be implemented (Dutta, 2008) to assess the clarity of images without text. Community-based partner organizations working with diverse audiences can be consulted for interpretation and translation services, testing facilitation, and recruitment.

Future research should consider interviewing participants from additional linguistic, racial, and cultural backgrounds and broader geographic scopes for enhanced generalizability of our findings. Furthermore, different theoretical frameworks, such as cognitive load theory (Sweller, et al., 1998), or dual coding theory (Paivio, 1991), can be explored to expand on how effective health communication materials with visuals should be presented.

Additional testing of images across different topics could provide a richer understanding of their universal effectiveness. Moreover, recommendations formed from this assessment could be applied and tested in images about lesser-known topics to further validate their applicability.

Several factors contributed to the successful implementation of this project. Collaborating with community-based partners enabled project staff to identify events for and locations frequented by populations that we were interested in interviewing. These partnerships also created trust for the interviewers in such locations versus approaching community members in unaffiliated public places. Participating partners were supportive in suggesting and making connections between the project staff and other organizations that could assist. Sharing the project plan and asking for referrals allowed us to expand the reach of this assessment.

Moreover, having translated the interview guide and providing access to native Spanish speaking interviewers allowed us to speak with Spanish speakers who had limited English proficiency, which ensured inclusion and enhanced the data quality. Additionally, the use of open-ended interview questions was successful for capturing nuanced perspectives.

Despite the strengths of the methodology and the relatively large number and diversity of participants for a qualitative method, this assessment had several limitations. Foremost, at the time of the assessment most people were familiar with COVID-19 symptoms and prevention messaging and could have guessed or had previous knowledge of the images’ intended meaning versus confirming the images’ clarity of content that is new to the participant. Additionally, because of the diversity of the convenience sample, there were not enough participants in population categories to identify patterns by education, age, race and ethnicity, language spoken at home, or country of birth for any of the images. We also included results from both group and individual interviews, that, while supplementary, were not balanced for a comparison. Further, the assessment was only conducted in three geographic areas and in English and Spanish, which limits the generalizability of the findings. Participants that also spoke another language may have understood interview questions or infographic text better and responded differently in interviews with materials in other languages.

To ensure images used in communication materials are understood by broadly diverse audiences in the United States, images should be understandable without text. This assessment revealed areas for improvement in images used during a public health response. While participants generally understood the images without text, some had different interpretations. Seven practical ways to improve image clarity and increase audience understanding were presented: 1) using realistic graphics, 2) exaggerated body position and actions, 3) using details to show image context and background, 4) using icons to encourage or discourage actions (as appropriate), 5) using symbols to show movement, 6) closely matching image to associated text, and 7) using consistent recommended behavior in each image. These themes can be applied by public health professionals in combination with CDC’s principles for developing inclusive public health communication and visual products. This includes showing diverse representation, appropriate cultural settings and dress, and including the intended population of focus in the decision-making process (CDC, 2023).

This assessment also supports the testing of images used in educational materials for general audiences with diverse population groups. Such a process will help to ensure the clarity of meaning of vital health communication messages to protect the health of the increasingly diverse population. Feedback from this assessment can be applied to future health communication materials, particularly during public health emergencies.