Medical researchers have an ethical and legal obligation to thoroughly inform research participants about studies for which they volunteer.¹ The informed consent process was developed to protect participants from harm, and promote informed decision-making.^{2, 3} Despite advances in research ethics and standardization of the informed consent process, many research participants remain misinformed about research terms, procedures and goals.^4-9

A meta-analysis of recent clinical trials measuring participant understanding of informed consent¹⁰ showed that 25-50% of research participants did not understand specific components of informed consent; estimates remained consistent over the last three decades.¹⁰ Poor comprehension of informed consent is coupled with misunderstanding of therapeutic aspects of clinical trials. Some research participants believe that research is done for their personal advantage, rather than for generalized knowledge or future patients’ benefits.⁷

Several tools have been developed to assess comprehension of informed consent and the research process.¹¹ However, few have been validated,^11-13 and their effectiveness, unexplored.¹¹ Among existing scales, none addressed the concepts we were interested in or were developed for diverse groups. Existing scales often measure only certain aspects of health-related research such as understanding informed consent, or therapeutic misconception. To our knowledge, there is a paucity of instruments that assess comprehensive understanding of health-related research, a significant concern when performing studies among vulnerable and diverse populations. Given the need to elucidate knowledge gaps among diverse research participants, validated surveys that assess comprehension in a literacy and culturally sensitive manner are essential.

We propose research literacy as a comprehensive concept incorporating individuals’ understanding about the goals and nature of health-related research with informed decision-making in research participation.¹⁴ We define research literacy as “the capacity to obtain, process, understand, and act on basic information needed to make informed decisions about research participation.” Our definition, adapted from the U.S. Surgeon General’s definition of health literacy, ¹⁵ was developed using mixed-method approaches with lay and expert participants. We sought to develop a novel scale, the Research and Knowledge Scale (RaKS), to assess general understanding of research by prospective research participants and the public, in a manner sensitive to diverse cultural backgrounds and literacy levels. This manuscript describes the development and psychometric properties of the RaKS.

Mandates from the National Institutes of Health, the Food and Drug Administration, and the National Commission for the Protection of Human Subjects of Biomedical and Behavioral Research, articulate the necessity of processes and methods to insure that participants in health-related research studies have a clear understanding of the studies they participate in and are able to make informed, un-coerced decisions about participation.^24-28 The RaKs is responsive to such mandates and was developed to measure individuals’ “capacity to obtain, process and understand basic information needed to make informed decisions about research participation.”

Our findings support the preliminary internal consistency reliability and validity of the RaKS as a tool to assess individual understanding of health-related research procedures and expectations. The good internal consistency estimate (KR-20 =0.81) and test-retest reliability (r=0.84) for the RaKS suggest that the 16 items comprising the scale collectively form a consistent and preliminarily reliable measure of research knowledge and understanding. Our exploratory factor analysis results suggest that the RaKS is unidimensional, and that all 16 items assess an aspect of one’s understanding of health-related research.

We examined preliminary construct validity by evaluating mean research literacy scale score across socio-demographics. The scale demonstrated high reliability within demographic subgroups (Table 2). While no gender differences were observed, our hypothesis of difference in mean scores by demographic subgroups was otherwise supported. Scores varied across race/ethnicity (mean research knowledge score: 12.3 vs. 11.3 vs. 9.9, Non-Latino Whites, Blacks, and Latinos, respectively). These differences in scores may speak to additional broad drivers of race-related treatment and racial discrimination both within greater society,^29-31 and specifically within the healthcare sector. ^32-34 Such experiences may impact the way minorities perceive and interact with the healthcare system,^{32, 34, 35} and thus their underlying knowledge as healthcare consumers. Thus, experiences of race-based treatment and racial discrimination may shape research knowledge and understanding.

Research understanding and knowledge was also lower among those with lower education and health literacy levels, consistent with literature that indicates level of education and health literacy proficiency are associated with generally better-informed healthcare consumers.^36-40 Research knowledge was also higher among older participants, particularly over age 50. Plausible explanations may point to prolonged exposure to the healthcare system throughout the lifespan, or increased e-health literacy (use of internet and social media to locate and evaluate health information) and health consumerism in this age group.⁴¹

The variations in scores across socio-demographic subgroups, demonstrates the potential of the RaKS to discriminate differences across substrata within the general population. Differences in levels of understanding by demographics coincide with literature deeming characteristics like race/ethnicity, as traits associated with individuals less likely to participate in health-related research.^{6, 42-44}

Misperceptions about health-related research may deter racial/ethnic minorities and individuals of low socioeconomic status from participating in research.⁴⁴ With growing racial/ethnic, and socioeconomic diversity,⁴⁵ researchers need to engage broad groups of potential participants by ensuring communication is clear and effective. The scale holds promise as a potential screener to verify participants’ understanding of research expectations and procedures prior to study enrollment. We envision the RaKS administered within a research setting by research assistants to prospective research participants before obtaining informed consent, or within community settings as a baseline assessment of how well individuals understand research, for future interventions. Such interventions could result in increased engagement of diverse populations.

Our findings should be viewed within the context of certain limitations. First, the RaKS was administered as a cross-sectional survey at one time-point (except for test-retest reliability participants). We cannot draw definitive longitudinal conclusions about research knowledge or research literacy as either a trait or state. Plausibly, research literacy is similar to health literacy—a trait for which proficiency is hypothesized to be context- and situation-specific.⁴⁶ But the overarching concept of research literacy should be viewed as separate from health, scientific, and general literacy. The world of health-related research has very specific goals, jargon, and outcomes. So understanding the multiple facets of health-related research requires knowledge specific to these nuances, distinguishing research literacy as a separate yet necessary concept. We observed a ceiling effect indicating the RaKS’ limitations for further distinction of relatively well-informed respondents, with very high scores. Conversely, this ceiling effect emphasizes the tool’s potentially strong ability to identify individuals who score lower and therefore struggle with understanding health-related research information—which is its purpose. Second, it is possible that there are context-specific facets to participation in health-related research that we were not able to assess through the RaKS. The focus group participants in the formative phase of survey development had been involved in survey studies, behavioral intervention studies and community-based studies. It is possible that their answers about research were colored by the nature of the research in which they participated. We recognize that we could not accommodate the unique aspects of the range of health-related research studies, so we chose to focus on core understanding essential to being an informed research participant, regardless of type of study in which one may choose to participate. Further work on this topic might include subscales specific to research literacy for different types of research studies. Third, we did not assess the sensitivity of the RaKS to change, in the context of an intervention. This is a topic that warrants future investigation. Fourth, we recognize that the vocabulary and reading level of the items included in our scale may be rather sophisticated. We completed comprehensive cognitive pre-testing and pilot phase testing with diverse lay community members to address this. Yet some of the challenges with the reading and vocabulary level of the scale relate directly to the very jargon and vocabulary that researchers use to communicate about research. This adds further credence to the necessity for a concept such as research literacy, which may prompt researchers to recognize the bidirectional communication skills needed to work and communicate effectively with research participants. Finally, the potential for bias borne from self-report and guessing are a threat to any psychometric self-administered assessment. Participants were asked to respond to items in the RaKS regarding information that they may have either been exposed to in the past, or never known. We cannot guarantee that guessing did not occur.

The RaKS attempts to evaluate how well individuals process and understand health-related research. To our knowledge, the scale is the first of its kind to: 1) evaluate the concept of research literacy in a diverse sample, 2) rely on both qualitative and literature findings and conceptual grounding as the basis for defining and measuring research literacy, and 3) incorporate the perspectives both of former/current research participants and researchers in its development. Research literacy is a new and dynamic concept that considers how individuals process and understand written and verbal information necessary for making an informed decision about initial and ongoing health research participation. Since the processing of written and verbal information are both underlying tenets of literacy, the term research literacy is an appropriate concept to capture this topic.

The RaKS is a tool that could be used for screening to better facilitate research participants’ understanding prior to consenting to a study. Both the domain of research knowledge and understanding and its accompanying scale are foundational elements of research literacy, created and defined through our study. Our study should prompt continued investigation to uncover other domains and components of the broader concept, research literacy. Future research should explore Rasch modeling to further refine the scale, whether levels of research literacy are associated with willingness to participate in research, and seek to expand upon operationalization and application of the concept of research literacy. The RaKS has the potential to foster transparency towards long-term improvements in engaging and communicating with research participants.