From December 2008 through November 2010, TBTC sites enrolled participants into an international, open-label, randomized Phase 2 clinical trial assessing the safety and activity of rifapentine, a long-acting rifamycin antibiotic, in comparison to the standard drug rifampin, during the first 2 months of treatment for pulmonary tuberculosis (TB) (Dorman et al. 2012).

As required for all clinical trials, participants who agreed to enroll were asked to provide informed consent, which was based on the elements of 45 CRF 46 (Common Rule) and 21 CFR 50 (U.S. Department of Health and Human Services 2009; 2013). Between August 2009 and November 2010, participants enrolled in the clinical trial were asked to answer ICAT questions to measure comprehension and satisfaction with the informed consent process. Site staff chose study participants for ICAT administration by convenience. ICATs were administered only to persons who already had enrolled in the clinical trial. A volunteer staff member at participating sites also completed an open-ended questionnaire about their site-specific consent process. All site staff members enrolling in the clinical trial were invited to participate in this evaluation.

TBTC Study 29, the clinical trial within which this project was conducted, was approved in May 2008 by the institutional review boards (IRBs) of the CDC and participating institutions, and was registered with ClinicalTrials.gov (Study 29: NCT00694629). Some local IRBs ceded oversight of the TBTC protocols to the CDC’s IRB (U.S. Department of Health and Human Services 2009). For this project, the CDC’s National Center for HIV/AIDS, Viral Hepatitis, STD, and TB Prevention determined that the assessment of the consent process constituted a program evaluation and did not require IRB review (Centers for Disease Control and Prevention 2010). Additionally, each site followed local review policies and procedures.

The original Informed Consent and Authorization Assessment Tool (ICAAT) was developed and validated by Breese et al. (2007) to assess participants’ comprehension and satisfaction with the consent process. The ICAAT contained 28 items to determine how persons from diverse backgrounds experience the consent process. It included 21 items to assess comprehension of representative required elements of informed consent and seven items to assess satisfaction with the process (Breese et al. 2007). The ICAAT was designed to be self-administered with three response options for each question (“Agree,” “Not Sure,” and “Disagree”) and included one set of paired questions that required opposite answers to assess the consistency of responses. For subsequent uses, questions pertinent to the Health Insurance Portability and Accountability Act (HIPAA) were deleted (Breese, personal communication) and the tool was renamed the Informed Consent Assessment Tool (ICAT) (Breese et al. 2007).

For this evaluation, the ICAT questions were adapted from a written, self-administered format to a verbal, interviewer-administered format in an effort to reduce barriers related to low literacy (Supplement 1). The ICAT was simultaneously translated into the participant’s language by the ICAT administrator, if needed. Study staff members, other than those who had administered informed consent, administered ICATs.

The adapted ICAT used in this evaluation contained 15 questions assessing comprehension of required consent components and nine evaluating participant satisfaction with the consent process (Supplement 1). Three response options (“Yes,” “No,” and “Not Sure”) were provided for each item. “Not sure” responses were grouped with the incorrect/unfavorable responses for analysis. Three questions were repeated in slightly modified versions to measure response consistency (Q.3 and Q.11; Q.6 and Q16; Q.10 and Q.17). ICATs were administered once during the 8-week intensive study phase when participants underwent clinical evaluations every 2 weeks. Participant scores were calculated based on the percentage of correct responses for comprehension (out of 15) or favorable responses for satisfaction (out of nine). ICAT scores equal to or greater than 80% were considered high.

To assess the informed consent process at each site, researchers created a 27-item, open-ended questionnaire, based on the informed consent process used at the TBTC study site in Kampala, Uganda (Supplement 2). Questionnaire items included inquiries about site staff who administered the consent process, the use of interpreters, time spent administering consent, the number of visits to complete the consent process, and the approaches for obtaining consent from persons with low literacy. Additional items addressed approaches to assess participant understanding such as the use of short forms or translated forms, the use of visual aids, the presence or absence of family and friends during the consent process, and protocols related to a participant signing the form. Suggestions for improving the informed consent process were also solicited.

Comprehension and satisfaction scores at the site in Kampala, Uganda, were compared to other sites with the Wilcoxon rank-sum test (Hollander, Wolfe, and Chicken 2013); a p-value <.05 was considered statistically significant (SAS v. 9.3, SAS Institute, Cary, NC). Medians and nonparametric tests were used for a more conservative approach in this evaluation.

Answers to the site consent process questionnaire were evaluated by identifying common themes among the responses, tabulated for comparison, and reviewed by three of the investigators. Finally, among sites where both informed consent practices and ICATs were reported, median ICAT scores were compared by reported consent practices to determine which practices were related to higher comprehension and satisfaction scores.

ICATs were collected by site staff in 2010 at one site in Spain (10 ICATs), one site in Uganda (30 ICATs), and five sites in the United States (14 ICATs). Participants (Table 1) were primarily male (76%) with a median age of 27 years (interquartile range [IQR]: 24–42), with the majority having a 12th-grade education or less (n = 42, 78%). Seventeen percent (n = 9) reported that they had at least some college education; 6% (n = 3) reported education as “other,” with no further information.

ICATs were completed in the same language in which participants had discussed informed consent, with the possible exception of one participant for whom the language used to administer the ICAT was not specified. The majority of ICATs from the Kampala, Uganda, site were completed on the day of enrollment, whereas at other sites, ICATs were completed an average of 15 days after enrollment.

The median ICAT scores were 80% (IQR: 67–93) on the comprehension scale and 89% (IQR: 78–100) on the satisfaction scale. The median comprehension score for participants in Kampala (87%, IQR: 80–93) was higher than the median for those enrolled at other sites (64%, IQR: 57–87) (p = .002 by the Wilcoxon rank-sum test); this was also true for satisfaction scores with 100% (IQR: 89–100) as the median score for those enrolled in Kampala and 78% (IQR: 67–100) for those enrolled at other sites (p = .017). For the three paired ICAT questions, 67% (36/54) of the responses were consistent for comprehension questions 3 and 11, and 82% (44/54) of the responses were consistent for comprehension questions 6 and 16. For the satisfaction questions 10 and 17, 87% (47/54) of the responses were consistent.

For seven of the 15 ICAT comprehension questions, the overall proportion of correct or favorable responses was less than 80% (Table 2). Fifty-seven percent of individuals were unclear about whether their private information could be seen by any medical person (ICAT Q.9), and 22% were unclear that their names could not be used in presentations of the study findings (Q.14). In responding to the paired questions (Q.6 and 16) about whether study staff could contact friends or family without their approval, 48% and 33% did not reply with a correct or favorable response. At least a quarter of participants did not demonstrate a clear understanding that they could terminate study participation at any time (Q.11: 26%) and that they could receive medical care elsewhere to treat their disease (Q.22: 35% and Q.24: 28%). For the satisfaction questions, almost all questions had a favorable response (≥80%), except for one question where one-third of the participants were unsure or agreed that there was too much information in the consent forms (Q.2).

The overall median ICAT score for comprehension for those reporting an eighth-grade education or less was 67% (IQR: 60–87); participants with more than an eighth-grade education but no college scored 80% (IQR: 70–93); and those with at least some college scored 87% (IQR: 93–100). For satisfaction, the median score for those with at least some college education (IQR: 100–100) was 100%, versus 89% (IQR: 78–100) for those with a high school education or less.

Staff members from 14 sites in five countries (Canada [n = 1], South Africa [n = 2], Spain [n = 1], Uganda [n = 1], and the United States [n = 9]) completed the consent process questionnaire (Table 3). At 13 of the sites, the study coordinator was reported to be one of the persons who administered informed consent. Staff members from 12 sites reported not using a generic short consent form, and two of these stated that they are not permitted to do so by their respective IRBs (U.S. Department of Health and Human Services 2009). Nearly all site staff surveyed (n = 13) reported that they allowed patients to defer their enrollment decision until a later follow-up visit.

Site staff reported using different procedures for obtaining consent. For example, staff members from two sites read the consent form along with the patient; one had the patients silently read the form to themselves. Depending on the patients’ needs, staff members at three sites would read the consent form aloud to the patient. The length of time site staff members reported for administering consent ranged from less than 1 hour to up to 3 hours.

When asked to provide their overall impressions of the consent process, staff members from 13 of the 14 sites reported on concerns expressed by study participants. Staff members from six sites shared 11 different concerns related to risks and safety (e.g., possible drug side effects and toxicity). Staff members from three sites reported six different concerns related to study and treatment procedures.

When asked to provide suggestions for improving the consent process, staff members from seven sites recommended shorter documents and three suggested creating simpler, more understandable consent forms. Additional suggestions included ensuring that the administration of the consent process includes adequate time for patients to consider their decisions from three site staff members, and the use of supplementary materials such as explanation booklets, brochures, posters, or videos by eight site staff members. Site staff members also proposed utilizing a questionnaire to assess participant understanding at the time of requesting consent, providing a setting free from interruptions during consent administration, and providing on-going education on consent topics.

Six of the sites completed both the consent process questionnaire and the administration of ICATs (n = 44). Three of the consent process questionnaire items and the corresponding ICAT scores from Table 3 are provided here. The first explores the question of whether the Kampala site’s two-step process might be responsible for their higher comprehension and satisfaction results. The other two explore approaches that might be most modifiable in the informed consent process.

Limitations of this study included low and uneven participation in the consent evaluation across study sites. This can be attributed to some sites not being staffed for activities peripheral to the primary study protocol, investigators being skeptical of the value of evaluating their informed consent processes, and differences in TB prevalence and research capacity.

The Uganda site enrolled approximately 35% of the participants in the clinical trial (Dorman et al. 2012). Of the 54 participants who volunteered to complete an ICAT, 30 (56%) were from the Uganda site. The participants from Uganda are not representative of participants at other sites.

The revised ICAT used in this evaluation was based on a self-administered form validated by Breese et al. (2007). After adapting the statements to be interviewer-administered, the tool was not revalidated, and participants may have responded to satisfaction questions in a socially desirable manner. In addition, the ICAT was only provided to the participating sites in English and study staff simultaneously translated questions for some patients, which had the potential to introduce variability.

Additional biases may have been introduced through the use of convenience sampling, variability among the staff members who administered consent, the possibility that staff members inadvertently led participants in responding to ICAT questions, and the time between when informed consent was conducted and when the ICAT was administered. Furthermore, with each site submitting a single questionnaire, variability in administrative practices amongst multiple staff members at a site may not have been fully captured. Finally, not all sites that reported consent practices administered ICATs, limiting opportunities to correlate practices with ICAT scores. These factors, along with a small sample size, limit the useful depth of analysis, precluding, for example, multivariate analysis.

Clinical trial sites expend considerable effort, striving for high comprehension of informed consent principles among trial participants. The findings of our pilot evaluation present opportunities to improve the consent process and the assessment of the process itself, particularly more focused training, including the need to reemphasize specific components of the Common Rule and FDA 21 CFR 50 multiple times throughout a study (U.S. Department of Health and Human Services 2009; 2013).

Observers have speculated that a tool similar to the ICAT could be used throughout the study to reemphasize the rights and protections afforded to study participants. Adapting an ICAT for patient-level, real-time use would require a development and validation process. A more immediate course of action could include training site staff members to routinely assess participants’ comprehension both during and after presentation of specific components of the Common Rule and FDA 21 CFR 50 and evaluating the impact of the educational intervention.

Our pilot evaluation provides insights for a more robust evaluation of the consent process in future multicenter clinical trials. Specifically, it may be beneficial to incorporate the evaluation of the consent process into the main study protocol to ensure all sites’ participation; dedication of resources to evaluation activities; and the systematic collection of data. Additionally, it may be advantageous to develop a tool to quantify site-specific consent practices to help pinpoint specific practices associated with higher comprehension and satisfaction.

As seen in this pilot evaluation, further investigation is needed to identify reasons for the Uganda site’s higher comprehension and satisfaction scores. While this evaluation supported CRAG’s hypothesis of comprehension and satisfaction being higher at the TBTC site in Uganda, it did not support the additional hypothesis that a two-part informed consent process would result in higher comprehension among participants outside of Uganda. A median satisfaction score of 100% from participants surveyed at the Ugandan site is impressive, yet further evaluation is needed to determine whether the following influenced the score: cultural factors, the influence of a staff-administered satisfaction survey on patient responses, patients’ perceptions of the care available in a research setting compared to regular program care, the time period between enrollment and ICAT administration, length of consent process, and median age of the participants.

The IRBs at many sites declined to take advantage of the short consent form as permitted by the Code of Federal Regulations (U.S. Department of Health and Human Services 2009; 2013). Future evaluation efforts of the consent process could include using a long version of the consent form as the “written summary of what is to be said” coupled with use of the short form for all participants. The short form could also be used throughout the study to reinforce important topics. When considering the benefits that may be derived from use of the short consent form, researchers must also consider additional resources that are needed. For example, per regulations, another individual may be needed to witness the oral presentation (bU.S. Department of Health and Human Services 2011b).

Finally, while ethical regulations stress the importance of comprehension, participant satisfaction with the informed consent process also deserves consideration. The ICAT’s satisfaction questions focus on actions that, in theory, contribute to comprehension. This calls attention to the challenge of balancing these two domains in order to achieve a “successful” informed consent from both the participants’ and investigators’ perspectives.