One of most important mandates of the National Institutes of Health’s (NIH) Clinical and Translational Science Awards (CTSA) is to accelerate the spread of scientific discoveries beyond academic health centers so that the public benefits from health related research.^1,2 The University of Michigan’s Michigan Institute for Clinical and Health Research (MICHR) exists to carry out the CSTA’s mandate. The overarching mission of the MICHR Community Engagement and Research Core (CERC) is to foster equitable and bi-directional research partnerships with those who will use scientific discoveries to benefit health. The CERC increases community decision-making and action for health promotion, disease prevention, and treatment by involving the community in all phases of clinical research planning, design, implementation, evaluation and dissemination.^3,4

Clinical researchers go through many years of training in order to be able to design and conduct rigorous clinical studies. However, the ultimate value of such studies is determined by their dissemination and application in communities to prevent and treat illness. Communities often include people with many varying cultures. Many of which are not part of the dominant culture such as underserved and economically disadvantaged groups, ethnic and racial minority groups, and marginalized groups (e.g., lesbian, gay, and transgendered groups). Such communities may have valuable, but often-muted insights about causes of health problems and viable solutions to those problems. To have significant impact the results of clinical research must pass muster at both the scientific and community level. It is hoped that over time that the interaction of the academic/science culture with community cultures will result in a shared perspective across cultures that will enhance the translation of health related research into improved health.

MICHR’s Scientific Review Committee (SRC) reviews multiple pilot applications as part of the overall Pilot Grant Program (PGP), including the Community University Research Partnership (CURES) pilot applications. When SRC reviews are available, they are shared with the CECC but only the CECC’s preliminary scores have been submitted. CECC review criteria are distinct from the SRC, which structures review based on the standard NIH criteria. Instead, CECC reviewers assess applications for the quality of the community-academic partnership, level of community participation, equitable distribution of budget between community and academic investigators, and relevance of the research topic to the community, as well as a project’s potential for securing external funds for a larger study. While proposals with poor SRC scores are typically not considered for funding, final award decisions are based on final CECC review scores.

Although many CECC members were comfortable reviewing the pilot study applications a subset of our community partners felt uneasy reviewing university/community partner research projects. They felt that they did not possess the expertise or experience to judge the merits of scientific research grant applications. On occasion these members felt uncomfortable assigning a proposal a score that was significantly at odds with the score given by the SRC. This was unfortunate because the community perspective typically adds essential insights that will ultimately drive the dissemination and sustainability of a research study’s benefit to human health.

These guidelines were developed in partnership with our CECC during the process of reviewing community engaged research proposals. They are intended to help community partners bring to bear their unique expertise and experience in the evaluation of proposed research studies that include community partners as co-principle investigators. These guidelines offer an approach for ensuring that the community’s voice is heard when pilot project proposals for community-engaged research are developed and reviewed. A discussion of these guidelines could be used to orient new members of a community council. They could also be adapted to better serve specific needs of various community engagement councils.

After thinking about and discussing the discomfort expressed by some of the community members on our CECC about reviewing scientific research proposals, we realized the discomfort was caused (or at least exacerbated) by the fact that the academic partners, had not provided specific guidelines and/or support to CECC community members regarding their role in reviewing community based pilot study applications. After considering the issue we realized that when evaluating pilot studies the expertise possessed by our community partners relates primarily to the feasibility of conducting the study in their communities rather than the scientific merit of the proposal. Bringing this expertise to bear on pilot study reviews is essential if these pilot studies are to succeed in contributing to the health of communities locally and nationally.

This led us to develop a draft set of community guidelines (Table 1) organized under the headings used in the guidelines NIH provides to members of its study sections. The guidelines were rewritten to match the unique experience and expertise community partners bring to the review process. The guidelines were reviewed and revised based on input from the entire CECC, as well as by three groups of community partners, i.e. from the CECC’s three focal communities and then again by the entire CECC. The guidelines in Table 1 have incorporated to the extent possible the suggestions and advice of the community members who reviewed them. The answers to the questions in Table 1 were designed to empower our community partners to make more informed judgments when reviewing and scoring engaged community research grant applications.

CURES pilot study applications are submitted through the pilot grant program’s online application submission system and are reviewed by MICHR’s Scientific Review Committee (SRC). They are also sent to the CERC for review by the CECC. CERC program staff members work with CECC members to identify one lead reviewer and one secondary reviewer for each proposal who are responsible for presenting a summary score and critique at the pilot study review meeting. Prior to the review meeting all CECC members submit preliminary scores and comments on key strengths and weaknesses for each proposal. CECC uses the NIH’s 9-point scoring scale to assess proposals for the criteria listed in Table 2.⁵

All CECC members receive compiled scores prior to the pilot study review meeting. They also receive the scores and critiques from the SRC prior to (depending on the timing of the two review processes) the CECC review meeting. The final CECC scores are averaged together (including the range) and the SRC scores are also averaged together (including the range) for each submission.

After preliminary scoring, the CECC uses a review meeting format similar to NIH study sections, where the primary and secondary reviewers present their critiques. If an agreement between SRC and CECC scores is reached then the application is either awarded or not awarded. If there is a disagreement between SRC and CECC scores, the CECC takes into consideration if there are any moderate to significant scientific weaknesses in the application. However, CECC scores are used to make the final funding decision. CECC and SRC scores and comments are shared back with all applicants. Applicants may then consult with CERC staff on addressing reviewer critiques for potential resubmission.

These guidelines can be used to increase the expertise and effectiveness of community reviewers, who have a unique (i.e. community) perspective. They allow community partners not trained in the scientific method to have a viable and valued voice in the academic paradigm of scientific review. Informed community input is essential if engaged community research is to be successful.

A final note, an appendix explaining common research designs and a glossary of terms used by the scientific community is included with this article. These materials have been extensively revised to improve their clarity and accessibility.