A Protocol to Assess Contextual Factors During Program Impact Evaluation: A Case Study of a STEM Gender Equity Intervention in Higher Education

Details

• Alternative Title:
  Am J Eval
• Personal Author:
  Nobrega, Suzanne ; Edwards, Kasper ; Ghaziri, Mazen El ; Giacobbe, Lauren ; Rice, Serena ; Punnett, Laura
• Description:
  Program evaluations that lack experimental design often fail to produce evidence of impact because there is no available control group. Theory-based evaluations can generate evidence of a program's causal effects if evaluators collect evidence along the theorized causal chain and identify possible competing causes. However, few methods are available for assessing competing causes in the program environment. Effect Modifier Assessment (EMA) is a method previously used in smaller-scale studies to assess possible competing causes of observed changes following an intervention. In our case study of a university gender equity intervention, EMA generated useful evidence of competing causes to augment program evaluation. Top-down administrative culture, poor experiences with hiring and promotion, and workload were identified as impeding forces that might have reduced program benefits. The EMA addresses a methodological gap in theory-based evaluation and might be useful in a variety of program settings.
• Keywords:
  Case Studies Higher Education Impact Evaluation Qualitative Methods Theory-based Evaluation
• Source:
  Am J Eval. 45(4)
• Pubmed ID:
  39664915
• Pubmed Central ID:
  PMC11633285
• Document Type:
  Journal Article
• Funding:
  U19 OH008857/OH/NIOSH CDC HHSUnited States/ ; U19 OH012299/OH/NIOSH CDC HHSUnited States/
• Volume:
  45
• Issue:
  4
• Collection(s):
  CDC Public Access
• Main Document Checksum:
  urn:sha-512:d7360d5efde7164f6a6e4ca35175f08669ded95b55d8de26033846e73a3d63b739125c874edf047e70f5e9f05b4d28e89a90796b611450244556168a408f5234
• Download URL:
  https://stacks.cdc.gov/view/cdc/174798/cdc_174798_DS1.pdf
• File Type:
  [PDF - 562.97 KB ]