A Protocol to Assess Contextual Factors During Program Impact Evaluation: A Case Study of a STEM Gender Equity Intervention in Higher Education

Details

• Personal Author:
  Edwards K ; El Ghaziri M ; Giacobbe L ; Norbrega S ; Punnett, Laura ; Rice S
• Description:
  Program evaluations that lack experimental design often fail to produce evidence of impact because there is no available control group. Theory-based evaluations can generate evidence of a program's causal effects if evaluators collect evidence along the theorized causal chain and identify possible competing causes. However, few methods are available for assessing competing causes in the program environment. Effect Modifier Assessment (EMA) is a method previously used in smaller-scale studies to assess possible competing causes of observed changes following an intervention. In our case study of a university gender equity intervention, EMA generated useful evidence of competing causes to augment program evaluation. Top-down administrative culture, poor experiences with hiring and promotion, and workload were identified as impeding forces that might have reduced program benefits. The EMA addresses a methodological gap in theory-based evaluation and might be useful in a variety of program settings. [Description provided by NIOSH]
• Subjects:
  Education Occupational Health Safety
• Keywords:
  Author Keywords: Impact Evaluation Case Studies Higher Education Intervention Qualitative Methods Theory-based Evaluation
• ISSN:
  1098-2140
• Document Type:
  Text
• Funding:
  Cooperative Agreement
• Genre:
  Journal Article
• Place as Subject:
  Boston Region [OSHA] ; Massachusetts
• CIO:
  National Institute for Occupational Safety and Health (NIOSH)
• Topic:
  Workplace Safety & Health
• Location:
  North America
• Volume:
  45
• Issue:
  4
• NIOSHTIC Number:
  nn:20067662
• Citation:
  Am J Eval 2024 Dec; 45(4):592-609
• Contact Point Address:
  Suzanne Nobrega, University of Massachusetts Lowell, One University Avenue, Falmouth Hall 305, Lowell, MA 01854
• Email:
  Suzanne_Nobrega@uml.edu
• Federal Fiscal Year:
  2025
• NORA Priority Area:
  Services
• Performing Organization:
  University of Massachusetts, Lowell
• Peer Reviewed:
  True
• Start Date:
  20060701
• Source Full Name:
  American Journal of Evaluation
• End Date:
  20210831
• Collection(s):
  National Institute for Occupational Safety and Health
• Main Document Checksum:
  urn:sha-512:132a9a14b9a123d78c8b274c6870c48235346d9a296cb655d66c56b2a5eccf33ffa5d4841885715833f59a938674ac5923f948ac16be3b184393ad7a6287bd5c
• Download URL:
  https://stacks.cdc.gov/view/cdc/229931/cdc_229931_DS1.pdf
• File Type:
  [PDF - 790.34 KB ]