Optimizing Measurement Reliability in Within-Person Research: Guidelines for Research Design and R Shiny Web Application Tools

Details

• Personal Author:
  Huang P-H ; Nguyen A ; Wang W ; Yang L-Q
• Description:
  Within-person research has become increasingly popular over recent years in the field of organizational studies for its unique theoretical and methodological advantages for studying dynamic intrapersonal processes (e.g., Dalal et al., Journal of Management 40:1396-1436, 2014; McCormick et al., Journal of Management 46:321-350, 2020). Despite the advancements, there remain serious challenges for many organizational researchers to fully appreciate and appropriately implement within-person research-more specifically, to correctly conceptualize and compute the within-person measurement reliability, as well as navigate key within-person research design factors (e.g., number of measurement occasions, T; number of participants, N; and scale length, I) to optimize within-person reliability. By conducting a comprehensive Monte Carlo simulation with 3240 data conditions, we offer a practical guideline table showing the expected within-person reliability as a function of key design factors. In addition, we provide three easy-to-use, free R Shiny web applications for within-person researchers to conveniently (a) compute expected within-person reliability based on their customized research design, (b) compute observed validity based on the expected reliability and hypothesized within-person validity, and (c) compute observed within-person (as well as between-person) reliability from collected within-person research datasets. We hope these much-needed evidence-based guidelines and practical tools will help enhance within-person research in organizational studies. [Description provided by NIOSH]
• Subjects:
  Occupational Health Safety
• Keywords:
  Analytical Processes Author Keywords: Within-person Research Level-specific Alpha Occupational Psychology Psychological Testing Reliability R Shiny Web Application Statistical Quality Control Validity Work Organization
• ISSN:
  0889-3268
• Document Type:
  Text
• Funding:
  Grant
• Genre:
  Journal Article
• Place as Subject:
  New York ; Oregon ; OSHA Region 10 ; OSHA Region 2
• CIO:
  National Institute for Occupational Safety and Health (NIOSH)
• Topic:
  Workplace Safety & Health
• Location:
  North America
• Volume:
  37
• Issue:
  6
• NIOSHTIC Number:
  nn:20067941
• Citation:
  J Bus Psychol 2022 Dec; 37(6):1141-1156
• Contact Point Address:
  Liu-Qin Yang, Department of Psychology, Portland State University, P.O. Box 751, Portland, OR, 97207, USA
• Email:
  lyang@pdx.edu
• Federal Fiscal Year:
  2023
• Performing Organization:
  Portland State University
• Peer Reviewed:
  True
• Start Date:
  20050701
• Source Full Name:
  Journal of Business and Psychology
• End Date:
  20250630
• Collection(s):
  National Institute for Occupational Safety and Health
• Main Document Checksum:
  urn:sha-512:b428bbec1ebfa5b54178b273aa4841b850249ac215bdc8916951c0664946c74bc793a4c4e418c8fb0294051aabf1c70bec61e25917ef28f517f696ca6d72986c
• Download URL:
  https://stacks.cdc.gov/view/cdc/230174/cdc_230174_DS1.pdf
• File Type:
  [PDF - 1.03 MB ]