A1 Journal article (refereed)
A cautionary note on the finite sample behavior of maximal reliability (2019)
Aguirre-Urreta, M. I., Rönkkö, M., & McIntosh, C. N. (2019). A cautionary note on the finite sample behavior of maximal reliability. Psychological Methods, 24(2), 236-252. https://doi.org/10.1037/met0000176
JYU authors or editors
Publication details
All authors or editors: Aguirre-Urreta, Miguel I.; Rönkkö, Mikko; McIntosh, Cameron N.
Journal or series: Psychological Methods
ISSN: 1082-989X
eISSN: 1939-1463
Publication year: 2019
Volume: 24
Issue number: 2
Pages range: 236-252
Publisher: American Psychological Association
Publication country: United States
Publication language: English
DOI: https://doi.org/10.1037/met0000176
Publication open access: Not open
Publication channel open access:
Publication is parallel published (JYX): https://jyx.jyu.fi/handle/123456789/64145
Abstract
Several calls have been made for replacing coefficient α with more contemporary model-based reliability coefficients in psychological research. Under the assumption of unidimensional measurement scales and independent measurement errors, two leading alternatives are composite reliability and maximal reliability. Of these two, the maximal reliability statistic, or equivalently Hancock's H, has received a significant amount of attention in recent years. The difference between composite reliability and maximal reliability is that the former is a reliability index for a scale mean (or unweighted sum), whereas the latter estimates the reliability of a scale score where indicators are weighted differently based on their estimated reliabilities. The formula for the maximal reliability weights has been derived using population quantities; however, their finite-sample behavior has not been extensively examined. Particularly, there are two types of bias when the maximal reliability statistic is calculated from sample data: (a) the sample maximal reliability estimator is a positively biased estimator of population maximal reliability, and (b) the true reliability of composites formed with maximal reliability weights calculated from sample data is on average less than the population reliability. Both effects are more pronounced in small-sample scenarios (e.g., <100). We also demonstrate that the composite reliability estimator for equally weighted composite exhibits substantially less bias, which makes it a more appropriate choice for the small-sample case.
Keywords: statistical methods; reliability (science)
Free keywords: reliability estimators; maximal reliability; composite reliability; sample size
Contributing organizations
Related projects
- Measurement and modeling practices in business research – problems and solutions
- Rönkkö, Mikko
- Research Council of Finland
Ministry reporting: Yes
Reporting Year: 2019
JUFO rating: 2