A1 Journal article (refereed)
A cautionary note on the finite sample behavior of maximal reliability (2019)

Aguirre-Urreta, M. I., Rönkkö, M., & McIntosh, C. N. (2019). A cautionary note on the finite sample behavior of maximal reliability. Psychological Methods, 24(2), 236-252. https://doi.org/10.1037/met0000176

JYU authors or editors

Publication details

All authors or editors: Aguirre-Urreta, Miguel I.; Rönkkö, Mikko; McIntosh, Cameron N.

Journal or series: Psychological Methods

ISSN: 1082-989X

eISSN: 1939-1463

Publication year: 2019

Volume: 24

Issue number: 2

Pages range: 236-252

Publisher: American Psychological Association

Publication country: United States

Publication language: English

DOI: https://doi.org/10.1037/met0000176

Publication open access: Not open

Publication channel open access:

Publication is parallel published (JYX): https://jyx.jyu.fi/handle/123456789/64145

Abstract

Several calls have been made for replacing coefficient α with more contemporary model-based reliability coefficients in psychological research. Under the assumption of unidimensional measurement scales and independent measurement errors, two leading alternatives are composite reliability and maximal reliability. Of these two, the maximal reliability statistic, or equivalently Hancock's H, has received a significant amount of attention in recent years. The difference between composite reliability and maximal reliability is that the former is a reliability index for a scale mean (or unweighted sum), whereas the latter estimates the reliability of a scale score where indicators are weighted differently based on their estimated reliabilities. The formula for the maximal reliability weights has been derived using population quantities; however, their finite-sample behavior has not been extensively examined. Particularly, there are two types of bias when the maximal reliability statistic is calculated from sample data: (a) the sample maximal reliability estimator is a positively biased estimator of population maximal reliability, and (b) the true reliability of composites formed with maximal reliability weights calculated from sample data is on average less than the population reliability. Both effects are more pronounced in small-sample scenarios (e.g., <100). We also demonstrate that the composite reliability estimator for equally weighted composite exhibits substantially less bias, which makes it a more appropriate choice for the small-sample case.

Keywords: statistical methods; reliability (science)

Free keywords: reliability estimators; maximal reliability; composite reliability; sample size

Contributing organizations

Related projects

Ministry reporting: Yes

Reporting Year: 2019

JUFO rating: 2