A1 Journal article (refereed)
A checklist to assess the quality of survey studies in psychology (2020)
Protogerou, C., & Hagger, M. S. (2020). A checklist to assess the quality of survey studies in psychology. Methods in Psychology, 3, 100031. https://doi.org/10.1016/j.metip.2020.100031
JYU authors or editors
Publication details
All authors or editors: Protogerou, Cleo; Hagger, Martin S.
Journal or series: Methods in Psychology
eISSN: 2590-2601
Publication year: 2020
Volume: 3
Pages range: 100031
Publisher: Elsevier
Publication country: Netherlands
Publication language: English
DOI: https://doi.org/10.1016/j.metip.2020.100031
Publication open access: Openly available
Publication channel open access: Open Access channel
Publication is parallel published (JYX): https://jyx.jyu.fi/handle/123456789/71235
Abstract
Study quality is emerging as an essential component of evidence syntheses and allows practitioners and policymakers to make informed decisions based on the quality of the evidence reviewed. Study quality is typically assessed by checklists of pre-determined quality criteria. Few study quality checklists have been systematically evaluated, and none have been developed specifically for survey studies in psychology. The present study addresses this evidence gap by developing the quality of survey studies in psychology (Q-SSP) checklist, using an expert-consensus method. An international panel of experts in psychology research and quality assessment (N = 53) evaluated the inclusion and importance of candidate quality items and offered commentary. The resulting checklist was used to evaluate a set of survey studies and inter-rater reliability of checklist scores was computed. A preliminary test of criterion validity of checklist scores was conducted using on a sample of survey studies with ‘known differences’ in study quality verified by experts. Experts exhibited high agreement on inclusion and importance ratings of the candidate items. Minor adjustments were made to the candidate items based on experts' feedback. Inter-rater reliability of study quality scores using the checklist was high. Some evidence for criterion validity of scores using the checklist was obtained. Overall, we provide preliminary data to support the Q-SSP checklist as a potential means to evaluate the quality of survey studies in psychology. We recommend a large-scale study using the Q-SSP checklist to assess study quality in studies with known differences in quality verified by experts.
Keywords: psychology; research; research methods; questionnaire survey; survey research; quality; evaluation; quality assessment; correlation; transparency
Free keywords: study quality assessment; psychology; correlational studies; survey studies; evidence syntheses; transparency
Contributing organizations
Related projects
- Motivation: Theory- and Evidence -based Interventions to increace Physical activity
- Lintunen, Taru
- TEKES
Ministry reporting: Yes
Reporting Year: 2020
JUFO rating: 1