A1 Alkuperäisartikkeli tieteellisessä aikakauslehdessä
All types of experience are equal, but some are more equal : The effect of different types of experience on rater severity and rater consistency (2024)

Neittaanmäki, R., & Lamprianou, I. (2024). All types of experience are equal, but some are more equal : The effect of different types of experience on rater severity and rater consistency. Language Testing, OnlineFirst. https://doi.org/10.1177/02655322241239362

JYU-tekijät tai -toimittajat

Neittaanmäki, Reeta

Julkaisun tiedot

Julkaisun kaikki tekijät tai toimittajat: Neittaanmäki, Reeta; Lamprianou, Iasonas

Lehti tai sarja: Language Testing

ISSN: 0265-5322

eISSN: 1477-0946

Julkaisuvuosi: 2024

Ilmestymispäivä: 09.04.2024

Volyymi: OnlineFirst

Kustantaja: SAGE Publications

Julkaisumaa: Britannia

Julkaisun kieli: englanti

DOI: https://doi.org/10.1177/02655322241239362

Julkaisun avoin saatavuus: Avoimesti saatavilla

Julkaisukanavan avoin saatavuus: Osittain avoin julkaisukanava

Julkaisu on rinnakkaistallennettu (JYX): https://jyx.jyu.fi/handle/123456789/94521

Tiivistelmä

This article focuses on rater severity and consistency and their relation to different types of rater experience over a long period of time. The article is based on longitudinal data collected from 2009 to 2019 from the second language Finnish speaking subtest in the National Certificates of Language Proficiency in Finland. The study investigated whether rater severity and consistency are affected differently by different types of rater experience and by skipping rating sessions. The data consisted of 45 rating sessions with 104 raters and 59,899 examinees and were analyzed using the Many-Facets Rasch model and generalized linear mixed models. The results showed that when the raters gained more rating experience, they became slightly more lenient, but different types of experience had quantitatively different magnitudes of impact. In addition, skipping rating sessions, and in that way disconnecting from the rater community, increased the likelihood of a rater to be inconsistent. Finally, we provide methodological recommendations for future research and consider implications for practice.

YSO-asiasanat: arviointi; kieli ja kielet; kielitaito; suomen kieli

Vapaat asiasanat: generalized linear mixed models; Many-Facets Rasch model; rater consistency; rater experience; rater severity; speaking assessment

Liittyvät organisaatiot

Hankkeet, joissa julkaisu on tehty

OKM-raportointi: Kyllä

Alustava JUFO-taso: 2