A1 Journal article (refereed)
Audiovisual Processing of Chinese Characters Elicits Suppression and Congruency Effects in MEG (2019)

Xu, W., Kolozsvari, O., Oostenveld, R., Leppänen, P. H., & Hämäläinen, J. (2019). Audiovisual Processing of Chinese Characters Elicits Suppression and Congruency Effects in MEG. Frontiers in Human Neuroscience, 13, Article 18. https://doi.org/10.3389/fnhum.2019.00018

JYU authors or editors

Xu, Weiyong
Leppänen, Paavo
Hämäläinen, Jarmo
Kolozsvari, Orsolya

Publication details

All authors or editors: Xu, Weiyong; Kolozsvari, Orsolya; Oostenveld, Robert; Leppänen, Paavo H.T.; Hämäläinen, Jarmo

Journal or series: Frontiers in Human Neuroscience

ISSN: 1662-5161

eISSN: 1662-5161

Publication year: 2019

Volume: 13

Issue number: 0

Article number: 18

Publisher: Frontiers Research Foundation

Publication country: Switzerland

Publication language: English

DOI: https://doi.org/10.3389/fnhum.2019.00018

Publication open access: Openly available

Publication channel open access: Open Access channel

Publication is parallel published (JYX): https://jyx.jyu.fi/handle/123456789/63102

Abstract

Learning to associate written letters/characters with speech sounds is crucial for reading acquisition. Most previous studies have focused on audiovisual integration in alphabetic languages. Less is known about logographic languages such as Chinese characters, which map onto mostly syllable-based morphemes in the spoken language. Here we investigated how long-term exposure to native language affects the underlying neural mechanisms of audiovisual integration in a logographic language using magnetoencephalography (MEG). MEG sensor and source data from 12 adult native Chinese speakers and a control group of 13 adult Finnish speakers were analyzed for audiovisual suppression (bimodal responses vs. sum of unimodal responses) and congruency (bimodal incongruent responses vs. bimodal congruent responses) effects. The suppressive integration effect was found in the left angular and supramarginal gyri (205–365 ms), left inferior frontal and left temporal cortices (575–800 ms) in the Chinese group. The Finnish group showed a distinct suppression effect only in the right parietal and occipital cortices at a relatively early time window (285–460 ms). The congruency effect was only observed in the Chinese group in left inferior frontal and superior temporal cortex in a late time window (about 500–800 ms) probably related to modulatory feedback from multi-sensory regions and semantic processing. The audiovisual integration in a logographic language showed a clear resemblance to that in alphabetic languages in the left superior temporal cortex, but with activation specific to the logographic stimuli observed in the left inferior frontal cortex. The current MEG study indicated that learning of logographic languages has a large impact on the audiovisual integration of written characters with some distinct features compared to previous results on alphabetic languages.

Keywords: Chinese language; language learning; reading; sense of hearing; eyesight; graphic characters; speech sounds; correspondence (equivalence); brain research; MEG

Free keywords: audiovisual integration; magnetoencephalography; auditory cortex; Chinese characters; Multisensory Integration; Audiovisual Equipment; Cross-Modal

Fields of science: