A4 Article in conference proceedings
Use of Sign Language Videos in EEG and MEG Studies : Experiences from a Multidisciplinary Project Combining Linguistics and Cognitive Neuroscience (2022)

Hernández, D., Puupponen, A., Keränen, J., Wainio, T., Pippuri, O., Ortega, G., & Jantunen, T. (2022). Use of Sign Language Videos in EEG and MEG Studies : Experiences from a Multidisciplinary Project Combining Linguistics and Cognitive Neuroscience. In J. H. Jantunen, J. Kalja-Voima, M. Laukkarinen, A. Puupponen, M. Salonen, T. Saresma, J. Tarvainen, & S. Ylönen (Eds.), Diversity of Methods and Materials in Digital Human Sciences : Proceedings of the Digital Research Data and Human Sciences DRDHum Conference 2022, December 1-3, Jyväskylä, Finland (pp. 148-155). Jyväskylän yliopisto. http://urn.fi/URN:ISBN:978-951-39-9450-1

JYU authors or editors

Hernández Barros, Doris
Puupponen, Anna
Keränen, Jarkko
Wainio, Tuija
Pippuri, Outi
Jantunen, Tommi

Publication details

All authors or editors: Hernández, Doris; Puupponen, Anna; Keränen, Jarkko; Wainio, Tuija; Pippuri, Outi; Ortega, Gerardo; Jantunen, Tommi

Parent publication: Diversity of Methods and Materials in Digital Human Sciences : Proceedings of the Digital Research Data and Human Sciences DRDHum Conference 2022, December 1-3, Jyväskylä, Finland

Parent publication editors: Jantunen, Jarmo Harri; Kalja-Voima, Johanna; Laukkarinen, Matti; Puupponen, Anna; Salonen, Margareta; Saresma, Tuija; Tarvainen, Jenny; Ylönen, Sabine

Conference:

Digital Research Data and Human Sciences

Place and date of conference: Jyväskylä, Suomi, 1.-3.12.2022

eISBN: 978-951-39-9450-1

Publication year: 2022

Pages range: 148-155

Number of pages in the book: 243

Publisher: Jyväskylän yliopisto

Publication country: Finland

Publication language: English

Persistent website address: http://urn.fi/URN:ISBN:978-951-39-9450-1

Publication open access: Openly available

Publication channel open access: Open Access channel

Publication is parallel published (JYX): https://jyx.jyu.fi/handle/123456789/84357

Abstract

In this paper, we describe our experiences of bringing together methodologies of two disciplines – sign language (SL) linguistics and cognitive neuroscience – in the multidisciplinary ShowTell research project (Academy of Finland 2021–2025). More specifically, we discuss the challenges we encountered when creating and using video materials for the study of SL processing in the brain. Rather than using still images, the study of SL comprehension is better performed by using videos, thus providing more naturalistic stimuli as observed in face-to-face interaction. On the other hand, in neuroimaging (electroencephalography [EEG]/magnetoencephalography [MEG]), it is vital to track the timing of the stimulation exactly and to minimize the noise that could arise from inside and outside the brain. Any brain activity not related to the specific aspect being studied could create artifacts that diminish the signal-to-noise ratio of the measurements, thus compromising the quality of the data. This creates significant challenges when integrating both disciplines into the same study. In the paper, we (i) describe the process of, and requirements for, creating signed video materials that try to mirror naturalistic signing; (ii) discuss the problems in the synchronization of the video stimuli with the brain imaging data; and (iii) introduce the steps we have taken to minimize these challenges in different phases of the process, such as the design, recording, and processing of the video stimuli. Finally, we discuss how, with the use of these steps, we have been able to deal successfully with the resulting data and creating materials that integrate the naturalistic nature of human communication.

Keywords: sign language; linguistic interaction; cognitive neuroscience; neurolinguistics; multidisciplinary research; video; EEG; MEG