A4 Article in conference proceedings
What Comes First : Combining Motion Capture and Eye Tracking Data to Study the Order of Articulators in Constructed Action in Sign Language Narratives (2020)


Jantunen, Tommi; Puupponen, Anna; Burger, Birgitta (2020). What Comes First : Combining Motion Capture and Eye Tracking Data to Study the Order of Articulators in Constructed Action in Sign Language Narratives. In Calzolari, Nicoletta; Béchet, Frédéric; Blache, Philippe; Choukri, Khalid; Cieri, Christopher; Declerck, Thierry; Goggi, Sara; Isahara, Hitoshi; Maegaard, Bente; Mariani, Joseph; Mazo, Hélène et al. (Eds.) LREC 2020 : Proceedings of the 12th Conference on Language Resources and Evaluation, LREC proceedings. European Language Resources Association, 6003-6007. https://www.aclweb.org/anthology/2020.lrec-1.735.pdf


JYU authors or editors


Publication details

All authors or editors: Jantunen, Tommi; Puupponen, Anna; Burger, Birgitta

Parent publication: LREC 2020 : Proceedings of the 12th Conference on Language Resources and Evaluation

Parent publication editors: Calzolari, Nicoletta; Béchet, Frédéric; Blache, Philippe; Choukri, Khalid; Cieri, Christopher; Declerck, Thierry; Goggi, Sara; Isahara, Hitoshi; Maegaard, Bente; Mariani, Joseph; Mazo, Hélène; Moreno, Asuncion; Odijk, Jan; Piperidis, Stelios

Conference:

International Conference on Language Resources and Evaluation

Place and date of conference: Marseille, France, 11.-16.5.2020

ISBN: 979-10-95546-34-4

Journal or series: LREC proceedings

eISSN: 2522-2686

Publication year: 2020

Pages range: 6003-6007

Number of pages in the book: 7353

Publisher: European Language Resources Association

Publication country: France

Publication language: English

Persistent website address: https://www.aclweb.org/anthology/2020.lrec-1.735.pdf

Open Access: Publication published in an open access channel

Publication is parallel published (JYX): https://jyx.jyu.fi/handle/123456789/71022


Abstract

We use synchronized 120 fps motion capture and 50 fps eye tracking data from two native signers to investigate the temporal order in which the dominant hand, the head, the chest and the eyes start producing overt constructed action from regular narration in seven short Finnish Sign Language stories. From the material, we derive a sample of ten instances of regular narration to overt constructed action transfers in ELAN which we then further process and analyze in Matlab. The results indicate that the temporal order of articulators shows both contextual and individual variation but that there are also repeated patterns which are similar across all the analyzed sequences and signers. Most notably, when the discourse strategy changes from regular narration to overt constructed action, the head and the eyes tend to take the leading role, and the chest and the dominant hand tend to start acting last. Consequences of the findings are discussed.


Keywords: sign language; Finnish Sign Language; motion capture; eye tracking

Free keywords: motion capture; eye tracking, sign language; constructed action; narration


Contributing organizations


Related projects

Aspects of the Grammar and Prosody of Finnish Sign Language
Jantunen, Tommi
Academy of Finland
01/09/2013-31/08/2018
Aspects of the Grammar and Prosody of Finnish Sign Language (research costs/2)
Jantunen, Tommi
Academy of Finland
01/09/2016-31/12/2018
Dancing to the same beat - Music-induced social bonding
Burger, Birgitta
Academy of Finland
01/09/2016-30/08/2019


Ministry reporting: Yes


Last updated on 2020-02-07 at 10:35