A4 Article in conference proceedings
CRTypist : Simulating Touchscreen Typing Behavior via Computational Rationality (2024)
Shi Danqing, Zhu Yujun, Jokinen Jussi P., P., Acharya Aditya, Putkonen Aini, Zhai Shumin, Oulasvirta Antti. (2024). CRTypist : Simulating Touchscreen Typing Behavior via Computational Rationality. In F. F. Mueller, P. Kyburz, J. R. Williamson, C. Sas, M. L. Wilson, P. T. Dugas, & I. Shklovski (Eds.), CHI '24 : Proceedings of the CHI Conference on Human Factors in Computing Systems (Article 942). ACM. https://doi.org/10.1145/3613904.3642918
JYU authors or editors
Publication details
All authors or editors: Shi Danqing; Zhu Yujun; Jokinen Jussi P., P.; Acharya Aditya; Putkonen Aini; Zhai Shumin; Oulasvirta Antti
Parent publication: CHI '24 : Proceedings of the CHI Conference on Human Factors in Computing Systems
Parent publication editors: Mueller, Florian Floyd; Kyburz, Penny; Williamson, Julie R.; Sas, Corina; Wilson, Max L.; Dugas, Phoebe Toups; Shklovski, Irina
Conference:
- ACM SIGCHI annual conference on human factors in computing systems
Place and date of conference: Honolulu, USA, 11.-16.5.2024
eISBN: 979-8-4007-0330-0
Publication year: 2024
Publication date: 11/05/2024
Article number: 942
Publisher: ACM
Publication country: United States
Publication language: English
DOI: https://doi.org/10.1145/3613904.3642918
Research data link: https://crtypist.github.io/
Publication open access: Openly available
Publication channel open access: Partially open access channel
Abstract
Touchscreen typing requires coordinating the fingers and visual attention for button-pressing, proofreading, and error correction. Computational models need to account for the associated fast pace, coordination issues, and closed-loop nature of this control problem, which is further complicated by the immense variety of keyboards and users. The paper introduces CRTypist, which generates human-like typing behavior. Its key feature is a reformulation of the supervisory control problem, with the visual attention and motor system being controlled with reference to a working memory representation tracking the text typed thus far. Movement policy is assumed to asymptotically approach optimal performance in line with cognitive and design-related bounds. This flexible model works directly from pixels, without requiring hand-crafted feature engineering for keyboards. It aligns with human data in terms of movements and performance, covers individual differences, and can generalize to diverse keyboard designs. Though limited to skilled typists, the model generates useful estimates of the typing performance achievable under various conditions.
Keywords: machine learning; reinforcement learning; human-computer interaction; modelling (representation); simulation; writing; touch screen
Free keywords: simulation models; reinforcement learning; touchscreen typing; computational modeling
Contributing organizations
Ministry reporting: Yes
Preliminary JUFO rating: 3
