Collaborative Learning
Use Case Description
This use case on collaborative learning meets the increasing need for training and education in society, while at the same time trying to foster learner engagement through multimodal interaction with an embodied robot or a VR/AR environment. The use case focuses on 3 major challenges, i.e., knowledge grounding, multimodal interaction and situational awareness, all demonstrated in a Proof of Concept (POC). It is supported by a large and diverse team of PIs and will showcase the potential of AI in education.
Results, Highlights & Research Outputs
Anthropomorphic user interfaces
An online database categorising research on anthropomorphic user interfaces, offering a taxonomy and web tool that help designers analyse existing systems and guide the development of more human-like, intuitive, and sustainable interfaces.
UniC
UniC is the first multimodal natural emotion dataset (in English) with independent annotations at both unimodal and multimodal levels. UniC enable to investigate multimodal emotion detection (and the impact of each of the underlying modalities) in naturalistic settings.
Publication
Du, Q., Labat, S., Demeester, T., & Hoste, V. (2025). UniC : a dataset for emotion analysis of videos with multimodal and unimodal labels. Language Resources and Evaluation, 59(3), 2857–2892.
IDAS: Intent Discovery with Abstractive Summarization
A repository containing the datasets and implementation of the NLP4ConvAI@ACL 2023 paper "IDAS: Intent Discovery with Abstractive Summarization" by Maarten De Raedt, Fréderic Godin, Thomas Demeester, and Chris Develder. https://github.com/maarten-deraedt/IDAS-intent-discovery-with-abstract-summarization
EduQG
A high-quality dataset containing 3,397 samples comprising (i) multiple choice questions, (ii) answers (including distractors), and (iii) their source documents, from the educational domain.
CookDial
The first Cooking Dialog dataset (CookDial), a task-oriented dialog dataset in which all the dialogs are grounded in procedural documents (i.e. cooking recipes in our research). It contains 260 dialogs and 260 corresponding recipes. All the dialogs are collected in a human-to-human WoZ setting, in which an agent tries to instruct a user to accomplish a cooking process based on a recipe text.
Publication
Jiang, Y., Zaporojets, K., Deleu, J., Demeester, T., & Develder, C. (2023). CookDial: a dataset for task-oriented dialogs grounded in procedural documents. Applied Intelligence, 53(4), 4748-4766.
DEMONSTRATOR
Second Language Learning (UGent)
Practicing a second language through conversation with a social robot. This method steps away from the traditional formal method of language education, and instead relies on conversation to practice and gain confidence in a target language. The demonstrator relies on Large Language Models, Diffusion Models, Transformer-based speech recognition, and fine-tuning to steer the conversation within the Zone Proximal Development of the learner.
Tangible/measurable results
The use case has led to a number of results, highlights include:
Speech recognition is now reliable enough to be used for atypical speakers, such as young children or learners who are not competent yet in a language, allowing for new applications in language learning with AI (Janssens, Verhelst, et al., 2024).
Large Language Models are powerful tools to automate the creation of educational content, such as distractors for multiple choice questions (Bitew et al., 2023; Hadifar et al., 2024).
We expect embodied and situated AI to be integral to collaborative learning. These rely on the integration of AI systems, for example, enabling robots to engage in a conversation about the visual context in which the robot and the user find themselves (Janssens et al., 2024).
Outputs & Resources
Publications
Amioka, S., Janssens, R., Wolfert, P., Ren, Q., Pinto Bernal, M. J., & Belpaeme, T. (2023, March). Limitations of Audiovisual Speech on Robots for Second Language Pronunciation Learning. In Proceedings of the 2023 ACM/IEEE International Conference on Human-Robot Interaction (pp. 359-367).
Bitew, S. K., Deleu, J., Develder, C., & Demeester, T. (2023). Distractor generation for multiple-choice questions with predictive prompting and large language models. In Proc. 1st Int. Tut. Workshop on Responsible Knowledge Discovery in Education (RKDE 2023) at ECML-PKDD 2023, Turin, Italy, 18 Sep. 2023, pp. 1-16.
Bitew, S. K., Deleu, J., Doğruöz, A. S., Develder, C., & Demeester, T. (2023). Learning from partially annotated data: Example-aware creation of gap-filling exercises for language learning. In Proc. 18th Workshop Innovative Use of NLP for Building Educational Applications (BEA 2023) at ACL 2023, Toronto, Canada, 13 Jul. 2023, pp. 598-609. doi:10.18653/v1/2023.bea-1.51S.
Bitew, S. K., Hadifar, A., Sterckx, L., Deleu, J., Develder, C., & Demeester, T. (2022). Learning to Reuse Distractors to Support Multiple-Choice Question Generation in Education. IEEE Transactions on Learning Technologies, 17, 375-390.
Janssens, R., Verhelst, E., Abbo, G. A., Ren, Q., Bernal, M. J. P., & Belpaeme, T. (2024). Child Speech Recognition in Human-Robot Interaction: Problem Solved?. In Proceedings of the International Conference on Social Robotics 2024, Odense, Denmark.
Janssens, R., Wolfert, P., Demeester, T., & Belpaeme, T. (2024). Integrating Visual Context into Language Models for Situated Social Conversation Starters. IEEE Transactions on Affective Computing. 1-14. doi: 10.1109/TAFFC.2024.3428704.
Ren, Q., Hou, Y., Botteldooren, D., & Belpaeme, T. (2024). No More Mumbles: Enhancing Robot Intelligibility Through Speech Adaptation. IEEE Robotics and Automation Letters. 9(7):6162-6169.
De Baer, J., Doğruöz, A. S., Demeester, T., & Develder, C. (2025). Single-vs. Dual-Prompt Dialogue Generation with LLMs for Job Interviews in Human Resources. arXiv preprint arXiv:2502.18650.
Janssens, R., De Bock, J., Labat, S., Verhelst, E., Hoste, V., & Belpaeme, T. (2025). Why Robots Are Bad at Detecting Their Mistakes: Limitations of Miscommunication Detection in Human-Robot Dialogue. arXiv preprint arXiv:2506.20268.
Geurts, E., & Luyten, K. (2024, October). Anthropomorphic User Interfaces: Past, Present and Future of Anthropomorphic Aspects for Sustainable Digital Interface Design. In Proceedings of the European Conference on Cognitive Ergonomics 2024 (pp. 1-7).
Dhiman, H., Rovelo Ruiz, G. A., Ramakers, R., Leen, D., & Röcker, C. (2024, May). Designing instructions using self-determination theory to improve motivation and engagement for learning craft. In Proceedings of the 2024 CHI Conference on Human Factors in Computing Systems (pp. 1-16).
PhD Dissertations
Maarten De Raedt (2024, UGent)
Yiwei Jiang (2024, UGent)
Semere K. Bitew (2024, UGent)
Yuanbo Hou (2025, UGent)
Pengfei Sun (2025, Ugent)
Tomasz Grzywalski (2025, UGent)
Software & Tools
Anthropomorphic user interfaces online database / web tool.
IDAS repository.
Datasets
UniC.
EduQG.
CookDial.
Demonstrators
Second Language Learning demonstrator.
Events & Workshops
Workshops related to the Use Case:
Outreach
VOKA: Opleiding Generatieve AI
Prototypes demonstrations
Learning Bytes festival, 20/3/2024