Human-Machine Flexible Production
Use Case Description:
In today's rapidly evolving manufacturing landscape with an increasing level of automation, human operators continue to play a crucial role to ensure flexibility. As tasks become increasingly complex and product variants more diverse, the need for intuitive operator support with AI rises. Operators nowadays need to manage a large variety of skills, beyond the physical work (incl. problem solving, machine tending, robot interaction, etc.). Together with the societal challenges on aging workforce, this increased workload raised to importance of worker wellbeing, which is confirmed by the concept of 'Production Employee 5.0'.
The use case Human-Machine Flexible Production, targets innovative technologies that can be used to assist operators in their daily tasks in order to upskill them and improve their wellbeing. This is reflected in the 3 POCs being tackled within the use case:
Assisted Operator: Providing an intuitive Human-Machine Interface (HMI) in the shape of a Digital Colleague, that is able to provide cognitive and physical support on demand through advanced reasoning capabilities & intelligent robotics.
Healthy Operator: Addressing the physical & congitive wellbeing of operators though objective assessment using multiple wearable or non-intrusive sensor systems, with specific attention to preserving the privacy of the operator.
Augmented Operator: To overcome the human limits on the cognitive level, we want to provide the user with advanced reasoning capabilities based on intelligent perception of the environment and connected sensor systems. Exoskeletons and intuitive robotics will be used to extend also the physical capabilities of assembly workers.
Different Flanders Make set-ups will be used to generate hybrid datasets in order to develop & validate AI techniques to provide relevant information to operators in their tasks, e.g., step-by-step assembling complex parts, provide physical assistance to intelligently handle a part in a cooperation with the human, provide information to operators about their ergonomics, etc.
Results, Highlights & Research Outputs
The digital colleague
POC1 & POC3
The Digital Colleague is an AI-powered assistant designed to support operators in manufacturing environments. It acts as a smart, on-demand partner that you can talk to. Operators can use their voice to ask for instructions, which are then displayed by a smart projector, or get help from a collaborative robot for certain assembly steps. The system aims to make complex and varied tasks simpler and more efficient for the operator.
Photo / Video
Tangible results
We conducted a preliminary user study to see how people would interact with the Digital Colleague. In the study, 21 participants were asked to assemble a light switch using the system for about 5 to 10 minutes each. The feedback was generally positive. Users found the system to be engaging, appealing, and novel. However, the study also highlighted areas for improvement. The main issue was the system's efficiency, with users noting that slow response times made tasks take longer. Some users also reported that the Digital Colleague occasionally provided incorrect instructions, a phenomenon known as "hallucination".
Publications
E. Verhelst, N. Harley, B. Van Doninck, and A. Bey-Temsamani, "A Digital Colleague as intuitive operator support system in a HMLV Production Environment," Procedia CIRP, 2025.
Privacy aware ergonomics
POC2
This demonstrator tackles workplace ergonomics without sacrificing privacy. Cameras in the factory track how people move and work, but before anything leaves the camera, smart software obfuscates the footage. All the system keeps is what’s needed to estimate body pose. This means no faces, skin tone, or identifying features. The result lets computers check for risky postures and calculate ergonomic scores, but keeps personal details out of the picture.
How it works:
Cameras record people at work from different angles.
Footage runs through a privacy filter, scrambling personal info but keeping key points for posture analysis.
The system figures out the worker’s position and calculates REBA scores, so you can spot and fix ergonomic problems fast.
Traditional methods like blur and pixelation cut privacy risks but ruin pose accuracy. This new approach strikes a balance: lots of privacy, good enough accuracy for practical safety checks.
Tangible results
The system was tested on 20 people doing routine assembly tasks.
It reliably flagged low, medium, and high ergonomic risks.
The accuracy for posture scoring stayed strong, especially in more common, lower-risk poses.
Faces and personal traits were successfully hidden, meeting modern privacy expectations.
Publication
Publications
S. De Coninck, E. Gamba, B. Van Doninck, A. Bey-Temsamani, S. Leroux, P. Simoens, "Enabling Privacy-Aware AI-Based Ergonomic Analysis," Procedia CIRP, 2025
Operator Pose Dataset
We made a small dataset that other teams can use if they want to test out ideas about tracking how people work. 6 people did a straightforward assembly job in a space about 4 by 5 meters. Four cameras watched them from different angles. Each camera used software to pick out where their arms and legs were. All these videos got put together so we could see how each person’s body moved in 3D. While the recording happened, we also had a RADAR sensor turned on to track movement in a different way.
Here’s how it works:
Four cameras record people doing their tasks from multiple sides.
Software reads the camera feeds and figures out each person’s body pose in each view.
The different views get combined to make a 3D skeleton of how the person is moving.
At the same time, RADAR tracks everyone’s movement, providing a different kind of data.
The point is to help others check if activity tracking, posture analysis, or RADAR-based detection holds up in real shop-floor settings.
Tangible Results
The dataset has both video (with 3D pose estimates) and RADAR data, all from the exact same assembly task.
It’s set up for anyone needing data to test or compare new tracking methods, ergonomic risk spotting, or activity detection using vision or RADAR.
Outputs & Resources
Publications
E. Verhelst, N. Harley, B. Van Doninck, and A. Bey-Temsamani, "A Digital Colleague as intuitive operator support system in a HMLV Production Environment," Procedia CIRP, 2025.
S. De Coninck, E. Gamba, B. Van Doninck, A. Bey-Temsamani, S. Leroux, P. Simoens, "Enabling Privacy-Aware AI-Based Ergonomic Analysis," Procedia CIRP, 2025.
Datasets
Operator Pose Dataset.
Demonstrators / Applications
Digital Colleague for operator support in high-mix, low-volume production environments.
Privacy-aware ergonomics demonstrator for pose-based ergonomic risk assessment.
Operator Pose Dataset for testing activity tracking, posture analysis and RADAR-based detection.
Events & Demonstrations
2024 - Flanders AI Forum (11/6/2024): Demo & Deep-dive
2024 - GenAI workshop by AIM NET (10/7/2024): Presentation
2024 - Flanders AI Research Day (14/10/2024): Demo & Deep-dive
2024 - On-site demonstrations @FM Leuven: FTI Leuven, Augmented Human, etc.
2025 - RebootSkills (06/03/2025) - Demo of AI digital colleague + hands-on session about skiros and LLM interaction
2025 - Advanced Engineering FAIR (21/03/2025) - FAIR demonstrating multiple demo's coming from FAIR (AGV, obfuscated ergonomics, ISaac sim VR)
2025 - Flanders AI Forum (17/06/2025) - Showcasing the assembly tracking for the flashlight assembly
2025 - Several on-site company visits / demonstrations of the AI digital colleague (DANA, Ansomat, VOKA)
Uptake
PRIVAI (enrichment project): Privacy-aware Vision-based AI for Industry
CRISP (enrichment project : Enhancing Real-Time ASR for Flemish Speech
Team & Contact
Contact
Ewoud Verhelst (Use case lead) - ewoud.verhelst@flandersmake.be - Flanders Make IM
Industry Use Cases follow up & Valorization: Wouter De Nijs / Abdel Bey - Flanders Make IM
Use Case Lead(s)
Ewoud Verhelst, Flanders Make IM
Principal Investigators / AI Expert Leads
Geraint Wiggins, VUB AI Lab; Pieter Simoens, UGent IDLab; Hugo Van Hamme, KU Leuven PSI; Nikolaos Deligiannis, VUB ETRO; Bart Dhoedt, UGent IDLab; Wilfried Philips, UGent IDLab; Matthias Hartmann, imec ARF; Kris Luyten, UHasselt EDM
PhD Researchers
Nick Harley, VUB AI Lab; Sander De Coninck, UGent IDLab; Jacob Poncelet, KU Leuven PSI; Stefano Ferraro, UGent IDLab
Research groups
Knowledge representation for Interactive & Cognitive Systems (VUB AI Lab - PI: Geraint Wiggins - PhD: Nick Harley)
Privacy-preservation of Operators (Ugent IDLAB - PI: Pieter Simoens - PhD: Sander De Coninck)
Non-native Dutch for Operator Interaction with Speech (KULeuven PSI - PI: Hugo Van Hamme - PhD: Jacob Poncelet)
Explainable AI for Multimodal Data (VUB ETRO - PI: Nikolaos Deligiannis)
Online Representation Learning (UGENT IDLAB - PI: Bart Dhoedt - PhD: Stefano Ferraro)
Multi-context sensing pipelines (UGENT IDLAB - PI: Wilfried Philips) - Being refined
SNN-based HW for event based sensing (IMEC ARF - PI: Matthias Hartmann) - Being refined
Human-AI machine interface in the context of Human-machine production (UHasselt EDM - PI: Kris Luyten) - Being refined
