AI-Assisted Operator
FAIR 1 Archive Notice
This page describes research activities and results from the FAIR 2019-2023 program period.
The content is retained for reference purposes and is no longer actively maintained. Information on this page, including research outputs, team members, contact details, collaborations and links, may be incomplete or outdated.
This use case has been succeeded by the Human-machine flexible production use case within the FAIR 2024-2028 program.
Use Case Description
Production employees are encountering significant challenges due to the increased complexity and workload in today’s manufacturing environments. In order to remain competitive, the industry must align with the market trends towards smaller batch sizes and more customized products, without increasing costs. Despite innovative automation solutions, humans will continue to play a crucial role in providing the required flexibility in production, but their jobs will become more demanding. Sustaining this viability on the longer term requires empowering existing employees with more intelligent support systems. Furthermore, given the scarcity of skilled operators in the job market, enhancing operator guidance through AI could mitigate the need for extensive expertise among new hires and at the same time integrating them faster into the workforce. This use case explores how AI can be leveraged to address these challenges and provide better operator guidance in the future.
Seamless interaction through a chatbot or voice assistant
A chatbot is trained upon available colleague interaction dialogues, while making abstraction from the specific assembly process that was followed. This allows to easily adjust the chatbot for different assembly processes by simply changing the knowledge source. Natural language understanding (NLU) helps to interpret the intuitive questions of the operator, and formulate the best known answer. Large Language Models (LLMs) are used as fallback scenario, in case the answer could not be derived from the provided custom knowledge base. Finally, to even further increase the ease of interaction, speech is added. While a basic speech-to-text module could be placed in front of the NLU of the chatbot, Spoken language understanding (SLU) is implemented to immediately detect the intent from speech, without loosing accuracy due to intermediate conversions.
Increased accuracy through advanced knowledge representation & context-awareness
To be able to answer a range of questions, it’s important that the digital assistant has access to various information sources (assembly knowledge, execution history, robot capabilities, etc.). To simplify this integration of scattered knowledge, a unified hierarchical knowledge representation is adopted, which allows the assistant to reason over all sources as if they would be in a single database. Additionally, it’s important that the assistant is aware of it’s environment to be able to answer urgent, context-aware questions about the quality or task at hand. Therefore, a world model of the work cell is introduced which continuously updates based on vision.
Added physical support capabilities
Finally, to give the assistant also some physical support capabilities, a skill-based robot programming framework is added which builds upon the same world model. The assistant is able to instruct the robot on performing assembly tasks by sequencing a set of parametrizable skills.
Demonstrator
Flanders Make has build an open and modular demonstrator platform for human-robot collaborative assembly, where the operator is supported in his tasks by a digital assistant. This digital assistant is able to provide on-demand support to the operator by providing information through speech, by highlighting the information on the work table or by commanding the collaborative robot to execute a specific task. To maximize the added value of this digital assistant, the interaction should be seamless and the information accurate. This is where AI starts playing a role:
Seamless interaction with a chatbot or voice assistant, through a configurable knowledge base
Written or spoken language understanding
Skill-based robot programming
Basic Use Case Info
Period
2019-2023
Contact
Flanders Make - Bart Van Doninck - bart.vandoninck@flandersmake.be
Research Groups
Flanders Make, KULeuven-PSI, VUB-AI Lab-Di-Mo-MOBI, UA CLiPS-IDLab, UGent WAVES
VUB (Geraint Wiggins), KUL (Hugo Van Hamme), UAntwerp (Jeska Buhman)