Use cases

Digital Humanities

human

Use Case Description

In the Humanities there is a persistent need for data digitization and the subsequent information extraction, synthesis and valorization. With the Digital Humanities use case we aim to address exactly these issues. To do so, we focus on three key areas of investigation, each targeting specific AI-related challenges and outcomes: Information Extraction, Information Representation, and Exhibition.

In "Information Extraction", we create state-of-the-art techniques for image and audio data. We develop AI methods ranging from Computer Vision to Natural Language Processing (NLP) over automated speech recognition. Example applications include, but are not limited to: photometric image capturing hardware, the automated read-out of music folia containing Flemish polyphony, the automated parsing of the newspaper page or its spoken counterpart on the TV news. Hence, we facilitate and upscale the data digitization task. 

One of the main obstacles the Humanities are facing in uncovering new knowledge is data fragmentation. In "Information Representation", we devise techniques focusing on polyvalent data representation and the subsequent linking, i.e. defragmentation of data. Deliverables include low-rank tensor decomposition techniques for representing audio data and knowledge graphs for historic data. 

In "Valorization and Exhibition", we develop the means to interact with the data gathered and produced in the other areas of investigation. We centralize our efforts in an AI-supported data-visualization and data-auralization web portal greatly facilitating access to sources and research in the Humanities.  With the Digital Humanities use case we intend to support research across the different branches of the Humanities: Artificial Intelligence aids in collecting data, putting the pieces of the puzzle together and exhibiting our Flemish cultural heritage. 

Objective

The objective of this use case is to develop novel AI-based solutions to facilitate and improve the digitization, subsequent information extraction, synthesis and valorization of our Flemish cultural heritage. 

Research Tracks

This use case consist of five research tracks:

  • RT1 - Unlocking Polyphony (led by Prof. Bart De Moor): Develop AI-based pipelines for transcribing music in mensural notation

  • RT2 - Acoustics as intangible cultural heritage (led by Prof.  Toon van Waterschoot ): Spatial acquisition, digital archiving, and interactive auralization of historically valuable acoustics

  • RT3 - AI-driven structure analysis for music knowledge representation (led by Geraint Wiggins): Create federated repositories that integrate diverse music corpora and musicological knowledge, supported by semi-automatic annotation of hierarchical music structures, in order to enable advanced querying and analysis by digital musicologists and scholars.

  • RT4 - 2D and 3D Digital heritage (led by Prof. Tinne Tuytelaars & Prof. Marc Proesmans): Detailed 2D and 3D digitalization of cultural heritage artefacts using advanced photometric stereo techniques (minidome and/or neural rendering) as well as representation learning for image data.

  • RT5 - Curation of spoken documents​ (Led by Prof. Hugo Van hamme):  Develop and implement AI-driven transcription methods for spoken content in audio and audiovisual documents dated from 1950 onward, enabling their integration into searchable digital archives to support Humanities research and improve accessibility.

Results, Highlights & Research Outputs

RT1: Unlocking Polyphony

AI pipeline for transcribing music in mensural notation (Yoeri Uyttendaele, Prof. Bart De Moor, KULeuven-STADIUS)

We have developed a complete Agnostic Optical Music Recognition (OMR) pipeline for transcribing music in mensural notation: starting from a scanned handwritten or printed manuscript, our model produces a MusicXML or MEI file, from which the user can preserve the mensural notation or transcribe to Common Music Notation (CMN).  Furthermore, we have generated the "Alamire dataset"  written in mensural notation comprising annotated folia from illuminated choir books, printed works and chansonniers. This dataset aims to be representative for polyphonic renaissance music written in monophonic notation: folia span the whole of the renaissance period, contain several hand writing styles and show various degrees of image degradation. On this dataset and the SEILS dataset, our ALOMURE (ALamire Optical MUsic REcognition) pipeline achieves state-of-the art music symbol recall. Our pipeline builds on an ensemble of YOLOv7 object detection models with which  we mitigate class imbalance and the problem of small symbol detection.  Moreover, we have developed a novel staff line parsing algorithm that  detects all lines on the constituting staff.   Our staff line detection algorithm is capable of consistently detecting even severely degraded staff lines, and is equally applicable to the domain of music written in polyphonic notation. With our OMR pipeline, we facilitate access to music written in mensural notation for early music researchers, musicians and their audiences. With our dataset we  aim to fill the gap of a versatile benchmark dataset for mensural notation. In doing so, we provide a means to make validation of OMR research on mensural notation even more robust.

yoeri

Restoring Early Music Manuscripts with Diffusion Models

Flanders has a unique heritage of polyphonic music preserved in highly ornate, handwritten choir books. However, some of these works have not withstood the test of time: poor preservation conditions have led to missing notes and text, ink bleed‑through, and faded staff lines. We introduce a retrieval‑augmented diffusion model, trained iteratively alongside an object detection model, that can faithfully restore these images with contextual awareness. Our results show that this restoration approach facilitates the Optical Music Recognition task. Tested on both synthetic and real‑world data, it delivers significant improvements in perceptual quality, achieving over 28% better visual fidelity than traditional baselines.

RT2: Acoustics as intangible cultural heritage

Low-rank multi-channel fast convolution for real-time auralization of church acoustics (Hannes Rosseel, Prof. Toon van Waterschoot, KULeuven-STADIUS)

Interactive auralization of room acoustics is a key research tool for investigating the impact of performance space acoustics on music practice. In the context of early music, particularly plainchant and polyphony, the historically relevant performance spaces are usually churches. The interactive auralization of church acoustics is highly challenging due to the long reverberation times involved, which conflict with the real-time, low-latency requirements of interactive systems. We propose the use of low-rank compression with partially invariant transformation to achieve low-latency convolution in real-time multi-channel auralization systems. A case study of the interactive auralization of the 16th-century Nassau Chapel in Brussels assessed the auralization quality at different compression rates by analyzing objective acoustic parameters and by calculating the normalized mean squared error between auralized signals for different frequency bands. The findings showed a significant reduction in computational complexity of up to 63.62%, while maintaining a just noticeable difference of less than one for key acoustic parameters.

Publication
toon

Acoustic preservation of historical worship spaces through auralization (Hannes Rosseel, Prof.  Toon van Waterschoot, KULeuven-STADIUS):

Historical Worship Spaces are significant architectural landmarks which hold both cultural and spiritual value. The acoustic properties of these spaces play a crucial role in historical and contemporary religious liturgies, rituals, and ceremonies, as well as in the performance of sacred music. However, the original acoustic characteristics of these spaces are often at risk due to repurposing, renovations, natural disasters, or deterioration over time. We have presented a comprehensive review of the current state of research on the acquisition, analysis, and synthesis of acoustics, with a focus on Historical Worship Spaces. An example case study of the Nassau chapel in Brussels, Belgium, is presented to demonstrate the application of these techniques for the preservation and auralization of historical worship space acoustics. The paper concludes with a discussion of the challenges and opportunities in the field, and outlines future research directions.

Publication
picture-rt2-2nd-paper

RT3: AI-driven structure analysis for music knowledge representation

Knowledge representation and data federation for early music corpora and musicological scholarship (Dr. Nick Harley, Prof. Geraint Wiggins, VUB-AI Lab)

Early music manuscripts are an important part of Flanders' cultural heritage as well as a valuable research resource for early music scholars and computational musicologists. Building on existing digitisation efforts, including encoding formats for 16th century lute tablature, we apply knowledge representation technologies developed in the VUB's Computational Creativity (CC) Lab to build rich, distributed knowledge bases of capturing both the graphical content of manuscripts as well symbolic musicological interpretations of their meaning. The representation framework also affords the opportunity to link symbolic representations with high resolution spectral representations (also work from the CC Lab) of music performances. Work is proceeding to formalise ontologies for these representations and operationalise them via a uniform interface pattern (CHAKRA) which enable the use of tools for search and manipulation of sequential, hierarchical music knowledge bases.

Publications

  • Harley, N., Wiggins, G. A., Forth, J., Lewis, D., & Crawford, T. (2024) A Common Hierarchical Abstract Representation for Music: applications in historical musicology. Second international Conference on Computational and Cognitive Musicology, Utrecht, 17-18 October 2024.

RT4: 2D and 3D Digital heritage

Larger scale digitization using the Minidome (Prof. Marc Proesmans, Prof. Tinne Tuytelaars, KULeuven-PSI)

KU Leuven‑PSI has an extensive track record in designing PS‑Portable Light Dome systems based on photometric stereo (PS) technology, enabling the recovery of high‑frequency surface details as well as color and material reflectance properties with photographic quality. These systems have been regularly deployed for the detailed visualization and documentation of objects in cultural heritage collections.

In our research we extend the use of the dome in two main aspects:

  • On the one hand, support for larger scale objects (larger than the domes) was missing. We developed a procedure to scan larger surfaces in a grid like fashion and automatically align and merge them into a larger surface representation purely based on the PS surface features that are extracted from the scans. This research continues to integrate actual 3D priors from multi-view insights and create full 3D models.  In the case study of the Saint Gall Map (the only surviving major architectural drawing from the roughly 700-year period between the fall of the Western Roman Empire and the 13th century), we digitized the map at location in St Gallen. While hovering the dome over the map, around 60 “scans” were created. The full processing pipeline shows how to go from (a) imagery to (b) integrated normal mapping and to (c) a full 3D representation. 

digi1

  • On the other hand, PS traditionally imposes assumptions on the material characteristics to make the mathematics tractable, and avoid direct analysis of complex material characteristics. Several approaches have been proposed to overcome classical assumptions. We tackle the problem by adopting a data-driven approach that jointly learns surface normals and reflectance characteristics. It is intended to not only recover shape, but also to support material classification and photorealistic visualization and relighting. Because of lack of actual ground truth, we currently focus on the typical material characteristics in the cultural heritage domain and use physics based rendering to create photorealistic virtual samples. The picture below shows the simulation of a virtual dome.

picture-simulation-virtual-dome

RT5: Curation of spoken documents

Spoken language is a part of our culture. However, spoken and audiovisual documents are far less accessible than written documents, for which we have search engines. Automatic transcription of spoken documents enables humanities research in performing arts, longitudinal linguistic studies on spoken language, slang, political debates, inclusive usage of language in the media.

We focus on transcription of audio(-visual) archives spoken in Belgian Dutch (Flemish) that are up to 70 years old. The challenge is the huge variation in speech and acoustic conditions that are found in this type of data. While current technology works well for contemporary recordings, it fails here: strong dialect, formal speaking style and lexical preference, different topics and issues of audio quality, to name a few.

Our data set is very challenging with recorded intervies of over 300 "steady" persons made in the 1960's and 1970's, obtained from www.dialectloket.be. Apart from the strong dialects, the data distribution is affected by the recording equipment, storage media (tapes), far-talk speech, elderly speech and background talkers. 

Enhancing Dialect-Aware Speech Recognition with Geographic Metadata (Pouya Mehralian, Prof. Hugo Van hamme, KULeuven-PSI):

Automatic Speech Recognition (ASR) systems struggle with dialectal variations due to significant phonetic, morphosyntactic, and lexical differences across regions. Traditional approaches rely on dialect-specific models, per-dialect fine-tuning, or unified models that struggle to generalize across diverse dialects. Using the GCND dataset of dialectal speech across Flanders, we show that conditioning models on a continuous dialect representation via geopositional embeddings significantly improves word recognition accuracy compared to dialect‑independent or dialect‑specific approaches. This approach also boosts extrapolation accuracy for dialects unseen in training but geographically close. Compared to the state‑of‑the‑art OpenAI Whisper model (word error rate: 60–75%), our model achieves error rates as low as 25%, approaching the limits of annotator agreement on this challenging dataset.

Automatically generated transcript of a recording made in the 1960s (The testimony starts with "I was born in 89". He means 1889 ... )

Publication

Outputs & Resources

Publications

  • Hannes Rosseel, Martin Jälmby, and Toon van Waterschoot, “Low-rank multi-channel fast convolution for real-time auralization of church acoustics,” in Proc. 32nd European Signal Process. Conf. (EUSIPCO ’24), Lyon, France, pp. 1017-1021, Aug. 2024.

  • Hannes Rosseel and Toon van Waterschoot, “A state-of-the-art review on acoustic preservation of historical worship spaces through auralization,” Signal Processing, vol. 234, 2025.

  • Harley, N., Wiggins, G. A., Forth, J., Lewis, D., & Crawford, T. (2024). A Common Hierarchical Abstract Representation for Music: applications in historical musicology. Second International Conference on Computational and Cognitive Musicology, Utrecht, 17-18 October 2024.

  • Mehralian, P., Van hamme, H. (2025). Leveraging Geographic Metadata for Dialect-Aware Speech Recognition. Proc. Interspeech 2025, 1153-1157, doi: 10.21437/Interspeech.2025-1839.

Datasets

  • Alamire dataset: annotated folia from illuminated choir books, printed works and chansonniers, written in mensural notation and representative for polyphonic renaissance music written in monophonic notation.

  • GCND dataset: dialectal speech recordings across Flanders, used for dialect-aware automatic speech recognition with geographic metadata.

  • Dialectloket recordings: recorded interviews of over 300 “steady” persons made in the 1960s and 1970s, used as challenging Belgian Dutch/Flemish speech archive material.

Software & Tools

  • ALOMURE: an Agnostic Optical Music Recognition pipeline for music in mensural notation, producing MusicXML or MEI files from scanned handwritten or printed manuscripts.

  • Low-rank multi-channel fast convolution for real-time auralization of church acoustics.

  • Common Hierarchical Abstract Representation for Music / CHAKRA: a representation and interface pattern for sequential, hierarchical music knowledge bases.

  • AI-supported data-visualization and data-auralization web portal for humanities sources and research.

  • Dialect-aware automatic speech recognition using geographic metadata.

Demonstrators

  • Interactive auralization of the 16th-century Nassau Chapel in Brussels.

  • Minidome / photometric stereo workflow for large-scale cultural heritage digitization, including the Saint Gall Map case study.

  • Virtual dome simulation for photorealistic cultural heritage surface and reflectance modelling.

  • Automatically generated transcript demo for a 1960s dialect recording.

Videos & Media

  • https://youtu.be/z9-m4CoqqnU

Team & Contact

Contact

Bart De Moor, KU Leuven - STADIUS: bart.demoor@esat.kuleuven.be

Use Case Lead(s)

Bart De Moor, KU Leuven - STADIUS

Principal Investigators / AI Expert Leads

Bart De Moor, KU Leuven - STADIUS; Toon van Waterschoot, KU Leuven - STADIUS; Geraint Wiggins, VUB - AI Lab; Tinne Tuytelaars, KU Leuven - PSI; Marc Proesmans, KU Leuven - PSI; Hugo Van hamme, KU Leuven - PSI

Postdoctoral Researchers

Valerio Lorenzoni, KU Leuven - STADIUS; Nick Harley, VUB - AI Lab

PhD Researchers

Yoeri Uyttendaele, KU Leuven - STADIUS; Hannes Rosseel, KU Leuven - STADIUS; Pouya Mehralian, KU Leuven - PSI; Simen Cassiman, KU Leuven - PSI; Georgios Kouros, KU Leuven - PSI

Research Groups

KU Leuven - STADIUS; KU Leuven - PSI; VUB - AI Lab