International Audio Laboratories Erlangen

Computer-Assisted Analysis of Harmonic Structures (CAS)

Logo_DFG Teaser_CAS Logo_FAU Logo_UniSaarland Logo_UniWuerzburg

In the CAS project, we explore how computer-assisted methods can be applied in musicological studies for anlayzing harmonic and other musical structures in large-scale music corpora (including the piano sonatas by Ludwig van Beethoven and the tetralogy "Der Ring des Nibelungen" by Richard Wagner). This interdisciplinary project is funded by the German Research Foundation. On this website, we summarize the project's main objectives and provide links to project-related resources (data, demonstrators, websites) and publications.

Project Description

Computer-Assisted Analysis of Harmonic Structures (CAS)

This is a follow-up project continuing the previous DFG-funded project "Computergestützte Analyse harmonischer Strukturen" [MU 2686/7-1, KL 864/4-1]. Our interdisciplinary project deals with the development of automated techniques for the analysis of harmonic structures. On a broader level, we aim at investigating to which extent musicology may benefit from using computer-based methods and, vice versa, musicological research may introduce new scientific challenges into computer science. In addition to the development of computer-based analysis techniques, our further goal is to explore novel navigation and visualization concepts that allow researchers to browse, search, and analyze large music collections with regard to harmonic structures in an intuitive and interactive way. The concepts are paradigmatically developed, verified, and discussed on the basis of concrete music corpora. In particular, in the case of the tetralogy "Der Ring des Nibelungen" by Richard Wagner, unknown structural relationships may be discovered, thus gaining new musicological insights. In this follow-up project, we significantly extend the objectives of the previous project. By considering further parameters, we aim at expanding and refining the harmonic analyses. In addition to harmonic structures, musical aspects such as motifs, instrumentation, and performance practice as well as their interrelations are subject of our computer-assisted analyses. The two main corpora prepared in the first project phase, Beethoven's piano sonatas and Wagner's "Ring" (including the symbolically encoded scores and annotated music recordings), provide an excellent basis for these subsequent studies. The continuation of the project shall deepen a spirit of openness, mutual interest, and long-term thinking, which may serve as a positive example of interdisciplinary collaboration in the field of Digital Humanities.

Projektbeschreibung

Computergestützte Analyse harmonischer Strukturen

Dieses Projekt ist eine Fortsetzung von dem Projekt zum Thema "Computergestützte Analyse harmonischer Strukturen" [MU 2686/7-1, KL 864/4-1]. In diesem interdisziplinären Projekt soll am Beispiel der automatisierten Analyse harmonischer Strukturen erprobt werden, inwieweit informatische Methoden gewinnbringend im Bereich der Musikwissenschaft eingesetzt werden können und inwieweit musikwissenschaftliche Fragestellungen zu neuen Herausforderungen in der Informatik führen. Neben computerbasierten Methoden und Werkzeugen zur Harmonieanalyse sind neuartige Visualisierungs- und Navigationskonzepte zu erforschen, die es erlauben, große Musikdatenbestände hinsichtlich harmonischer Strukturen auf interaktive Weise zu durchsuchen und zu analysieren. Die zu entwickelnden Konzepte werden paradigmatisch anhand von konkreten Werkzyklen entwickelt, verifiziert und diskutiert. Insbesondere soll im Fall der Tetralogie "Der Ring des Nibelungen" von Richard Wagner auch musikwissenschaftliches Neuland bei der Erforschung bisher verborgener struktureller Bezüge betreten werden. In der zweiten Projektphase werden die Zielstellungen des Projekts signifikant erweitert. Durch die Miteinbeziehung weiterer Parameter sollen die harmonischen Analysen ausgeweitet und verfeinert werden. Neben harmonischen Strukturen sollen weitere musikalische Aspekte wie Motivik, Instrumentierung und Aufführungspraxis sowie deren Wechselbeziehungen Gegenstand der computergestützten Analyse werden. Die in der ersten Projektphase erarbeiteten Datensammlungen zu Beethovens Klaviersonaten und Wagners "Ring" inklusive der symbolisch kodierten Notentexte und annotierten Musikaufnahmen stellen hierfür eine exzellente Grundlage dar. Durch die Fortsetzung des Projekts soll der durch ein großes Maß an Offenheit, gegenseitigem Interesse und langfristigem Denken geprägte Austausch der Kooperationspartner weitergeführt werden und als Beispiel eines erfolgreichen interdisziplinären Dialogs im Bereich der "Digital Humanities" dienen.

Projected-Related Resources and Demonstrators

The following list provides an overview of the most important publicly accessible sources created in the CAS project:

Projected-Related Publications

The following publications reflect the main scientific contributions of the work carried out in the CAS project.

  1. Johannes Zeitler, Christof Weiß, Vlora Arifi-Müller, and Meinard Müller
    BPS: A Coherent Multi-Version Dataset for Analyzing the First Movements of Beethoven's Piano Sonatas
    Transaction of the International Society for Music Information Retrieval (TISMIR), 2024. Details Demo DOI 
    @article{ZeitlerWAM24_BPSD_TISMIR,
author = {Johannes Zeitler and Christof Wei{\ss} and Vlora Arifi-M{\"u}ller and Meinard M{\"u}ller},
title = {{BPS}: {A} Coherent Multi-Version Dataset for Analyzing the First Movements of {B}eethoven's Piano Sonatas},
journal = {Transaction of the International Society for Music Information Retrieval ({TISMIR})},
volume = {},
number = {},
pages = {},
year = {2024},
doi = {},
url-details = {},
url-pdf   = {},
url-demo = {}
}
  2. Christof Weiß and Meinard Müller
    From Music Scores to Audio Recordings: Deep Pitch-Class Representations for Measuring Tonal Structures
    ACM Journal on Computing and Cultural Heritage (JOCCH), 2024. Details Demo DOI 
    @article{WeissM24_DeepPitchClass_ACM-JOCCH,
author   = {Christof Wei{\ss} and Meinard M{\"u}ller},
title    = {From Music Scores to Audio Recordings: {D}eep Pitch-Class Representations for Measuring Tonal Structures},
journal  = {{ACM} Journal on Computing and Cultural Heritage ({JOCCH})},
volume   = {},
number   = {},
pages    = {},
year     = {2024},
doi      = {doi.org/10.1145/3659103},
url-details = {https://dl.acm.org/doi/10.1145/3659103}
url-demo = {}
}
  3. Christof Weiß, Vlora Arifi-Müller, Michael Krause, Frank Zalkow, Stephanie Klauk, Rainer Kleinertz, and Meinard Müller
    Wagner Ring Dataset: A Complex Opera Scenario for Music Processing and Computational Musicology
    Transaction of the International Society for Music Information Retrieval (TISMIR), 6(1): 135–149, 2023. PDF Details Demo DOI 
    @article{WeissAKKM23_Wagner_TISMIR,
author = {Christof Wei{\ss} and Vlora Arifi-M{\"u}ller and Michael Krause and Frank Zalkow and Stephanie Klauk and Rainer Kleinertz and Meinard M{\"u}ller},
title = {{W}agner {R}ing {D}ataset: {A} Complex Opera Scenario for Music Processing and Computational Musicology},
journal = {Transaction of the International Society for Music Information Retrieval ({TISMIR})},
volume = {6},
number = {1},
pages = {135--149},
year = {2023},
doi = {10.5334/tismir.161},
url-details = {https://transactions.ismir.net/articles/10.5334/tismir.161},
url-demo = {https://zenodo.org/doi/10.5281/zenodo.7672156},
url-pdf     = {2023_WeissAJZJJM_WagnerRingDataset_TISMIR_ePrint.pdf}
}
  4. Stephanie Klauk, Rainer Kleinertz, Pascal Schmolenzky, Christof Weiß, and Meinard Müller
    Perspektiven computergestützter harmonischer Analyse: Beethovens op. 14 Nr. 1 als Gegenstand gattungsübergreifender Korpusanalyse
    In: INFORMATIK 2023 — Designing Futures: Zukünfte gestalten, Gesellschaft für Informatik e.V.: 883–889, 2023. PDF DOI 
    @incollection{KlaukKWM23_CompHarmAnalysisBeet_GI,
author    = {Stephanie Klauk and Rainer Kleinertz and Pascal Schmolenzky and Christof Wei{\ss} and Meinard M{\"u}ller},
title     = {{P}erspektiven computergestützter harmonischer {A}nalyse: {B}eethovens op. 14 Nr. 1 als {G}egenstand gattungsübergreifender {K}orpusanalyse},
booktitle = {INFORMATIK 2023 -- Designing Futures: Zuk{\"u}nfte gestalten},
publisher = {Gesellschaft f{\"u}r Informatik e.V.},
address   = {Bonn, Germany},
year      = {2023},
doi       = {10.18420/inf2023_98},
pages     = {883--889},
url-pdf   = {2023_KlaukKWM_CompHarmAnalysisBeet_GI_Preprint.pdf}
}
  5. Christof Weiß, Meinard Müller, Stephanie Klauk, and Rainer Kleinertz
    Neue Wege für die Musikforschung: Computergestützte Analyse harmonischer Strukturen
    In: INFORMATIK 2023 — Designing Futures: Zukünfte gestalten, Gesellschaft für Informatik e.V.: 805–814, 2023. PDF DOI 
    @incollection{WeissMKK23_NeueWege_GI,
author    = {Christof Wei{\ss} and Meinard M{\"u}ller and Stephanie Klauk and Rainer Kleinertz},
title     = {{N}eue {W}ege f{\"u}r die {M}usikforschung: {C}omputergest{\"u}tzte {A}nalyse harmonischer {S}trukturen},
booktitle = {INFORMATIK 2023 -- Designing Futures: Zuk{\"u}nfte gestalten},
publisher = {Gesellschaft f{\"u}r Informatik e.V.},
address   = {Bonn, Germany},
year      = {2023},
doi       = {10.18420/inf2023_90},
pages     = {805--814},
url-pdf   = {2023_WeissMKK_NeueWege_GI_Preprint.pdf}
}
  6. Mark Gotham, Rainer Kleinertz, Christof Weiß, Meinard Müller, and Stephanie Klauk
    What if the 'When' Implies the 'What'?: Human Harmonic Analysis Datasets Clarify the Relative Role of Separate Steps in Automatic Tonal Analysis
    In Proceedings of the International Society for Music Information Retrieval Conference (ISMIR): 229–236, 2021. PDF DOI 
    @inproceedings{GothamKWMK21_Harmony_ISMIR,
author    = {Mark Gotham and Rainer Kleinertz and Christof Wei{\ss} and Meinard M{\"u}ller and Stephanie Klauk},
title     = {What if the {'When'} Implies the {'What'}?: {H}uman Harmonic Analysis Datasets Clarify the Relative Role of Separate Steps in Automatic Tonal Analysis},
booktitle = {Proceedings of the International Society for Music Information Retrieval Conference ({ISMIR})},
address   = {Online},
year      = {2021},
pages     = {229--236},
doi       = {10.5281/zenodo.5624493},
url-pdf   = {2021_GothamKWMK_Harmony_ISMIR.pdf}
}
  7. Michael Krause, Meinard Müller, and Christof Weiß
    Singing Voice Detection in Opera Recordings: A Case Study on Robustness and Generalization
    Electronics, 10(10): 1–14, 2021. PDF DOI 
    @article{KrauseMW21_OperaSingingActivity_Electronics,
author    = {Michael Krause and Meinard M{\"u}ller and Christof Wei{\ss}},
title     = {Singing Voice Detection in Opera Recordings: A Case Study on Robustness and Generalization},
journal   = {Electronics},
volume    = {10},
number    = {10},
pages     = {1214:1--14},
year      = {2021},
doi       = {10.3390/electronics10101214},
url-pdf   = {2021_KrauseMW_OperaSingingActivity_Electronics.pdf}
}
  8. Michael Krause, Meinard Müller, and Christof Weiß
    Towards Leitmotif Activity Detection in Opera Recordings
    Transactions of the International Society for Music Information Retrieval (TISMIR), 4(1): 127–140, 2021. PDF DOI 
    @article{KrauseMW21_LeitmotifDetection_TISMIR,
author    = {Michael Krause and Meinard M{\"u}ller and Christof Wei{\ss}},
title     = {Towards Leitmotif Activity Detection in Opera Recordings},
journal   = {Transactions of the International Society for Music Information Retrieval ({TISMIR})},
volume    = {4},
number    = {1},
year      = {2021},
pages     = {127--140},
publisher = {Ubiquity Press},
doi       = {10.5334/tismir.116},
url       = {https://doi.org/10.5334/tismir.116},
url-pdf   = {2021_KrauseMW_LeitmotifDetection_TISMIR_ePrint.pdf}
}
  9. Christof Weiß and Meinard Müller
    Computergestützte Visualisierung von Tonalitätsverläufen in Musikaufnahmen. Möglichkeiten für die Korpusanalyse
    In Stephanie Klauk (ed.): Instrumentalmusik neben Haydn und Mozart. Analyse, Aufführungspraxis und Edition, Saarbrücker Studien zur Musikwissenschaft 20, Königshausen & Neumann: 107–130, 2021. 
    @incollection{WeissM21_TonalityVisualization_SSM,
author    = {Christof Wei{\ss} and Meinard M{\"u}ller},
title     = {{C}omputergest{\"u}tzte {V}isualisierung von {T}onalit{\"a}tsverl{\"a}ufen in {M}usikaufnahmen. {M}{\"o}glichkeiten f{\"u}r die {K}orpusanalyse},
booktitle = {{I}nstrumentalmusik neben {H}aydn und {M}ozart. {A}nalyse, {A}uff{\"u}hrungspraxis und {E}dition},
pages     = {107--130},
year      = {2021},
editor    = {Stephanie Klauk},
publisher = {Saarbr{\"u}cker Studien zur Musikwissenschaft 20, Königshausen {\&} Neumann},
isbn = {978-3-8260-7021-1}
}
  10. Michael Krause, Frank Zalkow, Julia Zalkow, Christof Weiß, and Meinard Müller
    Classifying Leitmotifs in Recordings of Operas by Richard Wagner
    In Proceedings of the International Society for Music Information Retrieval Conference (ISMIR): 473–480, 2020. PDF DOI 
    @inproceedings{KrauseZZWM20_LeitmotifClassification_ISMIR,
author    = {Michael Krause and Frank Zalkow and Julia Zalkow and Christof Wei{\ss} and Meinard M{\"u}ller},
title     = {Classifying Leitmotifs in Recordings of Operas by {R}ichard {W}agner},
booktitle = {Proceedings of the International Society for Music Information Retrieval Conference ({ISMIR})},
address   = {Montr{\'{e}}al, Canada},
pages     = {473--480},
year      = {2020},
doi       = {10.5281/zenodo.4245472},
url-pdf   = {2020_KrauseZZWM_LeitmotifClassification_ISMIR.pdf}
}
  11. Christof Weiß, Stephanie Klauk, Mark Gotham, Meinard Müller, and Rainer Kleinertz
    Discourse Not Dualism: An Interdisciplinary Dialogue On Sonata Form in Beethoven's Early Piano Sonatas
    In Proceedings of the International Society for Music Information Retrieval Conference (ISMIR): 199–206, 2020. PDF DOI 
    @inproceedings{WeissKGMK20_Beethoven_ISMIR,
author    = {Christof Wei{\ss} and Stephanie Klauk and Mark Gotham and Meinard M{\"u}ller and Rainer Kleinertz},
title     = {Discourse Not Dualism: {A}n Interdisciplinary Dialogue On Sonata Form in {B}eethoven's Early Piano Sonatas},
booktitle = {Proceedings of the International Society for Music Information Retrieval Conference ({ISMIR})},
address   = {Montr{\'{e}}al, Canada},
pages     = {199--206},
year      = {2020},
doi       = {10.5281/zenodo.4245402},
url-pdf   = {2020_WeissKGMK_Beethoven_ISMIR.pdf}
}

Projected-Related Ph.D. Theses

  1. Michael Krause
    Activity Detection for Sound Events in Orchestral Music Recordings
    PhD Thesis, Friedrich-Alexander-Universität Erlangen-Nürnberg, 2023. Details 
    @phdthesis{Krause23_ActivityDetectionMusic_PhD,
author      = {Michael Krause},
year        = {2023},
title       = {Activity Detection for Sound Events in Orchestral
Music Recordings},
school      = {Friedrich-Alexander-Universit{\"a}t Erlangen-N{\"u}rnberg},
url-details = {},
url-pdf     = {}
}

Links

Logo_GfM