PCP Teaser

Lecture: Music Synchronization

After working through the material of this lecture, you should be able to answer the following questions:

  • What is the general goal of music synchronization? What is the processing pipeline of a typical music synchronization procedure?
  • What is the objective of dynamic time warping (DTW)? What is the input and what is the output of a DTW procedure?
  • How are the concepts of a feature space, a feature sequence, a local cost measure, and a cost matrix related? (See Eq. 3.12 and Eq. 3.13.)
  • What is the mathematical definition of a warping path? What are the conditions that need to be satisfied? (See Eq. 3.15 to Eq. 3.18).
  • How is the cost of a warping path defined? (See Eq. 3.20)
  • What is the mathematical definition of the DTW distance? (See Eq. 3.21.)
  • Is, in general, an optimal warping path uniquely defined?
  • How can the DTW distance computed efficiently? (See Table 3.2.)
  • What is meant by interdocument and intradocument navigation?
  • What is encoded by a tempo curve? How can it be visualized? How can one compute a tempo curve from an alignment path?

Reading Assignments

Müller, FMP, Springer 2021
Chapter 3: Music Synchronization

  • Introduction of Chapter 3
  • Section 3.2: Dynamic Time Warping (DTW)
    • Section 3.2.1: Basic Approach
  • Section 3.3: Applications
    • Section 3.3.1: Multimodal Music Navigation
    • Section 3.3.2: Tempo Curves
  • Exercises
    • Exercise 3.8
    • Exercise 3.10
    • Exercise 3.11
    • Exercise 3.12

FMP Notebooks

Slides

Videos

Question & Answer Session

MPA footer