Audio Signal Processing for Music Applications

Audio Signal Processing for Music Applications

Audio Signal Processing for Music Applications

Universitat Pompeu Fabra of Barcelona, Stanford University

About this course: In this course you will learn about audio signal processing methodologies that are specific for music and of use in real applications. We focus on the spectral processing techniques of relevance for the description and transformation of sounds, developing the basic theoretical and practical knowledge with which to analyze, synthesize, transform and describe audio signals in the context of music applications.
The course is based on open software and content. The demonstrations and programming exercises are done using Python under Ubuntu, and the references and materials for the course come from open online repositories. We are also distributing with open licenses the software and materials developed for the course.

Who is this class for: This course is primary aimed at advanced undergraduate or master students, along with professionals, interested in signal processing, programming and music.

Introduction to the course, to the field of Audio Signal Processing, and to the basic mathematics needed to start the course. Introductory demonstrations to some of the software applications and tools to be used. Introduction to Python and to the sms-tools package, the main programming tool for the course.

11 videos, 1 reading

Video: Teaser

Video: Welcome

Video: Introduction to Audio Signal Processing

Video: Course outline

Video: Basic mathematics

Video: Introduction to Audacity

Video: Introduction to SonicVisualizer

Video: Introduction to sms-tools

Video: Introduction to Python

Video: Python and sounds

Video: sms-tools software

Reading: Advanced readings and videos

Graded: Basics

Graded: Python and sound

WEEK 2

Discrete Fourier transform

The Discrete Fourier Transform equation; complex exponentials; scalar product in the DFT; DFT of complex sinusoids; DFT of real sinusoids; and inverse-DFT. Demonstrations on how to analyze a sound using the DFT; introduction to Freesound.org. Generating sinusoids and implementing the DFT in Python.

6 videos, 1 reading

Video: DFT 1

Video: DFT 2

Video: Analyzing a sound

Video: Introduction to Freesound

Video: Sinusoids

Video: DFT

Reading: Advanced readings and videos

Graded: DFT

Graded: Sinusoids and DFT

WEEK 3

Fourier theorems

Linearity, shift, symmetry, convolution; energy conservation and decibels; phase unwrapping; zero padding; Fast Fourier Transform and zero-phase windowing; and analysis/synthesis. Demonstration of the analysis of simple periodic signals and of complex sounds; demonstration of spectrum analysis tools. Implementing the computation of the spectrum of a sound fragment using Python and presentation of the dftModel functions implemented in the sms-tools package.

7 videos, 1 reading

Video: Fourier properties 1

Video: Fourier properties 2

Video: Periodic signals

Video: Complex sounds

Video: Spectrum

Video: Fourier properties

Video: dftModel

Reading: Advanced readings and videos

Graded: Fourier properties

Graded: Fourier Properties

WEEK 4

Short-time Fourier transform

STFT equation; analysis window; FFT size and hop size; time-frequency compromise; inverse STFT. Demonstration of tools to compute the spectrogram of a sound and on how to analyze a sound using them. Implementation of the windowing of sounds using Python and presentation of the STFT functions from the sms-tools package, explaining how to use them.

6 videos, 1 reading

Video: STFT 1

Video: STFT 2

Video: Spectrogram

Video: Analyzing a sound

Video: Windows

Video: STFT

Reading: Advanced readings and videos

Graded: Short-time Fourier transform

Graded: Short-time Fourier Transform (STFT)

WEEK 5

Sinusoidal model

Sinusoidal model equation; sinewaves in a spectrum; sinewaves as spectral peaks; time-varying sinewaves in spectrogram; sinusoidal synthesis. Demonstration of the sinusoidal model interface of the sms-tools package and its use in the analysis and synthesis of sounds. Implementation of the detection of spectral peaks and of the sinusoidal synthesis using Python and presentation of the sineModel functions from the sms-tools package, explaining how to use them.

8 videos, 1 reading

Video: Sinusoidal model 1

Video: Sinusoidal model 2

Video: Sinusoidal model 3

Video: Sinusoidal model

Video: Analyzing a sound

Video: Peak detection

Video: Sinusoidal synthesis

Video: sineModel

Reading: Advance reading

Graded: Sinusoidal model

Graded: Sinusoidal model

WEEK 6

Harmonic model

Harmonic model equation; sinusoids-partials-harmonics; polyphonic-monophonic signals; harmonic detection; f0-detection in time and frequency domains. Demonstrations of pitch detection algorithm, of the harmonic model interface of the sms-tools package and of its use in the analysis and synthesis of sounds. Implementation of the detection of the fundamental frequency in the frequency domain using the TWM algorithm in Python and presentation of the harmonicModel functions from the sms-tools package, explaining how to use them.

7 videos, 1 reading

Video: Harmonic model

Video: F0 detection

Video: Pitch detection

Video: Harmonic model

Video: Analyzing a sound

Video: F0 detection

Video: harmonicModel

Reading: Advanced readings

Graded: Harmonic model

Graded: Harmonic Model

WEEK 7

Sinusoidal plus residual model

Stochastic signals; stochastic model; stochastic approximation of sounds; sinusoidal/harmonic plus residual model; residual subtraction; sinusoidal/harmonic plus stochastic model; stochastic model of residual. Demonstrations of the stochastic model, harmonic plus residual, and harmonic plus stochastic interfaces of the sms-tools package and of its use in the analysis and synthesis of sounds. Presentation of the stochasticModel, hprModel and hpsModel functions implemented in the sms-tools package, explaining how to use them.

8 videos, 1 reading

Video: Stochastic model

Video: Sinusoidal plus residual modeling

Video: Stochastic model

Video: Harmonic plus residual model

Video: Harmonic plus stochastic model

Video: stochasticModel

Video: hprModel

Video: hpsModel

Reading: Advanced readings

Graded: Sinusoidal plus residual model

Graded: Sinusoidal plus residual

WEEK 8

Sound transformations

Filtering and morphing using the short-time Fourier transform; frequency and time scaling using the sinusoidal model; frequency transformations using the harmonic plus residual model; time scaling and morphing using the harmonic plus stochastic model. Demonstrations of the various transformation interfaces of the sms-tools package and of Audacity. Presentation of the stftTransformations, sineTransformations and hpsTransformations functions implemented in the sms-tools package, explaining how to use them.

9 videos, 1 reading

Video: Sounds transformations 1

Video: Sounds transformations 2

Video: Morphing with STFT

Video: Time scaling

Video: Pitch changes

Video: Morphing with HPS

Video: stftTransformations

Video: sineTransformations

Video: hpsTransformations

Reading: Advanced readings

Graded: Sound transformations

Graded: Transformations

WEEK 9

Sound and music description

Extraction of audio features using spectral analysis methods; describing sounds, sound collections, music recordings and music collections. Clustering and classification of sounds. Demonstration of various plugins from SonicVisualiser to describe sound and music signals and demonstration of some advance features of freesound.org. Presentation of Essentia, a C++ library for sound and music description, explaining how to use it from Python. Programming with the Freesound API in Python to download sound collections and to study them.

6 videos

Video: Audio features

Video: Sound and music description

Video: Sound descriptors

Video: Freesound

Video: Intro to Essentia

Video: Freesound API

Graded: Sound and music description

Graded: Sound and music description

WEEK 10

Concluding topics

Audio signal processing beyond this course. Beyond audio signal processing. Review of the course topics. Where to learn more about the topics of this course. Presentation of MTG-UPF. Demonstration of Dunya, a web browser to explore several audio music collections, and of AcousticBrainz, a collaborative initiative to collect and share music data.

6 videos, 1 reading

Video: Beyond audio processing

Video: Review

Video: MTG-UPF

Video: Goodbye

Video: Dunya

Video: AcousticBrainz

Reading: Advanced readings

Graded: Concluding topics

Graded: A music piece combining sounds and their transformations

FAQs

How It Works

Coursework

Each course is like an interactive textbook, featuring pre-recorded videos, quizzes and projects.

Help from Your Peers

Connect with thousands of other learners and debate ideas, discuss course material,
and get help mastering concepts.

Creators

Universitat Pompeu Fabra of Barcelona

Pompeu Fabra University (UPF) is a modern public university, conveniently located in the centre of Barcelona (Spain) with the aim of providing top quality education and standing out as a research-based university. UPF is both a specialised university with a unique teaching model and a cutting-edge research institution. UPF places a strong emphasis on quality teaching, based on comprehensive education and student-centred learning, and innovation in the learning processes. UPF’s MOOCs are produced within this general goal.

Stanford University

The Leland Stanford Junior University, commonly referred to as Stanford University or Stanford, is an American private research university located in Stanford, California on an 8,180-acre (3,310 ha) campus near Palo Alto, California, United States.

Ratings and Reviews

Rated 4.8 out of 5 of 95 ratings

Lots of learning in 10 weeks. Teacher is good takes time and explain things well. Lectures include theory, demonstration and programming which helped me learn the basics really well. Thank you !

Excellent

DA

Excellent material for an interesting subject. The video lectures are very good. The course does need some updates for Python3 and associated libraries to stay current.

SS

Great Course, lots of concepts to learn and implement! I had amazing time doing the assignments. Should be an ASPMA part 2 course.