Feature vector that contains the most commonly used features when converting a raw wavefile to a sequence of features.

Log Mel Spectrum

Spectral information is extracted on a mel scale, rather than equally across all frequency bands. We do this because humans hearing is more sensitive at lower frequencies than higher ones, so modeling speech recognition the same way helps performance.

Carleton College

 A mel is a unit of pitch defined such that pairs of sounds which are perceptually equidistant in pitch are separated by an equal number of mels.

Approximation of the Fourier Series on a finite interval. Converts the signal from the time domain to the frequency domain

Input: windowed signal $x[n]... x[m]$ 

Output (for each frequency band): a complex number $X[k]$ representing the magnitude and phase of that frequency component in the original signal


We compute $X[k]$ with:


$X[k] = \sum_{n=0}^{N-1} x[n]e^{-j\frac{2\pi}{N}kn} $

Discrete Fourier Transform (DFT)

An on-going but a helpful book resource about NLP
https://web.stanford.edu/~jurafsky/slp3/

Speech and Language Processing (3rd ed. draft) 

Mel Frequency

Mel Filter Bank and Log

The categorization of texts into organized groups, which is usually implemented by supervised learning.
Applications: sentiment analysis, spam detection, news categorization, topic analysis, and more.

Text Classification

The pre-training and fine-tuning paradigm is a method motivated by the goal of creating adaptable, general-purpose systems for universal language understanding and generation. It involves separating the common components of neural network architectures, such as Transformers, and training them on vast amounts of unlabeled data using self-supervision. The resulting systems, known as foundation models, can be easily adapted for specific downstream applications via fine-tuning or prompting. This paradigm shift has enormously transformed natural language processing, meaning that in many cases, large-scale supervised learning for specific tasks is no longer required.

The Pre-training and Fine-tuning Paradigm

When we plot the magnitude of each frequency from the DFT, we can visualize the spectrum of the given signal

Plotting a spectrum from DFT

https://mathworld.wolfram.com/FastFourierTransform.html

Fast Fourier Transform (Wolfram MathWorld)

An efficient algorithm for computing the DFT of a sequence. Runs in O(n log(n)) time

Learn Before

Related

Learn After