First, calculate the minimum edit distance from the recognized string to the reference transcription. Then, the Word Error Rate (WER) is defined as:
$$\text{Word Error Rate} = 100 \times \frac{\text{Insertions} + \text{Substitutions} + \text{Deletions}}{\text{Total Words in Correct Transcript}}$$

University of Michigan - Ann Arbor

Google

It's the standard evaluation metric for speech recognition systems. The word error rate is based on how much the word string returned by the recognizer (the hypothesized word string) differs from a reference transcription.

Word Error Rate

The minimum edit distance between two strings is defined as the minimum number of operations (also called edits) needed to transform one string into another. Operations includes insertion, deletion, substitution, and switch.

Minimum edit distance

An on-going but a helpful book resource about NLP
https://web.stanford.edu/~jurafsky/slp3/

Speech and Language Processing (3rd ed. draft) 

It's available from the National Institute of Standards and Technologies (NIST). It's a free script serving as the standard method for computing word error rates.  

SCLITE

The percentage of sentences with at least one-word error.

Sentence Error Rate

Computation Steps of Word Error Rate

Given two sequences, an alignment is "a correspondence between substrings of the two sequences". With a small change, minimum edit distance and alignment can provide the minimum cost alignment between two strings. For example, in speech recognition, minimum edit distance alignment can be used to compute the word error rate. In machine translation, alignment can help match sentences in a parallel corpus in two languages.

Alignment

The minimum edit distance algorithm is a dynamic programming algorithm commonly used in evaluating the similarity between strings. 

Given two strings, the source string $X$ of length n, and target string $Y$ of length m, we’ll define $D[i, j]$ as the edit distance between $X[1..i]$ and $Y[1.. j]$. 
- Initialization: the zeroth row and column is the distance from the empty string 
    - $D[0,0] = 0$
    - for each row i from 1 to n do: 
                           $D[i,0] \leftarrow D[i-1,0] + $del-cost$(source[i])$
    - for each column j from 1 to m do: 
                           $D[0,j] \leftarrow D[0, j-1] + $ins-cost$(target[j])$
- Recurrence relation:
    - for each row i from 1 to n do:
        - for each column j from 1 to m do:

Learn Before

Related