Improvement on Simple Greedy Inference Algorihtm 
One draw back of this greedy inference algorithm is that, for example, given $X$, let there be 3 most probable alignment “a_b”, “b_b”, “b_bb”, and let their probability be 0.4, 0.3, 0.3. The greedy algorithm would choose “a_b” and this led to output “ab”. However, both “b_b” and “b_bb” lead to “bb”, and “bb” has a total probability of 0.6, which is more likely than “ab”.

For this reason, we can improve simple greedy inference algorithm by choose the output $Y$ that has maximum conditional probability, instead of maximizing a single alignment.

Mathematically, $P_{CTC}(Y|X)=\sum_\text{all A that produce Y }p(A|X)$, and $\hat{Y}=\text{argmax}_\text{all possible Y} P_{CTC}(Y|X)$


Improvement on Simple Greedy Inference Algorithm 

In Connectionist Temporal Classification (CTC), we can simply find the best alignment by maximum its conditional probability. Mathematically, let $\mathcal{C}$ be the set of all possible alignments, then the best alignment is $\hat{a}_i=\text{argmax}_{c\in\mathcal{C}} p_i(c|X)$.

University of Notre Dame

In Connectionist Temporal Classification (CTC), we assume that the output vectors in a single sequence are all independent. Mathematically, let the input sequence be $X=\{x_1, x_2,..,x_n\}$ where $x_i$ are vectors, and let the output alignment be $A=\{a_1, a_2, …, a_n\}$ where $a_i$ are vectors, then $P_{CTC}(A|X)=\prod_{i=1}^{n}p(a_i|X)$.

Strong Conditional Independence Assumption 

An on-going but a helpful book resource about NLP
https://web.stanford.edu/~jurafsky/slp3/

Learn Before

Related

Learn After