The probability of an unambiguous sentence is simply the probability of the single parse tree for that sentence.

$P(T,S) = P(T)$

 Probability of an unambiguous sentence

The probability of an ambiguous sentence is the sum of the probabilities of all the parse trees for the sentence

$\sum_{T s.t. S=yield(T)}^{} P(T)$

Probability of an ambiguous sentence

 $P(w_i | w_1, w_2, ..., w_{i-1}) = \frac{P(w_1, w_2, ..., w_{i-1}, w_i)}{P(w_1, w_2, ..., w_{i-1})}$

Next word prediction with PCFG

PCFGs are useful in language modeling as they can predict the probability of a sentence or part of a sentence

Carleton College

Applications of Probabilistic Parsing

Probabilistic augmentation of context-free grammars in which each rule is associated with a probability. 

$A \rightarrow B [p]$

A PCFG is $consistent$ if for all sentences, the probabilities for that sentence sum up to 1.

Probabilistic Context-Free Grammar (PCFG)

An on-going but a helpful book resource about NLP
https://web.stanford.edu/~jurafsky/slp3/

Speech and Language Processing (3rd ed. draft) 

To resolve disambiguation, the PCFG will calculate a probability of each parse tree (derivation) $T$ of a given sentence $S$. 

The probability of tree $T$ given $S$ is defined as:

$P(T,S) = \prod_{i=1}^{n} P(RHS_i | LHS_i)$

From there, the PCFG will choose the parse tree with the greatest probability as the correct parse of $S$.

PCFGs for Disambiguation

PCFGs for Language Modeling

The goal of PCFGs is to propose the most likely parse *T̂* of a given sentence *S*.

PCFG Parsing Problem

The main problems of PCFGs are probability estimators are as follows:
- Poor independence assumptions
- Lack of lexical conditioning


Learn Before

Related

Learn After