 CFG rules impose an independence assumption on probabilities that leads to poor modeling of structural dependencies across the parse tree. In a CFD the expansion of a non-terminal is independent of the context, which means that it is not dependent on other nearby non-terminals in the parse tree. Similarly, in a PCFG, the probability of a particular rule is independent of the rest of the tree. Therefore, in a PCFG, we compute the probability of a tree by multiplying the probabilities of each non-terminal expansion. However, this results in poor probability estimates because in English the choice of how a node expands can actually depend on the location in the parse tree of that node. For example, it turns out that NPs which are syntactic subjects are more likely to be pronouns, and NPs which are syntactic objects are more likely not to be. There is no way to represent this contextual difference in the probabilities of a PCFG. 


Poor independence assumptions in PCFGs


CFG rules don’t model syntactic facts about specific words, leading to problems with subcategorization ambiguities, preposition attachment, and coordinate structure ambiguities. Words do play a role in PCFGs since the parse probability includes the probability of a word given a part-of-speech. Depending on how these probabilities are set, a PCFG will always either prefer NP attachment or VP attachment. As it happens, NP attachment is slightly more common in English, so if we trained these rule probabilities on a corpus, we might always prefer NP attachment, causing us to misparse this sentence. Probabilistic context-free grammars are incapable of modeling important structural and lexical dependencies. 

Lack of lexical conditioning in PCFGs


The main problems of PCFGs are probability estimators are as follows:
- Poor independence assumptions
- Lack of lexical conditioning


Columbia University

A **probabilistic context-free grammar (PCFG)** is a probabilistic augmentation of a context-free grammar in which each production rule is associated with a probability:

$$A \rightarrow B [p]$$

A PCFG is *consistent* if for all sentences, the probabilities for that sentence sum up to 1.

Probabilistic Context-Free Grammar (PCFG)

An on-going but a helpful book resource about NLP
https://web.stanford.edu/~jurafsky/slp3/

Speech and Language Processing (3rd ed. draft) 

To resolve ambiguity, a Probabilistic Context-Free Grammar (PCFG) calculates the probability of each possible parse tree (derivation) $$T$$ for a given sentence $$S$$. The joint probability of tree $$T$$ and sentence $$S$$ is defined as the product of the probabilities of all rules used in the derivation: $$P(T,S) = \prod_{i=1}^{n} P(RHS_i | LHS_i)$$. The PCFG then chooses the parse tree with the greatest probability as the correct parse of $$S$$.

PCFGs for Disambiguation

PCFGs are useful in language modeling as they can predict the probability of a sentence or part of a sentence

PCFGs for Language Modeling

The goal of PCFGs is to propose the most likely parse *T̂* of a given sentence *S*.

Learn Before

Related

Learn After