Node-splitting increases the size of the grammar and reduces the amount of training data available for each grammar rule. This leads to overfitting, so it is important to arrive at a correct level of granularity for a particular training set. The split and merge algorithm starts with a simple X-bar grammar and alternately splits and merges non-terminals to find the set of annotates nodes that maximizes the likelihood of training set treebank. 

Problems with Node-splitting

We can split NP non-terminals into two versions: one for subjects, and one for objects. Having two nodes ($NP_{subject}$ and $NP_{object}$) would allow us to correctly model their different distributional properties since we could have different probabilities for the rules $NP_{subject} \to PRP$ and $NP_{object} \to PRP$. The intuition of splits can be implemented by doing parent annotation, in which we annotate each node with its parent in the parse tree. A PCFG can also be improved by splitting pre-terminal POS nodes. Adding specific tags for parts of speech can be useful in improving PCFG modeling. 


Columbia University

The main problems of PCFGs are probability estimators are as follows:
- Poor independence assumptions
- Lack of lexical conditioning


Problems with PCFGs

An on-going but a helpful book resource about NLP
https://web.stanford.edu/~jurafsky/slp3/

Speech and Language Processing (3rd ed. draft) 

 CFG rules impose an independence assumption on probabilities that leads to poor modeling of structural dependencies across the parse tree. In a CFD the expansion of a non-terminal is independent of the context, which means that it is not dependent on other nearby non-terminals in the parse tree. Similarly, in a PCFG, the probability of a particular rule is independent of the rest of the tree. Therefore, in a PCFG, we compute the probability of a tree by multiplying the probabilities of each non-terminal expansion. However, this results in poor probability estimates because in English the choice of how a node expands can actually depend on the location in the parse tree of that node. For example, it turns out that NPs which are syntactic subjects are more likely to be pronouns, and NPs which are syntactic objects are more likely not to be. There is no way to represent this contextual difference in the probabilities of a PCFG. 


Poor independence assumptions in PCFGs


CFG rules don’t model syntactic facts about specific words, leading to problems with subcategorization ambiguities, preposition attachment, and coordinate structure ambiguities. Words do play a role in PCFGs since the parse probability includes the probability of a word given a part-of-speech. Depending on how these probabilities are set, a PCFG will always either prefer NP attachment or VP attachment. As it happens, NP attachment is slightly more common in English, so if we trained these rule probabilities on a corpus, we might always prefer NP attachment, causing us to misparse this sentence. Probabilistic context-free grammars are incapable of modeling important structural and lexical dependencies. 

Learn Before

Related

Learn After