Learn Before
Features in a CRF POS Tagger
the specific features are automatically populated by using feature templates. Here are some templates that only use information from , ,X,i): These templates automatically populate the set of features from every instance in the training and test set. Word shape features represent the abstract letter pattern of the word by mapping lower-case letters to ‘x’, upper-case to ‘X’, numbers to ’d’, and retaining punctuation. In summary, here are some sample feature templates that help with unknown words (the graph). The known-word templates are computed for every word seen in the training set; the unknown word features can also be computed for all words in training, or only on training words whose frequency is below some threshold. The result of the known-word templates and word-signature features is a very large set of features. Generally a feature cutoff is used in which features are thrown out if they have count < 5 in the training set. For CRF training and inference there is always a fixed set of K features with K weights, even though the length of each sentence is different.

0
1
Tags
Data Science