1Cademy - Historical Perspective: Natural Language Processing

Learn Before

Natural language processing

Reference

Historical Perspective: Natural Language Processing

Natural Language Processing has been developing alongside machine learning techniques for at least since 1990. In 1986 Rumelhart et al. developed one of the first instances of symbolic encoding for a neural network like structure. In 1990 Deerwester et al. then adapted this idea of symbolic encoding into word encoding which could then in theory be used in an NLP neural network.

From 1991 until 2001, however, encoded word processing was ignored in favor of attempts at creating a character recognizing network instead. Such a system would have to learn both grammatical structure and the spelling of every word it would use in that structure. As a result, in 2001 when Bengio et al. reintroduced word processing, it initially had a higher performance than most other character processing networks.

Although word processing networks were initially more performant, over time both methods for creating NLP networks have improved, and continue to show signs that more improvement may be possible. As such, it is hard to determine which method will ultimately prove more effective than the other.

0

1

Updated 2021-08-05

Contributors are:

JM

Who are from:

References

Deep Learning

Learn Before

Related