Code

RNNLMScratch Class

The RNNLMScratch class implements an RNN-based language model from scratch by composing a previously defined RNN module with an output projection layer. It extends a Classifier base class and accepts an RNN instance, the vocabulary size, and a learning rate as constructor arguments. Because a language model's inputs and outputs are drawn from the same vocabulary, both share the same dimensionality, which equals the vocabulary size. The output layer is defined by a learnable weight matrix WhqRhimesq\mathbf{W}_{hq} \in \mathbb{R}^{h imes q} (where hh is the number of hidden units and qq is the vocabulary size), initialized from a scaled normal distribution, and a bias vector bqRq\mathbf{b}_q \in \mathbb{R}^{q}, initialized to zeros. These parameters project each hidden state to a vector of logits over the vocabulary.

0

1

Updated 2026-05-14

Contributors are:

Who are from:

Tags

D2L

Dive into Deep Learning @ D2L