- Maximizing the similarity of target words with actual context words.
- Minimizing the similarity of target words with 'k' negative context words or non-neighbor words

Loss function for one w with cpos , cneg1 ...cnegk

A standard procedure for training deep neural networks involves optimizing the model's parameters. This is achieved by minimizing a loss function, which quantifies the model's error on the training data. An optimization algorithm, such as stochastic gradient descent, is used to iteratively adjust the parameters to reduce this loss.

Model Parameter Optimization via Loss Minimization

From the given set of positive and negative training examples and initial weight for embeddings the goal of the learning algorithm is to learn to maximize the similarity of target word and context word from the positive examples and to minimize the similarity of target word and context word from the negative examples.

San Jose State University

- A corpus of text is taken as input for learning the skip-gram embeddings by the learning algorithm.
- A vocabulary of size N is also chosen.
- For each word skip-gram algorithm learns two embeddings target embedding and context embedding.

Learning skip-gram embeddings


An on-going but a helpful book resource about NLP
https://web.stanford.edu/~jurafsky/slp3/

Speech and Language Processing (3rd ed. draft) 

- The figure below shows an example of training data, apricot being the target word and 'tablespoon ,of ,jam and a' being the context word.
- Context window is $ L = \pm 2 $.              

Learn Before

Related

Learn After