Learn Before
Concept

Classifier for word2vec

The classifier for word2vec is a binary logistic regression, which applies sigmoid function. given a tuple of words(w,c), where wwis the target word, ccis one of the context words: The possibility that cc is a context word: P(+w,c)=σ(cw)=11+exp(,cw)P\left ( +|w,c \right ) = \sigma(c \cdot w)=\frac{1}{1+exp(- , c \cdot w)} The possibility that cc is not a context word: P(w,c)=1P(+w,c)=σ(,cw)=11+exp(cw)P\left ( -|w,c \right ) = 1-P\left ( +|w,c \right )=\sigma(- , c\cdot w)= \frac{1}{1+exp(c \cdot w)} But there are several context words: P(+w,c1:L)=i=1Lσ(ciw)=i=1L11+exp(,ciw)P\left ( +|w,c_{1:L} \right ) = \prod^L_{i=1}\sigma(c_i \cdot w)=\prod^L_{i=1}\frac{1}{1+exp(- , c_i \cdot w)}

0

2

Updated 2021-10-15

Tags

Data Science