Due to the autoregressive nature of its language modeling objective, the Generative Pre-Training (GPT) model processes text strictly in a forward, left-to-right direction. Consequently, a word's representation is determined solely by the context to its left. For example, in the phrases 'i went to the bank to deposit cash' and 'i went to the bank to sit down', the left context for the word 'bank' is identical in both cases. Because GPT cannot look ahead to the rightward context that differentiates the intended meaning, it returns the exact same representation for 'bank' in both sentences, exposing a critical limitation in its context sensitivity.