ELMo and GPT represent fundamentally different paradigms for adapting pretrained models to downstream natural language processing tasks. On one hand, ELMo provides context-sensitive representations but requires crafting a customized, task-specific architecture for each target task, and its pretrained parameters remain frozen during supervised learning. On the other hand, GPT offers a task-agnostic approach by using a unified Transformer decoder architecture where downstream tasks are accommodated by simply adding a linear output layer; additionally, GPT fine-tunes all of its pretrained parameters during downstream training rather than freezing them.