Learn Before
A researcher is analyzing the architecture of several prominent Large Language Models to understand common design patterns. They are specifically investigating the type of activation function used in the feed-forward network layers. Which of the following pairs of model series are both known for implementing the SwiGLU (Swish-based Gated Linear Unit) activation function?
0
1
Tags
Ch.2 Generative Models - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Recall in Bloom's Taxonomy
Cognitive Psychology
Psychology
Social Science
Empirical Science
Science
Related
Activation Function Selection in Language Model Architecture
A researcher is analyzing the architecture of several prominent Large Language Models to understand common design patterns. They are specifically investigating the type of activation function used in the feed-forward network layers. Which of the following pairs of model series are both known for implementing the SwiGLU (Swish-based Gated Linear Unit) activation function?
Architectural Rationale for Activation Function Choice