Learn Before
What effect does selective execution have on computational efficiency and model quality in MoE inference?
0
1
Tags
Ch.5 Inference - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Analysis in Bloom's Taxonomy
Cognitive Psychology
Psychology
Social Science
Empirical Science
Science
Prep Sessions
Edge-Native Mixture-of-Experts Serving with FreeToken @ University of Michigan - Ann Arbor
Ch.1 Edge Serving Bottlenecks and Dynamics - Edge-Native Mixture-of-Experts Serving with FreeToken @ University of Michigan - Ann Arbor
Edge MoE Serving and Architectural Bottlenecks - Edge-Native Mixture-of-Experts Serving with FreeToken @ University of Michigan - Ann Arbor
OpenStax Psychology (2nd ed.) Textbook
Related
Experts as Modular FFNs in LLM MoE Models
A large language model is deployed for inference across 8 powerful processing units. In one configuration, the entire model's computational graph is activated across all 8 units for every input. In a second configuration, the model is structured with 8 distinct 'expert' sub-networks, one on each unit. For a given input, a routing mechanism selects only the 2 most relevant expert sub-networks to perform computations. What is the primary efficiency benefit of the second configuration for processin
In a Mixture-of-Experts (MoE) architecture, all expert sub-networks must be hosted on a single hardware device.
Systems Challenge of Edge MoE Serving
What effect does selective execution have on computational efficiency and model quality in MoE inference?
Based on MoE operational principles, which expert sub-networks are activated for computation on this request?