Evaluating an Architectural Modification for LLM Inference

A team is designing a large language model intended for deployment on edge devices with limited memory and processing power. They are considering two different architectural modifications to reduce computational demands during inference:

• Modification X: A design where, for any given input, only a specific subset of the model's total parameters are activated and used for computation. The full set of parameters must still be available in memory.
• Modification Y: A design where, aft

Choosing an Efficient LLM Architecture for a Chatbot