Learn Before
Explain the mathematical derivation used to determine the optimal expert cache-fill count for serving missing experts of size . Include the expressions for both concurrent branches, how they are equated, and the resulting closed-form policy.
0
1
Tags
Prep Sessions
Edge-Native Mixture-of-Experts Serving with FreeToken @ University of Michigan - Ann Arbor
Ch.3 Adaptive Runtime Policies and Device Execution - Edge-Native Mixture-of-Experts Serving with FreeToken @ University of Michigan - Ann Arbor
Bandwidth-Adaptive Decode and the q* Execution Policy - Edge-Native Mixture-of-Experts Serving with FreeToken @ University of Michigan - Ann Arbor
Related
A system encounters missing experts during decode, with PCIe bandwidth and host bandwidth . Based on the optimal execution policy, how many missing experts should be routed to the concurrent PCIe cache-fill branch?
In the runtime implementation of the execution policy, a minimum of at least one cache fill is retained to ensure continuous cache warming.
Under the execution policy, what behavior does the system dynamically degenerate to as the host bandwidth approaches PCIe bandwidth ()?
Explain the mathematical derivation used to determine the optimal expert cache-fill count for serving missing experts of size . Include the expressions for both concurrent branches, how they are equated, and the resulting closed-form policy.