A 54 GB Lookup Table You Can Put Anywhere
A third of Qwen's flash model is a predictive n-gram table that never does a matmul. It quantizes as a step function, offloads to RAM for free, and even runs from SSD.
Tag
moe
1 post tagged moe.
1 post tagged moe.
A third of Qwen's flash model is a predictive n-gram table that never does a matmul. It quantizes as a step function, offloads to RAM for free, and even runs from SSD.