Breaking the 1.58-bit Barrier for Ternary LLMs
- The prevailing deployment format packs five ternary weights into one byte (five-trit packing), and due to the power-of-two group sizes used in practice this rounds up to $1.625$ bits per weight.
- This effective storage bit-width treats the three symbols $\{-1,0,+1\}$ as equiprobable.
Unverified
- The prevailing deployment format packs five ternary weights into one byte (five-trit packing), and due to the power-of-two group sizes used in practice this rounds up to $1.625$ bits per weight.
- This effective storage bit-width treats the three symbols $\{-1,0,+1\}$ as equiprobable.
Sources: Arxiv