Introducing Bonsai 2 27B: Near-Lossless Compression in a 9x Smaller Footprint
- Two months ago, we released our first Bonsai 27B models and showed that a 27B-class multimodal model could be compressed enough to run efficiently on a local device.
- Today, we’re releasing Ternary Bonsai 2 27B, our most capable model yet.
- Ternary Bonsai 2 27B uses ternary {−1, 0, +1} weights with FP16 group-wise scaling
Unverified
- Two months ago, we released our first Bonsai 27B models and showed that a 27B-class multimodal model could be compressed enough to run efficiently on a local device.
- Today, we’re releasing Ternary Bonsai 2 27B, our most capable model yet.
- Ternary Bonsai 2 27B uses ternary {−1, 0, +1} weights with FP16 group-wise scaling
Sources: Prismml