Kimi K3 Architecture Notes
- The Kimi K3 architecture figure for yesterday’s big open-weight model release, along with some observations and thoughts.
- I omitted it in the figure below since it’s already very crowded, but that’s essentially the same LatentMoE as in Nemotron 3 Ultra (you can find it in my LLM Architecture Gallery if you are
Unverified
- The Kimi K3 architecture figure for yesterday’s big open-weight model release, along with some observations and thoughts.
- I omitted it in the figure below since it’s already very crowded, but that’s essentially the same LatentMoE as in Nemotron 3 Ultra (you can find it in my LLM Architecture Gallery if you are
Sources: Sebastianraschka