Recreating Minecraft is Not a Benchmark
- On paper these look like harder, more visual problems for a model to solve, there’s a reason they’re as big as they are.
- I’ve started calling them demo-benchmarks, visual and understandable enough for everyone to get but finite enough for the next model to be “perfect” on.
Unverified
- On paper these look like harder, more visual problems for a model to solve, there’s a reason they’re as big as they are.
- I’ve started calling them demo-benchmarks, visual and understandable enough for everyone to get but finite enough for the next model to be “perfect” on.
Sources: Kuber