NEEDLE: The benchmark your search engine can't memorize - Keenable.ai
- Existing benchmarks can't compare search enginesMost of the existing benchmarks are static: a fixed set of questions frozen in time.
- Static benchmarks allow easy overfitting.
- Systems can overfit on test data even indirectly.
Unverified
- Existing benchmarks can't compare search enginesMost of the existing benchmarks are static: a fixed set of questions frozen in time.
- Static benchmarks allow easy overfitting.
- Systems can overfit on test data even indirectly.
Sources: Keenable