LLM Judges for Identity Resolution at 4x Lower Cost | Hightouch
- We wanted to test whether LLMs and decision models, specifically Jev and Luna, could solve a hard problem that we encounter regularly (grading identity resolution results).
- We found that while both models performed okay out of the box (88.8–97.3% accuracy), implementing a hybrid model that combines Jev + Opus achieves frontier accuracy at > 4x lower costs.
- Our other findings: Luna performs better than Jev out of the box, but the gap disappears on harder cases.
Unverified
- We wanted to test whether LLMs and decision models, specifically Jev and Luna, could solve a hard problem that we encounter regularly (grading identity resolution results).
- We found that while both models performed okay out of the box (88.8–97.3% accuracy), implementing a hybrid model that combines Jev + Opus achieves frontier accuracy at > 4x lower costs.
- Our other findings: Luna performs better than Jev out of the box, but the gap disappears on harder cases.
Sources: Hightouch