Why I Hate LLM Confidence Scores
- Hello, Reader If someone sent you this post, you have probably tried to extrude a confidence score out of an LLM.
- I’ve now had this conversation at multiple companies with multiple people, and I’ve had it enough times that it seems like there’s a broader misunderstanding at work here.
- So my hope is that I can just send people this post instead of relitigating it in a thread every six months.
Unverified
- Hello, Reader If someone sent you this post, you have probably tried to extrude a confidence score out of an LLM.
- I’ve now had this conversation at multiple companies with multiple people, and I’ve had it enough times that it seems like there’s a broader misunderstanding at work here.
- So my hope is that I can just send people this post instead of relitigating it in a thread every six months.
Sources: Justinflick