The benchmarkpocalypse
- There's been a lot of talk about the vulnpocalypse, to which I don't have much to add because I'm not a security person, but I haven't seen much discussion on the closely related (and to be fair, less serious, issue), the benchmarkpocalypse.
- While it's become easier than ever to make serious performance gains, it's also become easier than ever to reward hack a benchmark and make fake performance gains.
- The former is probably happening quietly across many different companies, but the latter is something I see at least once a week nowadays.
Unverified
- There's been a lot of talk about the vulnpocalypse, to which I don't have much to add because I'm not a security person, but I haven't seen much discussion on the closely related (and to be fair, less serious, issue), the benchmarkpocalypse.
- While it's become easier than ever to make serious performance gains, it's also become easier than ever to reward hack a benchmark and make fake performance gains.
- The former is probably happening quietly across many different companies, but the latter is something I see at least once a week nowadays.
Sources: Danluu