logoalt Hacker News

bewareofscamstoday at 3:38 PM1 replyview on HN

Only useful benchmarks are those you (in particular) don't have access to.


Replies

rhdunntoday at 4:52 PM

The only useful benchmarks are those you've created for your specific workflow. Only then can you assess whether a given model is better or worse for what you are using it for.

There are tools like promptfoo designed for this.