logoalt Hacker News

acdtoday at 8:24 AM1 replyview on HN

This benchmark is probably also self promotion of services. Fireworks happens to make a router. Using the router gets better performance.

https://docs.fireworks.ai/deployments/routers


Replies

RALaBargetoday at 12:55 PM

I work for a router company too, I ran some tests on all of the cheapest models and came to the same outcome where a handful of small models ran together in conjunction outperform SoTA models -- outperforms in that it got a 95% vs a 94% and I bet that changes with the day of the week. Anyways, I did get a similar result in a different sort of measurement.

show 1 reply