alt Hacker NewsJev seems to have three benefits:
1) It's very cheap and fast - you provide one input and many potential classifications, and the compute to ingest the input is shared.
2) It generates structured output natively - guaranteed to be correct
3) It's output probabilities are calibrated to actually mean something
OpenAI, or anyone else, could certainly replicate it - there are already articles guessing how Jev achieves its "parallel" classifications, but it seems the AI companies need to decide are they in the business of providing intelligence/tokens, or are they in the application business trying to compete with all their customers (not that Jev uses OpenAI).
Replies
Just to clarify:
> 2) It generates structured output natively - guaranteed to be correct
It's not guaranteed to be correct: it's guaranteed to be _formatted in a particular way_. You can get the same thing with grammars on any LLM.
Jev and Jev-like models have other advantages, but I feel like people forget grammars exist for LLMs.
(3) seems to be the hard one, you have to have training data with accurate probabilities, maybe, but perhaps not since people are primed to trust
> It's output probabilities are calibrated to actually mean something
Don't fall for marketing BS so easily.
Jev can output drastically different probabilities if you simply reorder the list of choices. And Jev's "confidence" output is fake/redundant - it's just a formula applied to probabilities, it conveys no additional information.
I bet they will eventually "fix" (read hide under the rug) the ordering problem by ordering the list on the backend before feeding to the model.