logoalt Hacker News

gpugreg • yesterday at 11:02 PM • 1 reply • view on HN

    > I was training custom ML models back in 2017.
Maybe you are a good person to ask my question then. I have not looked into Jev much, but is it much different from using a regular LLM and constraining its token output to the action space? (e.g. like using llama.cpp's GBNF grammars). Is it just that Jev's "confidence scores" are significantly better than the softmaxed logits? Or is there something else I am missing?

Replies

baobabKoodaa • yesterday at 11:09 PM

What you're missing is: cost and speed. Otherwise, it's very much like running an LLM and constraining the output.

I don't know how good Jev's "confidence scores" are, but I would be surprised if they were in any sense better than logits from some good LLM. One advantage of Jev here is that the confidence scores are easy to access. Most LLM API providers don't provide an easy/convenient way to access the logits. But that's a minor point, you could of course build something like this with LLMs (and many people have).

➕ show 1 reply