alt Hacker NewsI fed into the hype at first. Testing Jev and Laya, they both suffer from the same issues as LLMs that stop them being useful beyond limited classifications.
I can't see any benefits that a typical ML classifier would not be better at.
Replies
I guess I'm circling toward this view. The question is, are there things that are 1. worth doing, 2. for which jev (or jev-like systems) works well, and 3. are not worth the effort to train a custom classifier. Probably yes, but it seems like it might be a pretty narrow path. But a lot depends on #2. The trade-off between #1 and #3 is less stark the more successful one shot models are at handling use cases successfully.
I think the main argument would just be that because the model is general, you don't need to retrain it from scratch for a new problem - just tweak the input prompt. For a typical classifier there's a lot more hassle - collecting the data, training it yourself, retraining under distribution shift... In that sense Jev seems great for prototyping or small-scale use cases.
Using Jev as a plain classifier is the least interesting case. See robotic control, navigation, computer use examples, none of it possible with a classifier.
Prompt ingestion is going to be the biggest differentiator.
Being able to route prompt to features that then route to special models would be a really solid implementation.
Agreed. I tested Jev on OpenRouter this past weekend and it’s “okay” but a specific classifier is significantly better. It used to require skill to import sklearn (ok, not really), but now it’s literally one prompt and upload your Excel file or whatever and you can get your classifier out. It’ll run free, instant, more accurate.