alt Hacker NewsBut zero-shot classifiers with this level of intelligence, world knowledge, ergonomics, cost profile, and ease of use are new.
I feel like good engineering doesn't just ignore those things, or at least it didn't before recently. Now I guess social media has added a pressure to reduce everything to a hot take.
Replies
catlifeonmars • yesterday at 11:04 PM
They almost certainly would perform worse than more specialized classifiers trained with less data. It’s kind of a paradox of generalization. I think there’s an interesting space where you use generalized models to generate ad hoc specialized classifiers.
➕ show 2 replies
> Ergonomics, cost profile, and ease of use are new.
Following AI from the academic papers side; jev really feels silly. They one-pass the LLM tranformer stack and tune the output network for a probability value.
(some clever pararellization optimisations to make it viable to offer as an api, since the normal kv cashing no longer works if you oneshot the tranformer)
The largest change is the packaging; An api with a tolken based pricing, and a schema to define the output structure for quick setup.
Previous projects would probably involve installing pytorch, running a converter script on Qwen, and write a fair bit of matrix math to change the output shape.
I'm kinda amused that it took this long though.