alt Hacker NewsTraining a model requires labels training data. Getting up and running with an LLM just requires a prompt.
I agree that a good model will outperform an LLM, but the LLM is so much faster/easier to build. That’s the usual tradeoff.
Replies
miltava • yesterday at 7:25 PM
That’s true. It could help with the cold start problem for training models and maybe replace heuristics/rules in some cases.
I just ran an experiment, I took the banking77 dataset, took the labels out, and got an LLM (Opus 5) to label them, then trained a logistic regression classifier (Jeffy http://github.com/nicobrenner/jeffy) on the resulting set
Got the same accuracy and f1 from the Opus-label dataset than the original one (I realize that Opus might have banking77 in its training data, so might need to account for that somehow)
But the point is: it's way cheaper to run the LLM model over your data to label once, and then use a cheap classifier, than using the LLM directly for classification