alt Hacker NewsI took the point as: don't make the LLM the classifier. Use it to turn messy input into useful features, then let a normal model make the actual decision. That gives you thresholds/calibration you can inspect.
What I'm not sure about is how stable those features are when you switch the underlying LLM or model version.
Replies
ltbarcly3 • yesterday at 10:53 PM
That isn't what they did here. They took the output of the LLM as one feature, then added 17 other features, then piped it into a crappy model and got a 3% improvement.
Much like how you shouldn't ask the LLM to solve a (repeated, logical) problem, but you should instead prompt it to generate code that you can inspect/test/fix/reuse.