alt Hacker NewsBitter lesson learned in 3... 2..
I know Dylan Field recently posted they made the Llama's 70B model do amazing things but the AI training field is an absolute snake pit.
But then again they have money to burn ig.
Bitter lesson learned in 3... 2..
I know Dylan Field recently posted they made the Llama's 70B model do amazing things but the AI training field is an absolute snake pit.
But then again they have money to burn ig.
You've misunderstood the bitter lesson if you think it applies here. It's not "frontier LLMs are always better", it's "algorithms that scale with compute/data will usually end up surpassing algorithms that are based on human understanding".
Fine-tuning an open-source LLM with reinforcement learning in order for it to be better at a specific task does not go against the bitter lesson at all.