alt Hacker NewsI think this leads to an interesting question. What happens when the money runs out?
Right now, a lot of money is going to train new models. And we need to train new models because they get gated by their training data. And models are only as useful as their training data.
So let's say the money stops.
Do we stop training models? Do we train them slowly? Do we accept the then current models as the limit?
Replies
fn-mote • today at 12:38 AM
The money is never going to stop. It’s basic economics.
Well, the money will stop when the value of problems the LLM can solve is not increased by adding compute. Since current LLMs are getting quite good at solving problems, that might be a while.
Governments, especially the US government has got a taste of how good LLMs are at hacking. This is something that has typically been very hard to get enough people that are good at it and willing to do it for a state. Now they can spin up as many hackers as they want.
Look at how much we spend on single bombers, how many training runs can you do for that much?