alt Hacker NewsTFA isn’t about consumer usage, it is about training the next generation models. An interesting thought I heard recently is that a SOTA model has about the same parameters as the number of synapses in a Golden Retriever’s brain that are not dedicated to biological processes like breathing. Most of that should be wrapped in double quotes, don’t take it literally! But that number is about 100x lower than the same synapses in a human brain.
If the next order of magnitude costs 40B, I wonder if it’s even possible to get to the one after.
just wait. the current gold rush has left any consideration of efficiency or price-performance at the side of the road. the entire enterprise is structured as a 'whoever can spend the most money wins the game for all time'. if we can get past that and invest more in theory and systems at a natural pace it'll just keep getting more affordable over time.