alt Hacker NewsAt 14,000t/s that's effectively a motor cortex for an android, you no longer need to train the robot to walk, it has a general idea for how to walk (baked into the 1b model), and then just corrects based on sensor input, in real time.
At 14,000t/s that's effectively a motor cortex for an android, you no longer need to train the robot to walk, it has a general idea for how to walk (baked into the 1b model), and then just corrects based on sensor input, in real time.
I still personally think that a heavy lean into MoE will be better for that sort of thing. Our brains are subdivided into large parts but I'm sure (and I'm not a brain scientist) that those parts can be subdivided even further into systems that run at various frequencies and latencies depending on what they're used for.
I was thinking about it the other day actually. How our brains evolved structure. I imagine it was purely just down to evolution adding/clustering additional cells around the areas where additional cells were needed. And after long enough a natural brain architecture emerged.
Makes me wonder if we're on the right track with transformer architecture/attention but if it'd be more effective on a larger scale, like MoE with a billion "experts".