alt Hacker NewsI wonder what the "waves" in LLMs are; The global firing patterns that can help us interpret what the weights are doing en-masse
Replies
altruios • yesterday at 8:35 PM
Care to define 'waves' in this context?
I suspect, if our conscious experience === the waves created by neuron's firing, that the hum of the GPU when running a LLM doesn't have the dynamic range to meaningfully talk about having a conscious experiences.
But that's armchair conjecturing.
randomImmigrant • yesterday at 11:02 PM
Why would you assume waves with LLMs? What about the structure or function of an artificial neuron prompts you to hypothesize they generate continuous waves in a bunch of nested frequency ranges?
➕ show 1 reply
I thought this research had already been done and published by Anthropic. https://www.anthropic.com/research/tracing-thoughts-language...