alt Hacker NewsThe NPU in previous AMD generation has unfortunately it's own dedicated RAM that is far too small for an LLM.
But even those should comfortably run an SLM of around 100-400M parameters at crazy high speeds and with minimal power usage.
The NPU in previous AMD generation has unfortunately it's own dedicated RAM that is far too small for an LLM.
But even those should comfortably run an SLM of around 100-400M parameters at crazy high speeds and with minimal power usage.
> an SLM of around 100-400M parameters at crazy high speeds and with minimal power usage
Software support for that kind of usage, when available, will be immensely impactful.