So does llm inference. You're lucky if you hit 40% of the advertised flops.
Right, but datacenter GPUs optimized for LLM training/inference would have a bandwidth:compute ratio scaled to that workload.
alt Hacker News
Right, but datacenter GPUs optimized for LLM training/inference would have a bandwidth:compute ratio scaled to that workload.