Why would it need to be a container?
My ollama and GPU are in k8s.
Are you asking why people run things in a container?
My ollama and GPU are in k8s.