One way I've seen of doing this is to include a PTC heater. It's a heating element that you feed DC. It has a positive coefficient of resistivity vs. temperature, so it'll asymptotically approach a temperature defined by the structure of the material. No PID controller required, it's just a sheet of material you include in the battery structure.

Granted, you have a minor bootstrapping issue wherein you need the battery to be warm before you use battery power, but at very low % of the battery's power capacity I suspect it's less of an issue.