> Just to heat my house for a few days would need tons of rock at temperatures I'd allow in my house (I have kids and so I worry they will open the rock container and get burnt). Rocks are not free

Sand/rock heat storage is for industrial and utility scale application, not your house.

> You can also use water for storage, but again you have the issue needing a tank (and water is a drowning hazard, plus the worry about leaks...)

Tank heat pump water heaters are available and can already do load shifting on a daily basis (using off peak cheaper and cleaner electricity). I have one and it works great, and I guarantee it isn't a drowning hazard.

There are versions of these systems that can store enough heat to heat a house, not just provide hot water at the tap.