We used to have a "cash card" in Sweden in the 90's[0]. It flopped because nobody wants to keep manually re-filling it with value all the time. It's much more convenient to have a card that pulls from your bank account (either instantly via debit or monthly via credit). In the mass market, convenience always trumps privacy.

The places where a "cash card" have gained popularity have all been using the "backdoor" of public transit payments that are so ubiquitous they also get accepted by retail (e.g. Suica in Japan, Octopus in HK, EasyCard in Taiwan, etc)

[0] https://sv.wikipedia.org/wiki/Cash_(betalsystem) based on the Belgian Proton https://en.wikipedia.org/wiki/Proton_(debit_card)