Is this prevented by physics or cost or just no one has the motivation?
There are a number of compact shortwave (radio amateurs prefer the term "high frequency" or HF, in contrast to VHF, UHF) transceivers. The impracticality is from the size of an efficient antenna.
I have personally made voice (single-sideband or SSB, which is analog like AM without wasting energy transmitting a carrier or redundant sideband) contacts with a 5 watt portable (Elecraft KX2) between countries in Europe, using a meter-long whip antenna and a trailing counterpoise wire.
These radios are incredibly complex weak-signal equipment, and that is reflected in the price.
That said, it is fun. Using morse code to do the same is even more fun.
I would never rely on this for off-the-grid communication, though.
Compact HF/shortwave radios with transmit capability exist, but they're pretty expensive and are generally definitely portable but not quite handheld. The biggest user of such equipment is the military, so a lot of the tech is engineered for that with civilian/amateur use as an afterthought. ICOM, Yaesu, and Xiegu are probably the best known makers, and you're looking at ~$1000 as table stakes for a modern one, though there are some slightly cheaper options.
Handheld CB radios do exist and are cheap, but I've never really used them.