It's a fairly shallow point that ignores how laws work.

The premise that the law doesn't apply because you're not in the country is false. The constitution applies generally everywhere to all Americans, it's just that what's regarded as reasonable differs during a border search. IANAL, so just my lay opinion on this. Just to validate this, it's only because the constitution exists that the border authorities have any legal basis in doing inspections.