> He won't let you into his house with a smartphone

His mistrust might be about the phone's owner and the recording apps they might use, more than the smartphone itself.

Socially it's much safer to direct suspicion at "the smartphone" rather than the people invited to your house.