I'm an absolute atheist and philosophical nihilist and I have done psychedelics and I've never had a problem with them. Typically find the experiences positive. So we have at least one counter-example.