Except you have no idea if what the LLM is telling you is true

I do a lot of astrophysics. Universally LLMs are wrong about nearly every astrophysics questions I've asked them - even the basic ones, in every model I've ever tested. Its terrifying that people take these at face value

For research at a PhD level, they have absolutely no idea what's going on. They just make up plausible sounding rubbish