Of course they will be influencable. It is already possible to poison LLM output. It will happen with agents too.