Normally with gpg agent that leaves the key unlocked it is not a problem.