Release pipeline should probably run completely isolated from the main GitHub project.

Maybe a private project, that can't share any cache from the main project where public development is done.

Also only the publish step itself should have access to the publish tokens, and shouldn't run any of the code from the repo. Just publish the previously built tarball, and do nothing more. This would still allow compromising the package somehow in the build step, but at least stealing tokens should become impossible.