That's just one way to do it. Even if you let them send you PRs or whatever, you can still act on them or not depending on how they behave, your available resources, health, mood or just whim. You don't owe anyone anything and "creating a community around a project" is not a goal you have to be striving for regardless of whether you take contributions or provide some user support or not.