alt Hacker NewsIf bandwidth cost is a concern the maybe you should reconsider how you publish your site.
Like, what if you actually post something that gains traction, is it going to bankrupt you or something?
If bandwidth cost is a concern the maybe you should reconsider how you publish your site.
Like, what if you actually post something that gains traction, is it going to bankrupt you or something?
It's not just financial, you're taking up a lot of bandwidth, resources etc.
It's not just some light bump in traffic. It's a headache that shouldn't need to be dealt with if they would respect ROBOTS.txt. Quite simple really.