logoalt Hacker News

cmiles8yesterday at 10:15 PM10 repliesview on HN

This interview with an AWS leader isn’t aging well, from CBS Sunday morning:

Pogue asked, "I don't mean to give anyone ideas, but let's say I figured out that one of these unmarked buildings was an AWS data center, and I blew it up. Are you saying that it's so backed up and redundant that you probably wouldn't notice?" Wood replied, "Yeah, you wouldn't notice. I mean, we might be a bit upset, but you wouldn't notice!"

https://www.cbsnews.com/news/cloud-computing-loudoun-county-...


Replies

krickyesterday at 10:49 PM

That is actually surprising to me. Claims like that are pretty common, they make sense and they should be true, so even though I don't really know AWS (/Backblaze/Azure/whatever) redundancy planning in enough detail, I used to trust them. It's really worrying when they outright say it will be ok, and then a week later it turns out to be not ok.

show 3 replies
echoangleyesterday at 10:54 PM

Isn’t the problem that multiple datacenters in one zone were blown up?

show 3 replies
jasonkhanlartoday at 8:07 AM

articleAuthorId: 30dacdbc-6a8a-11e2-9d12-0018fe8a00b0

articleAuthorName: cbsnews (hidden byline)

articleSecondaryAuthors: n/a

articleEditors: n/a

Is the author a human or machine? Google shows 1 result for "30dacdbc-6a8a-11e2-9d12-0018fe8a00b0" and Brave Search shows 5 results.

Betelbuddytoday at 6:52 AM

Clearly since the MBAs took over AWS standards are not anymore what they used to be. That marketing guy should not be talking to the press, as he does not have the skills, and if somebody happens to say...our data center we wont lose any data if we have an issue, without qualifying it will depend on what quality of service, and usage of our services you setup ...is the type of technical answer that should make a hiring interview stop at the moment.

He is also violating an enormous amount of compliance requirements, by disclosing the location of the data center, and having strange people inside making a tour. Did he vet the crew and their accompanying party? Did one of them accidentally left some kind of device within the insider perimeter? There at least one or two ISO certifications he is violating there. As customer I would be asking questions...

AWS always made very clear they wont copy your data to another region as only you know what your compliance and data residency requirements are. But at the same time they always said, its up to you to come your with your disaster recovery strategy based on your project requirements. And it has always been the case copying your critical data to another region is one of the first things on your check list.

And their Well Architected Framework and other docs make this plenty clear:

"It is a good practice to always make backups of your data, and copy these to another site (such as another AWS Region)."

Also...

"All DR strategies require that data sources are backed up within the AWS Region, and then those backups are copied to the recovery Region."

And also for single-Region / Multi-AZ architectures:

"Where possible, you should also copy data backups to another AWS Region as an additional layer of protection."

"AWS Architecture Blog — Disaster Recovery Architecture on AWS, Part II" has a whole section named "Backup to another AWS Region": "By copying your data to another Region, you can handle the largest scope of disasters."

https://aws.amazon.com/blogs/architecture/disaster-recovery-...

Or "Creating backup copies across AWS Regions" - https://docs.aws.amazon.com/aws-backup/latest/devguide/cross...

This whole thread of people literally saying , "on no I trusted them...I did not know they could lose my data", with no technical context...is the the kind of incompetence I would expect from a generation raised on vibe coding and llm prompt driven miseducation...

steveBK123today at 2:16 AM

There's something reassuring in this for me.

There's a lot of magic & handwaving from hyperscalers like AWS about redundancy. I always wondered about some of the engineering to make this absolutely (and literally) bullet proof. At the end of the day most of their answers when you push hard enough involved paying 2-3x to run everything across multiple zones/regions, and lots of awareness in your application to handle this.

In any case, I think it's good that when a data center blows up the data is lost. Noteworthy for future skynet situation, etc.

show 1 reply
jasonfarnonyesterday at 11:19 PM

that quote is definitely making its way into a lawsuit

show 1 reply
ExoticPearTreetoday at 6:11 AM

Unless you have a synchronous like setup where you don’t acknowledge data writes unless the remote has aconowledged them first, you will lose data in case your datacenter is hit by a warhead.

Now, there are a few things to consider:

- AWS best practices recommend multiple AZs for workloads and cross-region backups for things like databases and other “stateful” data

- You have to read the fine-print on what AWS offers in terms of recovery: do they reffer to their own infrastructure when they say “you won”t notice” or your data

When Google’s Paris colocation facility was flooded and all AZs there went dark, they sent an email saying “restore from backup in another region and if we can restore your data, we will make it availbale to you”. They did not even issue credits for the downtime.

show 1 reply
sparklingyesterday at 10:25 PM

He forgot to add "if it's Multi-AZ" ;)

show 3 replies
Dban1today at 6:05 AM

But AI chatbot advised him that was the case!