logoalt Hacker News

mgraczykyesterday at 2:29 PM1 replyview on HN

What evidence did you collect that was not automated?


Replies

yearolinuxdsktpyesterday at 7:11 PM

A startup might have trouble with, and might not have enough automation for:

- proving churned customer data was deleted completely and within the agreed-on period of time

  - - not enough to have a record

  - - auditors will ask you to prove the data is not laying around
- proving all changes shipped are reviewed and linked to tracked work

- proving branch rules are set to require PRs and prohibit changing history on release/trunk branches

  - - auditors will ask you to show live that you can’t approve your own changes

  - - some auditors might ask you for an audit log to prove no unexpected branch rule changes occurred —- depending on the observation period, you might have to build your own audit log capture to prove this
- proving you performed penetration testing

- proving you performed a disaster recovery test in production with the frequency you claim (e.g. annually)

  - - running a DR test might be more than a few hours depending on your data size and level of infra automation

  - - this is often something that startups are ready to execute, but don’t invest a lot of time automating
- proving you have and enforce full-disk-encryption on all your employee laptops

  - - this is automated with MDM but a startup might not be running an MDM yet
- proving you are rotating credentials on the frequency you ascribe to in your policies

  - - automated reports are available for some credentials, e.g. AWS keys, but takes more work for smaller vendors

  - - even with AWS, you might discover you forgot to rotate something, and it might be because it’s non-trivial to execute
- perform quarterly access reviews

  - - some systems are more difficult/time consuming to inspect against your employee and permissions list

  - - ideally this is automated, but often times at a startup, you might not have fully automated authorization and access control, such that when employees change teams or leave the company, that you get notified and don’t miss it
- proving that you act on performance or reliability alerts

  - - auditors will ask you to show live some examples of past alerts and that someone handled it

  - - auditors will often ask you to show live that these alerts are consistently configured for all your production system —- startups might not have the alerting and PagerDuty-like setup be fully automated (e.g. with Terraform)