alt Hacker NewsI don't understand why this issue only appeared after the upgrade, not immediately after the migration to add the column?
Replies
I think it happened right after the alter table, but it was discovered after the upgrade. It's normal to have more eyes on the system after a DB upgrade, and also common to blame the DB for post-upgrade problems.
Turns out this time the DB was to blame, but not because of the upgrade.
If the alter table in the blog post is not a simplified version of what was executed (barring changing column names, of course), that means the table had no primary key before the migration, which is a problem on its own.
To be honest, I don't even know if there's a safe way out of that situation in a replication setup, but one plan I would have tried to test in that situation is: - switch binlog_format to ROW (and never look back ...) - run a noop alter table to rebuild the table and hope that with ROW format, the rows get inserted in the same order (hope really hard please, with feeling) - run the alter table
Fortunately, recent versions of MySQL have ROW as the default binlog_format.
Versions are different during the upgrade. That's how upgrade is done.
You spin up replica, upgrade it to the target engine version, ensure it's up to date and switch over. [1]
[1] https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/USER_...