alt Hacker NewsMaybe it's just me, but the explanations of the cause don't align.
One clue was that during corruption incidents, our metrics showed that SQLite would report copying more pages from the WAL file than were actually available. If there are 10 pages in the WAL file and 20 pages get copied to the database, something is clearly wrong.
vs
it thinks some of the pages have been copied from the WAL into the main database file, but they haven’t. Those pages never get written to the database file, and that data is permanently lost.
The first says "more were copied than existed" but the second says "fewer were copied than should have been."
Like I said, it's probably just me interpreting something incorrectly.
Replies
My interpretation is that they haven't been copied because they didn't exist?
If you have 10 pages and it tries to copy 20, either those 10 pages wouldn't really be copied, or bogus data would be written.
That's how I read at least. Those things are not mutually exclusive.
I haven't looked into the actual code fix, but given the "reset" name I have to think it has to do with SQLite "thinking" it has copied more pages than it actually did.
i.e. The checkpoint starts, and a write hits after the modifications to data structures have been done but before the data has actually been put in the database. The process starts over again, but doesn't undo the changes it made to indexes etc. Hence the db thinks it holds pages that don't exist.
That's my interpretation, anyways.