logoalt Hacker News

Reverse engineering the storage format for an undocumented database

56 pointsby pintprintlast Friday at 7:17 AM5 commentsview on HN

Comments

whsoultoday at 4:36 AM

Just leave a comment about experience like this, as I pass by.

during developing Kafka search tool, it have to support many kind of decode formats ( such as Avro + schemaRegistry, protobuf, ConnectJson(json with schema), and so on ). it recommend a type of deserializer by looking at the leading bytes. After user select final decision, it is ok if error occurs during first data deserialize, But many case it’s not occurred error, just pass through with plausibly wrong parsed data. So my solution was display 10 pre decoded sample to user, and make user select one deserializer by eye.

ConnectJson is reliable, because it has data and schema embedded in every message ( but large ). In my case using schema-registry, server1 has schema no 10, and server2 has schema no 10 but it’s different. I make a mistake select wrong server 2, data parsing is progressed but plausibly wrong data It took me a long time to find and correct some thing wrong.

elias1233today at 12:44 AM

I love this new trend of reverse engineering, amplified by AI. Been doing numerous projects like this myself.

show 1 reply
gotosun1today at 7:14 AM

Where does the KOD key come from? Can it be recovered by reverse engineering CronosPro?

show 1 reply