alt Hacker NewsYes, in fact the name Lite³ was chosen because it is lighter than SQLite.
I thought about implementing something like jq or JSON query, and this is very possible. It is like sending a mini-database that can be queried at speeds thousands of times faster than any JSON library is able to parse.
One interesting effect of being a zero-copy format is that the 'parsing speed' can exceed the memory bandwidth of the CPU, since to fulfill a query you do not actually need to parse the entire dataset. You only walk the branches of the tree that are actually required.
I've talked to some other people that have also shown interest in this idea. There doesn't really seem to exist a good schemaless single-file format that supports advanced queries. There is only SQLite and maybe HDF5.
Replies
Building a jq around something like Lite^3 or JSONB is a very appealing thought.
I would very much love that. I like `jq` itself as a standard, even though I don't know how well it maps. Areas where I'd want to use Lite³:
- Microcontrollers. I find myself reaching for https://github.com/siara-cc/sqlite_micro_logger_c/tree/maste... because SQLite is just too big
- Shared memory regions/mapped files. Use it to share state between processes. Could you make mutations across processes/threads lock-free?
- Caching GPU-friendly data (i.e. image cache). I'm not sure if the current API surface/structure is page alignment friendly