logoalt Hacker News

sourcegriftyesterday at 9:54 AM6 repliesview on HN

We have everything optimized, and yet somehow DB queries need to be "interpreted" at runtime. There's no reason for DB queries to not be precompiled.


Replies

jpfryesterday at 3:18 PM

The "byte-code" coming from the query planner typically only has a handful of steps in a linear sequence. Joins, filters, and such. But the individual steps can be very costly.

So there is not much to gain from JITing the query plan execution only.

JITing begins to make more sense, when the individual query plan steps (join, filter, ...) themselves be specialized/recompiled/improved/merged by knowing the context of the query plan.

catlifeonmarsyesterday at 10:45 AM

This is a neat idea. I want to take it further and precompile the entire DBMS binary for a specific schema.

show 1 reply
Asm2Dyesterday at 12:16 PM

Many SQL engines have JIT compilers.

The problems related to PostgreSQL are pretty much all described here. It's very difficult to do low-latency queries if you cannot cache the compiled code and do it over and over again. And once your JIT is slow you need a logic to decide whether to interpret or compile.

I think it would be the best to start interpreting the query and start compilation in another thread, and once the compilation is finished and interpreter still running, stop the interpreter and run the JIT compiled code. This would give you the best latency, because there would be no waiting for JIT compiler.

show 2 replies
array_key_firstyesterday at 5:00 PM

DB queries do get pre compiled and cached if you use prepared statements. This is why you should always use prepared statements if you can.

show 1 reply
levkkyesterday at 1:23 PM

See prepared statements.

SigmundAyesterday at 10:52 AM

Postgresql uses a process per connection model and it has no way to serialize a query plan to some form that can be shared between processes, so the time it takes to make the plan including JIT is very important.

Most other DB's cache query plans including jitted code so they are basically precompiled from one request to the next with the same statement.

show 2 replies