alt Hacker NewsOne thing I've never fully grokked is how this differs from an observable pattern where one can publish new values to inputs, propagate that through the computation, and push newly computed values to listeners.
I guess there's probably optimizations around change detection and stopping the propagation if there's no change (though observables can do that as well). The stabilize command also makes things interesting as a way to batch changes together before recomputing (but again, doable with observables too).
Is the delta primarily coming from introspection and automatically building the compute graph? Or is there something more fundamental that I'm missing?
Replies
Roughly, you subscribe and listen to an observable. Incrementals are more like a cache across some DAG of computation + state that lets you optimize by only recomputing what needs to be recomputed.
There's a really good talk from Ron Minsky here: https://www.janestreet.com/tech-talks/seven-implementations-...
https://www.scattered-thoughts.net/writing/an-opinionated-ma...
Can help map out the landscape.
That's how I was thinking about it too. At first glance it feels very close to reactive programming with dependency tracking. I'm curious whether the real advantage is the API ergonomics or if there are optimizations under the hood that wouldn't be practical with a typical observable implementation.
I mean, it's fundamentally just a graph, but it's a way of correctly and efficiently computing changes in massive, dynamic graphs. Let's imagine you have a diamond shaped subgraph that fans out to hundreds of intermediary nodes before collapsing down again via and paths with different "lengths". And what if some of those paths have e.g. min(A, B) where the max side is the only one changing?
A naive observer approach will 1) compute that potentially exponential blow-up very inefficiently and 2) probably have "concurrency" issues. This library will be close to optimal and correct, even if you start dynamically changing the graph structure.
But yes, you can achieve the same thing with observers and other kinds of approaches. Most of them just a lot harder to get right while avoiding performance cliffs.
It depends on how you define the observable pattern.
The fundamental components here are laziness and weak connections between graph nodes. Node values are getting materialized only when you observe them, and the system is flexible for live structural changes.
Usually, you don't need to materialize the entire graph when you need to observe just some nodes. Additionally, you can halt computations at any point in time leaving the graph in semi-actualized state, make extra changes to the inputs, and continue materialization of the nodes of interest. The algorithm will sort out all changes for you.
Essentially, incremental computations is just a term covering these features. You can organize the same system in terms of observers and subscribers.
Perhaps, classical Excel spreadsheets is the best illustration of the idea. Also, see my article on the topic: https://medium.com/@eliah.lakhin/salsa-algorithm-explained-c...