Orchestration
What is incremental computation?
Incremental computation recomputes only the rows and downstream columns a change actually affects, instead of rerunning the whole pipeline.
Updated
How it works
- Dependencies between columns are known.
- An edit marks the descendants of that row.
- Everything else stays cached.
What it is not
It is not a nightly full refresh, and it is not a cache you remember to invalidate.
incremental computation: this, and the thing it is confused with
| This | Not this | |
|---|---|---|
| Work | Changed rows | The whole table |
| Invalidation | Recorded dependencies | Hope |
| Cost | Model calls for the delta | Model calls for the corpus |
Where Pixeltable fits
Pixeltable tracks those dependencies on computed columns. A new file recomputes that row’s descendants.
Questions
How does incremental computation work?
Dependencies between columns are known. An edit marks the descendants of that row. Everything else stays cached.
What is incremental computation often confused with?
It is not a nightly full refresh, and it is not a cache you remember to invalidate.
Next step
Install the Pixeltable skill, or sign up for Cloud and run the same file on a hosted database.