---
title: "Incremental recompute vs full reindex"
description: "Reduced run, 2000 rows, 200 changed. Incremental update 0.059 s wall vs full recompute 0.45 s wall. Stand-in UDF. Intel(R) Xeon(R) Processor, 4 cores, 15.6 GB, Linux 6.12.94+, Python 3.12.3, pixeltable 0.7.21."
url: "https://www.pixeltable.com/benchmarks/incremental-recompute-vs-full-reindex"
updated: "2026-10-28"
doc: "https://docs.pixeltable.com/datastore/computed-columns"
---

# Incremental recompute vs full reindex

On a reduced run of 2000 rows, changing 200 titles took 0.059 s wall. Recomputing the stand-in embedding column on all 2000 rows took 0.45 s wall. Rebuilding the embedding index took 0.39 s wall. The UDF is a 32-d numpy stand-in, not a hosted model. This is not a 1,000,000-row measurement.

Updated: 2026-10-28

## How to run

- `python benchmarks/run.py --write`
- `Or: PIXELTABLE_HOME=$(mktemp -d) python benchmarks/scenarios/incremental_recompute.py --rows 2000 --changed 200 --out /tmp/inc.json --scale reduced`

## Results

Insert of 2000 rows: 0.14 s wall, 0.14 s CPU. Initial embedding index: 0.41 s wall, 0.15 s CPU.

Update of 200 rows: 0.059 s wall, 0.022 s CPU. Full column recompute: 0.45 s wall, 0.16 s CPU. Drop index: 0.012 s wall, 0.0057 s CPU. Full reindex: 0.39 s wall, 0.14 s CPU.

| Measure | Value |
| --- | --- |
| Rows | 2000 |
| Changed | 200 |
| Insert | 0.14 s wall, 0.14 s CPU |
| Initial embedding index | 0.41 s wall, 0.15 s CPU |
| Incremental update | 0.059 s wall, 0.022 s CPU |
| Full recompute | 0.45 s wall, 0.16 s CPU |
| Full reindex | 0.39 s wall, 0.14 s CPU |

## Methods

- **Scale of this page:** reduced: 2000 rows, change 200
- **Full scale (not run here):** 1,000,000 rows, change 100,000. Documented in benchmarks/README.md.
- **UDF:** deterministic 32-d numpy stand-in, not a hosted model
- **Incremental step:** table.update on id < changed, which cascades to the computed column and the index
- **Full recompute:** table.recompute_columns on the embedding column
- **Full reindex:** drop_embedding_index then add_embedding_index
- **Machine:** Intel(R) Xeon(R) Processor, 4 cores, 15.6 GB, Linux 6.12.94+, Python 3.12.3, pixeltable 0.7.21
- **Recorded:** 2026-10-28, command python benchmarks/run.py --write

## Limitations

- The UDF is cheap on purpose, so wall times are fractions of a second. A hosted embedding model would dominate both sides.
- Each step ran once, with no warmup, so every time is one sample. scripts/release-bench repeats its runs.
- Intel(R) Xeon(R) Processor, 4 cores, 15.6 GB, Linux 6.12.94+, Python 3.12.3, pixeltable 0.7.21. Do not compare these times to the /engineering Mac numbers.
- 1,000,000 rows were not run.

## Where Pixeltable fits

If you already rebuild an external index on a schedule and that is the contract, Pixeltable is the wrong tool. Use it when a changed source row should recompute only its dependents.

```python
import pixeltable as pxt
from pixeltable.functions.huggingface import sentence_transformer

TableModel = pxt.model_base()
text_embed = sentence_transformer.using(model_id='intfloat/e5-large-v2')

class Passages(TableModel, name='passages'):
    text: pxt.String
    __indexes__ = [
        pxt.EmbeddingIndex(text, string_embed=text_embed),
    ]
```

## Questions

### Did you run a million rows? {#faq-1}

No. This page is a 2,000-row reduced run. The suite accepts --rows 1000000. That was not run in this VM.

### Is Pixeltable faster than rebuilding a vector database? {#faq-2}

This run compares Pixeltable incremental update with Pixeltable full recompute on the same table. It is not a bake-off against Qdrant or Pinecone.

## Related

- [Computed-column UDF calls: insert, read, recompute, update](https://www.pixeltable.com/benchmarks/computed-column-udf-calls)
- [Every Pixeltable release, measured](https://www.pixeltable.com/engineering)
- [What is a computed column?](https://www.pixeltable.com/learn/what-is-a-computed-column)

Docs: https://docs.pixeltable.com/datastore/computed-columns