How to

How do I switch embedding models without a reindex script?

Switching an embedding model is a schema change: the old index is dropped and a new one is added, then every stored row is embedded with the new function. There is no separate reindex script. A destructive schema plan applies nothing unless you pass allow-destructive. Changing the embedder is not the same as recomputing a stored computed column.

Updated 2026-10-28 · Part of How do I ... in Python?

Before you start

About 15 min.

Python 3.11+
pip install -U 'pixeltable[serve]'
pxt init in the project root
sentence-transformers for all-MiniLM-L6-v2 and all-mpnet-base-v2

1. Start with the model you already index with

The embedder is part of the index. all-MiniLM-L6-v2 is the documented fast local default. Query uses the same function. Do not keep a second model id in a worker.

import pixeltable as pxt
from pixeltable.functions.huggingface import sentence_transformer

TableModel = pxt.model_base()
text_embed = sentence_transformer.using(model_id='all-MiniLM-L6-v2')

class Notes(TableModel, name='notes'):
    content: pxt.String
    __indexes__ = [
        pxt.EmbeddingIndex(content, string_embed=text_embed),
    ]

# pxt schema update app.py notes -f

2. Change the embedder on the class

Point string_embed at the new model. all-mpnet-base-v2 is stronger and slower. The schema diff will drop the old index and add the new one. Vectors from the old model cannot be reused.

text_embed = sentence_transformer.using(model_id='all-mpnet-base-v2')

class Notes(TableModel, name='notes'):
    content: pxt.String
    __indexes__ = [
        pxt.EmbeddingIndex(content, string_embed=text_embed),
    ]

3. Apply the destructive plan

Dropping an index destroys its data. A plan that drops an index applies nothing unless you pass allow-destructive. Force skips the confirmation prompt. After it finishes, every stored row is embedded with the new function.

# pxt schema diff app.py notes
# pxt schema update app.py notes --allow-destructive -f

4. Query with the new model only

similarity(string=) embeds the question with the index that now exists. Mixing an old query model with a new index ranks noise. pxt recompute is for a computed-column expression, not for this index swap.

@pxt.query
def search_notes(query: str, limit: int = 5):
    sim = Notes.content.similarity(string=query)
    return (
        Notes.order_by(sim, asc=False)
        .limit(limit)
        .select(Notes.content, sim)
    )

Reindex script versus a schema plan

Reindex script versus a schema plan
Job	A reindex script	Schema update
What changes	A worker that rewrites a second store	string_embed on EmbeddingIndex
Old vectors	You delete or overwrite them	The dropped index is gone
Apply	A job you schedule	pxt schema update --allow-destructive -f
Without the flag	The script still runs	The plan applies nothing
pxt recompute	Not this job	Computed columns, not this index swap

Where Pixeltable fits

In Pixeltable, changing the embedder replaces the embedding index: the schema diff drops the old index and adds the new one. Apply with pxt schema update --allow-destructive -f. The new index rebuilds embeddings for rows already stored. pxt recompute is for a computed-column expression, not for this index swap.

import pixeltable as pxt
from pixeltable.functions.huggingface import sentence_transformer

TableModel = pxt.model_base()
text_embed = sentence_transformer.using(model_id='all-mpnet-base-v2')

class Notes(TableModel, name='notes'):
    content: pxt.String
    __indexes__ = [
        pxt.EmbeddingIndex(content, string_embed=text_embed),
    ]

@pxt.query
def search_notes(query: str, limit: int = 5):
    sim = Notes.content.similarity(string=query)
    return Notes.order_by(sim, asc=False).limit(limit).select(Notes.content, sim)

# pxt schema update app.py notes --allow-destructive -f

Questions

Why is allow-destructive required?

Dropping an index destroys its data. A destructive plan applies nothing at all without that flag, so a missed confirmation cannot delete vectors by accident. -f skips the prompt once you have reviewed the diff.

Is this pxt recompute?

No. pxt recompute reruns a computed-column expression. Swapping an embedder drops and adds an embedding index. Use schema update --allow-destructive.

Can I keep both models at once?

Two indexes on two columns can coexist if you declare both. One index cannot hold two models. Query each column with the function that built it.

Next step

Copy the commands into an agent, or run them locally with pip install pixeltable.

Get started Quick Start