PixeltablePixeltable Logo

Pixeltable and LanceDB

LanceDB is a strong multimodal lake and ANN index. Pixeltable is a live application schema: computed columns, provider UDFs, and HTTP. Often and, not or. Export embeddings with pxt.io.export_lancedb when the index is the product.

pip install 'pixeltable[serve]'
See how it works

Pipeline vs the index

SidePixeltableLanceDB
At a glance
  • Computed columns run ffmpeg, Whisper, CLIP, and chat on insert
  • EmbeddingIndex stays on the source row; no upsert glue
  • FastAPIRouter in the same file
  • Export a query to LanceDB when you want that serving engine
  • Lance format: columnar, versioned, multimodal blobs + vectors
  • ANN tuned for large embedding tables
  • Embedded or cloud; SQL-style filters on stored metadata
  • You still write chunking, model calls, and HTTP around it

What each actually owns

LanceDB wins embedding query and Lance layout. Pixeltable wins incremental pipelines and serving. “Single-modal vector search” is not a fair description of LanceDB in 2026.

FeaturePixeltableLanceDB
What it is
Application schema: store, transform, index, serve
Multimodal lake and vector engine on Lance
Vector query
EmbeddingIndex + similarity(string=query)
ANN the product is built around; usually faster at large scale
Media pipelines
Iterators and computed columns; insert runs the work
Store multimodal data; processing is UDFs or jobs you write
Embedding sync
Index is a schema object; insert and delete keep it current
Embedding functions and backfill; you own the re-embed job
HTTP serving
FastAPIRouter declared next to the tables
Not a web framework; you put FastAPI or a notebook in front
Lake / format
Catalog plus media refs; not a Lance lakehouse
Lance fragments, versioning, and ecosystem around the format
Embedded deploy
Local dir or Pixeltable Cloud
Process-local LanceDB with a small footprint

Document index

Pixeltable: chunking is a view, the index is on the class. LanceDB: embedding registry and search. Apply Pixeltable with pxt schema update app.py search.

Pixeltable

import pixeltable as pxt
from pixeltable.functions.document import document_splitter
from pixeltable.functions.huggingface import sentence_transformer


TableModel = pxt.model_base()
embed = sentence_transformer.using(
    model_id='sentence-transformers/all-MiniLM-L6-v2'
)


class Docs(TableModel, name='docs'):
    document: pxt.Document
    title: pxt.String


class Chunks(
    TableModel,
    name='chunks',
    base=Docs,
    iterator=document_splitter(Docs.document, separators='sentence', limit=512),
):
    __indexes__ = [pxt.EmbeddingIndex(text, embedding=embed)]


# pxt schema update app.py search
docs = pxt.get_table('search.docs')
docs.insert([{'document': 'handbook.pdf', 'title': 'Handbook'}])
chunks = pxt.get_table('search.chunks')
sim = chunks.text.similarity(string='refund policy')
chunks.order_by(sim, asc=False).limit(5).select(chunks.text, chunks.title)

LanceDB

import lancedb
from lancedb.pydantic import LanceModel, Vector
from lancedb.embeddings import get_registry


db = lancedb.connect('./lancedb')
func = get_registry().get('sentence-transformers').create(
    name='all-MiniLM-L6-v2'
)


class Document(LanceModel):
    title: str
    content: str = func.SourceField()
    vector: Vector(func.ndims()) = func.VectorField()


table = db.create_table('documents', schema=Document)
table.add([{'title': 'Handbook', 'content': '...'}])
hits = table.search('refund policy').limit(5).to_pandas()
# Chunking, ffmpeg, and HTTP are still outside this table.

Hand the index to LanceDB

When LanceDB should serve the vectors, export a Pixeltable query. Requires pip install lancedb pylance.

Pixeltable

from pathlib import Path
import pixeltable as pxt


chunks = pxt.get_table('search.chunks')
pxt.io.export_lancedb(
    chunks.select(chunks.text, chunks.title),
    Path('lancedb'),
    'chunks',
    if_exists='overwrite',
)

LanceDB

import lancedb


db = lancedb.connect('./lancedb')
table = db.open_table('chunks')
hits = table.search('refund policy').limit(5).to_pandas()
# Pixeltable remains the system of record for source files and compute.

When each belongs

Use Pixeltable when

  • The pipeline is the schema

    Documents, frames, transcripts, provider models, and an HTTP route in one file. Incremental column add without a re-embed script you maintain.

  • You still want LanceDB for serving

    Process in Pixeltable. export_lancedb when the contract is a Lance table. Integration notes: https://pixeltable.com/blog/pixeltable-lancedb-integration

Use LanceDB when

  • The index is the product

    You already have embeddings. You need ANN, Lance layout, or an embedded process-local store. Pixeltable is extra.

  • Lake-scale vector analytics

    Lance fragments, versioning, and query engines built around that format. Do not pick Pixeltable as a Lance replacement.

Making the right choice

  • And, not or

    • Do not migrate off LanceDB just to “complete the stack.” Keep it if ANN or Lance is the contract.
    • Use Pixeltable when transforms and HTTP belong next to the rows.
    • pxt.io.export_lancedb is the join, not a hidden rewrite of Lance.

Frequently asked questions

More comparisons

Pipeline in the schema. Export when the index is the product.

Declare tables and indexes in app.py. Apply with pxt schema update. Hand vectors to LanceDB when you want that engine.

pip install 'pixeltable[serve]'
See how it worksGet expert guidance