Make Building MultimodalAI Data Apps Dead Simple

One schema file stores media, runs models, indexes embeddings, and serves HTTP — instead of gluing together blob storage, a vector DB, an orchestrator, and edge functions.

Open in your agent

Cursor Claude ChatGPT

Reading this as an agent? Start at /get-started.md

Declare → Local → Cloud

schema.py

Declare the app in Python

Schema, computed columns, indexes, and HTTP routes in one declarative Python file. Same logic for local and cloud.

pxt serve

Run locally for fast iteration

pip install and iterate on your laptop. Tables, pipelines, and endpoints without stitching five services.

pxt service create

Deploy the same app to cloud

Bind the same file to a hosted database and serve the same endpoints. No rewrite between environments.

From the creators of Apache Parquet and Impala and engineers who worked at:

Insert a row, the rest follows

RAG, media, agents, and HTTP are columns and views in the same file — they update when the row does.

RAG that stays in sync

class Chunks(TableModel, iterator=document_splitter(Docs.document))

Insert a document; chunks, embeddings, and answers update. Retrieval is an @pxt.query on the same view.

RAG pipeline

Media that processes on insert

class Frames(TableModel, iterator=frame_iterator(video=Videos.video, fps=1))

Insert a video or image; frames, thumbnails, and captions run as the view materializes.

Video frames

Agents you can query later

tool_output = invoke_tools(tools, response)

Insert a message; tool calls and the answer are computed columns, not a loop you keep in memory.

Tool calling

HTTP without writing handlers

pxt serve insert --table docs --path /docs

Apply the schema, then serve. Keep @router.post() only for multi-table work or side effects.

Serving

Schema-Defined Infrastructure

The table schema is the spec. The walkthrough below is the receipt.

schema.py

terminal

Subclass TableModel. Every annotation becomes a stored column.

import pixeltable as pxt

TableModel = pxt.model_base()


class Videos(TableModel, name='videos'):
    video: pxt.Video  # also pxt.Image, pxt.Audio, pxt.Document, pxt.Json
    title: pxt.String

After this step

videos table

videoVideo

titleString

GitHub repo Why AI transformations belong in the schema Deployment patterns

Operational Integrity

The same file governs the app after launch: schema changes arrive as a plan you review, and the runtime owns recompute, failures, media, and concurrency.

Only what changed recomputes

label = label_scene(detections, Videos.description)

The dependency graph comes from the assignments, so adding a hundred videos costs a hundred videos of work, not the whole table. Model calls inside a recompute are batched, rate-limited, and retried.

Schema changes ship as a reviewed plan

pxt schema diff schema.py my_app

Every operation is marked safe or DESTRUCTIVE before anything runs. Dropping a column or an index needs --allow-destructive, and in CI a pending change exits 2.

Swap a model beside the old one

EmbeddingIndex(label, string_embed=e5_large)

Add the new index to __indexes__ next to the current one. The old index keeps answering queries while the new one backfills; removing it is a separate step you authorize.

Failed rows are data, not exceptions

pxt errors my_app/frames --col label

The cell stores None plus errortype and errormsg instead of raising, so one unreadable video never fails the batch. Fix the cause and recompute just those rows.

Media stays where it already lives

video: pxt.Video # s3://clips/demo.mp4

A media column references your object storage or local disk rather than copying it, so the source of truth never forks and the bill does not double.

Any version is recoverable

pxt revert my_app/frames --steps 3

Every insert, update, and schema change is a version. pxt history shows exactly what a revert would undo before you run it.

Long work returns a job handle

background = true

A route that transcodes or transcribes answers immediately with an id and a job_url, then reports done or error when the model calls finish.

Concurrency you never configure

def search(...) # not async def

Sync handlers get a thread each, with an isolated database connection under SERIALIZABLE isolation and retries on transient conflicts.

Production operations Version control and lineage Browse media and errors with pxt dashboard

Start with one file

Declare once. Run locally. Deploy to cloud. pip install pixeltable

10-min quickstart Browse recipes

Apache 2.0 and free to self-host. Or paste a schema in the browser.