Summary: Pixeltable is the unified multimodal backend agents build with. Database, orchestration, and serving live in one Python file. Images, video, audio, and documents are column types. A transform is a computed column. An index is a declaration, and so is an HTTP route. Insert a row and everything below it runs. Object storage, a vector database, an orchestrator, and the endpoint code that copies between them collapse into app.py. pip install pixeltable.
Four Products, One File#
The tax was never writing a column. It was the joints before you could write one.
Bytes in an object store. IDs in a warehouse. A DAG that re-runs a world because it cannot see which cells changed. Embeddings in a vector database that nobody invalidates when the source clip dies. A FastAPI file you babysit so the demo has a URL. Each piece is fine. The set is a second job. That is the Frankenstein stack. We spent 2026 deleting it: Never Fuck With Your Backend Anymore.
What remains is a habit. You declare tables, computed columns, indexes, and routes in one file. You apply it. You insert a row. The work runs. You serve the same file. That is the backend a coding agent can keep — not a shopping list of services it has to invent every session.
The File Agents Write#
pxt service example --out app.py writes this. Delete what you do not need. Annotations are stored columns. Assignments are computed columns: they run on insert and on update. A @pxt.udf in the same file is a function those columns can call. Production tables often use uuid7() as a generated primary key; the example uses an integer you insert so the first curl is obvious.
title: pxt.String is a value you provide. title_upper = ... is computed. POST /docs inserts and returns the computed columns. POST /titles computes without storing a row. Insert a row. The assignments run. That is the whole orchestration story.
Three Verbs That Are Not Interchangeable#
Agents mix these. The file does not.
pxt schema update creates the catalog my_app and its tables. It does not start HTTP. pxt service update starts HTTP. It does not create tables. On Cloud, pxt db update creates or updates the hosted database — image, workers, disk. It does not insert rows. pxt service run is local only. It cannot target pxt://.
The port is assigned. Read it back rather than hardcoding it:
Media Is a Column#
The same file holds pxt.Image, pxt.Video, pxt.Audio, or pxt.Document. A computed column over one of them is another assignment: frames, transcripts, captions, chunks. That is a multimodal data table, not a FILE path the engine does not understand. Flattening a Gong call into a SQL tag is not beyond SQL. It is also not a system of record for the audio. The post-AI stack still draws a warehouse. Media needs a typed table: The Post-AI Data Stack Needs a Multimodal Table.
An embedding index is a declaration on the class (__indexes__). A query is @pxt.query. HTTP for that query is a route on the same router. You do not stand up a second product to keep the index honest. Insert is the pipeline.
Same File on Cloud#
Create an API key in the Cloud dashboard, set PIXELTABLE_API_KEY, name the database in pixeltable.toml, then target it by URI. A @pxt.udf in that same app.py is in the image pxt db update builds. Promotion is the file you already ran, not a rewrite into a warehouse dialect. That is why the local-cloud loop is the lifecycle, not a staging cluster.
Order matters. Schema first against a hosted database that does not exist yet is a 404 on the UDF. Database, then schema, then service.
What Agents Copy vs Invent#
Hand the agent get-started.md. That playbook installs the package, the Pixeltable Skill, and MCP. The capability file is skill.md. How you design for that customer — install via npx, evaluate via a machine-readable overview, buy by writing app.py — is Designing Software for LLMs as Customers.
The skill writes a TableModel in app.py. If the agent writes create_table in application code, names the file schema.py, or writes the removed command pxt serve, the installed skill is stale. Reinstall it. Notebooks and tests still use pxt.create_table(). An app puts tables in app.py and creates them with pxt schema update. Do not mix both styles for the same tables.
pxt service example is the generic file. uvx pixeltable-new copies one app from the starter kit — the acquisition artifact we learned to ship so agents do not default to LangChain plus a vector DB: What We Learned Shipping Application Templates for AI Agents. The default copy is a chat app; pass agent as the last argument to pxt schema update. --video copies video search; pass videointel.
Inserting into the knowledge table needs no API key. The /ask route needs ANTHROPIC_API_KEY.
To mount the routes on an existing FastAPI app, app.include_router(...). To skip endpoints, run pxt schema update, insert from Python, then export_sql if you need a warehouse bridge. Self-hosting is the same schema without the hosted target.
What This Is Not#
It is not no-code. If you do not want a schema, Pixeltable is the wrong product. The point is that the schema is the backend. The persona version of shrinking that surface — Lovable for people who do not write backends, Pixeltable for people who do — is Making Multimodal More Lovable.
It is not “don’t use FastAPI.” FastAPIRouter is FastAPI derived from the catalog. You do not write the handlers. You still get HTTP.
It is not a warehouse replacement. Keep Snowflake or Databricks for Card TPV and NRR. Do not start the video graph in a generic FILE column.
And it is not “use our agent.” Use yours. Pixeltable is the catalog your agent writes to. MCP and HTTP are adapters on that catalog, not a second semantic language.
People Also Ask#
Is TableModel required? For new apps, yes — it is the public schema. The notebook form (create_table, add_computed_column) still exists for one-off scripts. Do not mix both styles for the same tables.
Do I still need Pinecone, Airflow, and an object store? Not as a default architecture. Media, computed transforms, embedding indexes, and HTTP live in one system. You can still point media at your bucket. You should not need a sync job to keep three products from drifting.
How do I go to Cloud? Same app.py. Set PIXELTABLE_API_KEY, declare pxt://org:db, then pxt db update → pxt schema update → pxt service update. Hosted HTTP is on the website dashboard. pxt service run is local only.
Can I mount this on an existing FastAPI app? Yes. app.include_router(...) the routers this file declares. To skip HTTP entirely, apply the schema and insert from Python.
The agent wrote create_table / pxt serve / schema.py. The skill is stale. Reinstall npx skills add pixeltable/pixeltable-skill and hand it get-started.md.
Keep the File, Delete the Stack#
The backend is one schema you keep, not a stack you assemble. Agents can keep a file. They cannot keep a Frankenstein.
Declare the class. Apply it. Insert a row. Serve the same file.
- Pixeltable on GitHub:
pip install pixeltable - Get started: CLI, Skill, and the one-file loop
- skill.md: the capability file the skill follows
- Never Fuck With Your Backend Anymore: the 2026 recap — what disappeared
- Making Multimodal More Lovable: the sibling essay on persona and surface
- Why the Local-Cloud Loop Matters: the sibling essay on promotion
- The Post-AI Data Stack Needs a Multimodal Table: warehouse stays SQL; media needs a typed table
- Pixeltable documentation
