Summary: Pixeltable is the unified multimodal backend agents build with. Database, orchestration, and serving live in one Python file. Images, video, audio, and documents are column types. A transform is a computed column. An index is a declaration, and so is an HTTP route. Insert a row and everything below it runs. Object storage, a vector database, an orchestrator, and the endpoint code that copies between them collapse into app.py. pip install pixeltable.
Four Products, One File#
The tax was never writing a column. It was the joints before you could write one.
Bytes in an object store. IDs in a warehouse. A DAG that re-runs a world because it cannot see which cells changed. Embeddings in a vector database that nobody invalidates when the source clip dies. A FastAPI file you babysit so the demo has a URL. Each piece is fine. The set is a second job. That is the Frankenstein stack. We spent 2026 deleting it: Never Fuck With Your Backend Anymore.
What remains is a habit. You declare tables, computed columns, indexes, and routes in one file. You apply it. You insert a row. The work runs. You serve the same file. That is the backend a coding agent can keep — not a shopping list of services it has to invent every session.
The File Agents Write#
pxt service example --out app.py writes this. Delete what you do not need. Annotations are stored columns. Assignments are computed columns: they run on insert and on update. A @pxt.udf in the same file is a function those columns can call. Production tables often use uuid7() as a generated primary key; the example uses an integer you insert so the first curl is obvious.
title: pxt.String is a value you provide. title_upper = ... is computed. POST /docs inserts and returns the computed columns. POST /titles computes without storing a row. Insert a row. The assignments run. That is the whole orchestration story.
Three Verbs That Are Not Interchangeable#
Agents mix these. The file does not.
pxt schema update creates the catalog my_app and its tables. It does not start HTTP. pxt service update starts HTTP. It does not create tables. On Cloud, pxt db update creates or updates the hosted database — image, workers, disk. It does not insert rows. pxt service run is local only. It cannot target pxt://.
The port is assigned. Read it back rather than hardcoding it:
Media Is a Column#
The same file holds pxt.Image, pxt.Video, pxt.Audio, or pxt.Document. A computed column over one of them is another assignment: frames, transcripts, captions, chunks. That is a multimodal data table, not a FILE path the engine does not understand. Flattening a Gong call into a SQL tag is not beyond SQL. It is also not a system of record for the audio. The post-AI stack still draws a warehouse. Media needs a typed table: The Post-AI Data Stack Needs a Multimodal Table.
An embedding index is a declaration on the class (__indexes__). A query is @pxt.query. HTTP for that query is a route on the same router. You do not stand up a second product to keep the index honest. Insert is the pipeline.
Same File on Cloud#
Create an API key in the Cloud dashboard, set PIXELTABLE_API_KEY, name the database in pixeltable.toml, then target it by URI. A @pxt.udf in that same app.py is in the image pxt db update builds. Promotion is the file you already ran, not a rewrite into a warehouse dialect. That is why the local-cloud loop is the lifecycle, not a staging cluster.
Order matters. Schema first against a hosted database that does not exist yet is a 404 on the UDF. Database, then schema, then service.
What Agents Copy vs Invent#
Hand the agent get-started.md. That playbook installs the package, the Pixeltable Skill, and MCP. The capability file is skill.md. How you design for that customer — install via npx, evaluate via a machine-readable overview, buy by writing app.py — is Designing Software for LLMs as Customers.
The skill writes a TableModel in app.py. If the agent writes create_table in application code, names the file schema.py, or writes the removed command pxt serve, the installed skill is stale. Reinstall it. Notebooks and tests still use pxt.create_table(). An app puts tables in app.py and creates them with pxt schema update. Do not mix both styles for the same tables.
pxt service example is the generic file. uvx pixeltable-new copies one app from the starter kit — the acquisition artifact we learned to ship so agents do not default to LangChain plus a vector DB: What We Learned Shipping Application Templates for AI Agents. The default copy is a chat app; pass agent as the last argument to pxt schema update. --video copies video search; pass videointel.
Inserting into the knowledge table needs no API key. The /ask route needs ANTHROPIC_API_KEY.
To mount the routes on an existing FastAPI app, app.include_router(...). To skip endpoints, run pxt schema update, insert from Python, then export_sql if you need a warehouse bridge. Self-hosting is the same schema without the hosted target.
What This Is Not#
It is not no-code. If you do not want a schema, Pixeltable is the wrong product. The point is that the schema is the backend. The persona version of shrinking that surface — Lovable for people who do not write backends, Pixeltable for people who do — is Making Multimodal More Lovable. It is also not a signup-wall remover that provisions Postgres, Redis, auth, and hosting — that job is Cohesivity; the query page is Backend for AI agents.
It is not “don’t use FastAPI.” FastAPIRouter is FastAPI derived from the catalog. You do not write the handlers. You still get HTTP.
It is not a warehouse replacement. Keep Snowflake or Databricks for Card TPV and NRR. Do not start the video graph in a generic FILE column.
And it is not “use our agent.” Use yours. Pixeltable is the catalog your agent writes to. MCP and HTTP are adapters on that catalog, not a second semantic language.
People Also Ask#
Is TableModel required? For new apps, yes — it is the public schema. The notebook form (create_table, add_computed_column) still exists for one-off scripts. Do not mix both styles for the same tables.
Do I still need Pinecone, Airflow, and an object store? Not as a default architecture. Media, computed transforms, embedding indexes, and HTTP live in one system. You can still point media at your bucket. You should not need a sync job to keep three products from drifting.
How do I go to Cloud? Same app.py. Set PIXELTABLE_API_KEY, declare pxt://org:db, then pxt db update → pxt schema update → pxt service update. Hosted HTTP is on the website dashboard. pxt service run is local only.
Can I mount this on an existing FastAPI app? Yes. app.include_router(...) the routers this file declares. To skip HTTP entirely, apply the schema and insert from Python.
The agent wrote create_table / pxt serve / schema.py. The skill is stale. Reinstall npx skills add pixeltable/pixeltable-skill and hand it get-started.md.
Keep the File, Delete the Stack#
The backend is one schema you keep, not a stack you assemble. Agents can keep a file. They cannot keep a Frankenstein.
Declare the class. Apply it. Insert a row. Serve the same file.
- Pixeltable on GitHub:
pip install pixeltable - Get started: CLI, Skill, and the one-file loop
- skill.md: the capability file the skill follows
- Never Fuck With Your Backend Anymore: the 2026 recap — what disappeared
- Making Multimodal More Lovable: the sibling essay on persona and surface
- Why the Local-Cloud Loop Matters: the sibling essay on promotion
- The Post-AI Data Stack Needs a Multimodal Table: warehouse stays SQL; media needs a typed table
- Pixeltable documentation
