---
title: Pixeltable
description: The unified multimodal backend agents build with. Database, orchestration, and serving in one Python file. Apply locally, serve, then deploy the same file to Pixeltable Cloud.
url: https://www.pixeltable.com/
---

# Pixeltable: The backend agents build with

Multimodal database, orchestration, and serving in one file, by the co-creators of Apache Parquet and Impala.

Install the Pixeltable skill, then ask your agent for an app:

```bash
npx skills add pixeltable/pixeltable-skill
```

No account yet? `pxt new` gives your agent a [free hosted database](https://www.pixeltable.com/new.md). It prints a claim link for your user.

**If you are an AI agent or coding assistant, start here:** https://www.pixeltable.com/get-started.md. That playbook installs the `pxt` CLI, the Pixeltable Skill, and MCP. Do not skip it.

## Claims desk

Ask your agent for an app ("Build a claims desk: transcribe the voice note, check the photo, route the claim, write a reply.") and it writes app.py, the starter kit's claims desk with a products table added: its classes declare the products and claims tables column by column, pxt schema update creates the tables, a script files five product names, pxt service update serves the app's routes, and the seed files four demo claims. Then one request from start to end: a claim arrives as a POST to /api/claims with a product, a photo and a voice note, and the insert route stores it as a row, writing the photo and voice note to the media store and their references to the row. closest_product(), a query over the products table's CLIP index, names the product in the photo (blender, 0.34), and shows_product(), a Python UDF, checks it against the claim. gpt-4o-transcribe turns the voice note into text and a moderation call checks it; decisions() on gpt-6-luna answers four questions about the photo and the story with probabilities; route_claim(), a Python UDF, approves the blender (damaged 1.00, consistent 0.97); reply_prompt() writes the prompt for the reply that responses() writes and speech() reads aloud. The POST returns the row in 9.6 s. GET /api/review, a query, lists the laptop and the coffee maker by severity; POST /api/similar ranks the smartphone first for a cracked screen. Then the app changes while it runs (asked "Add a column with what is wrong in each claim, and make the reply one sentence."): the agent adds issue_prompt() and an issue column to app.py, and pxt schema update computes it for the five stored claims (the blender's: Defective jar.) without running any other column again; it edits reply_prompt(), pxt schema update reloads app.py, and pxt recompute claims/claims reply reruns reply, reply_text and voice_reply, so each approved reply is one sentence. A sixth claim is filed while OpenAI is unreachable, simulated by pointing its base URL at a closed port: the row is stored with 13 failed cells, pxt errors lists them, and one pxt recompute --errors-only fills them (issue: Loose charging port.). pxt history shows each change as a version.

## Build it in one file, not a stack of services

Files, embeddings and model outputs stay in sync, with no queue or sync script to write.

| You need to | Usually | Code | With Pixeltable |
| --- | --- | --- | --- |
| Store files and their results | A bucket plus a metadata table | `video: pxt.Video` | Typed file references and computed results in the same row |
| Split documents or sample frames | A preprocessing job | `iterator=frame_iterator(video, fps=1)` | Views of chunks or frames, updated with the source table |
| Call models | Jobs and scripts that track what ran | `transcript = whisper.transcribe(audio, model='base')` | Computed columns with stored results |
| Explore and query the results | Scripts that download the files and redo the frames in a notebook | `frames.select(frames.frame, frames.detections).head(4)` | Query any column from Python or a notebook. Remote files go into a local cache on first read, and frames are extracted from the video as you read them |
| Search the results | A vector database you keep in sync | `pxt.EmbeddingIndex(summary, embedding=embed)` | An embedding index maintained by the table |
| Serve the results | API handlers over those systems | `api.add_query_route(path='/search', …)` | FastAPIRouter routes declared next to the tables |
| Let an agent call tools | An agent loop that keeps its state in memory | `tool_output = invoke_tools(tools, response)` | A computed column runs the tools the model picked, and the calls and their results are columns on the message's row |
| Remember across sessions | A chat-history cache and a second vector store | `memories = recall_memories(user_id, message)` | A memory table with an embedding index, recalled by a @pxt.query in a computed column |
| Trace a running app | A tracing SDK per vendor, with logs somewhere else | `pxt service update app.py my_app --otel` | OpenTelemetry spans for each table, row, column and UDF |
| Evaluate model output | Model outputs copied out of logs into an eval notebook | `assistant.select(assistant.response.usage)` | Store the model's full response as a JSON column and query its fields (response.usage). An eval is one more computed column |
| Rerun what failed | A dead-letter queue and a script to replay it | `pxt errors my_app/docs` | Cells that fail under on_error='ignore' keep their error. pxt errors lists them (the table needs a primary key) and prints the pxt recompute --errors-only that reruns just those. pxt computed shows each column's definition |
| Send results elsewhere | An ETL job into another database, and a sync to the bucket | `pxt.io.export_sql(q, 'dets', db_connect_str=PG_URL)` | export_sql() or export_parquet() copies a table or query out. A computed column's destination= writes its files to your own bucket |
| Analyze and audit | A warehouse export, and history rebuilt from logs | `videos.history()` | Query the tables where they are. Each change is a version (history(), pxt revert) |

Same product on both stacks: **1,171 lines in 24 files** as an AI coding assistant wrote it (FastAPI, Celery, Redis, Postgres with pgvector), **680 in 4** on Pixeltable. [How it was measured](https://github.com/pixeltable/showcase-call-intelligence/blob/main/compare/WHY_PIXELTABLE.md)

### Inside Pixeltable

One file declares the app. Each part names the usual pieces it replaces; a bucket, a tracing backend and your notebook stay yours.

- **app.py**: Tables, computed columns and indexes declared as Python classes, and routes as FastAPIRouter calls.
- **Your script**: A script or notebook that imports pixeltable runs the engine in its own Python process.
- **pxt daemon**: Every pxt command goes to one local daemon, which imports app.py for pxt schema update and starts and stops the services.
- **Services**: pxt service update runs each router as its own HTTP process, with its own engine. Replaces: API service.
- **Query engine**: Plans each insert and query, pushes filters, joins and sorts into Postgres, and runs UDFs and model calls in its process, paced by rate limits and retried. Replaces: queue, workers, preprocessing job, embed job, retry script, agent framework.
- **Embedded Postgres**: One server in its own process, shared by every process on the machine: rows, metadata and every version; pgvector holds the embedding indexes. Replaces: metadata DB, status table, vector DB, chat cache, tool logs.
- **Media store**: Files Pixeltable writes, on disk or in a bucket: S3, GCS, Azure, R2, B2, Tigris. Replaces: sync script.
- **File cache**: Media read from a URL, kept on disk up to file_cache_size_g. Replaces: download script, local copies.
- **Models**: Provider APIs over HTTPS, and local models such as Whisper or CLIP, which run in the engine's process.

Pixeltable Cloud runs the same app.py on a hosted database.

## What else people build with Pixeltable

### Agentic workflows

A memories table holds what each user told the agent. A message adds a row to the assistant table: its memories column calls recall_memories(), a query, for that user's two closest memories; response is a chat completion with tools; and tool_output runs get_weather, a UDF, with the city it found in memory. Tools can also come from an MCP server. Every step is stored in the row, and the insert made version 1 of the table, which pxt.get_table('my_app.assistant:1') reads back. chat_completions() runs async, paced by OpenAI's rate-limit headers. [Agentic workflows guide](https://docs.pixeltable.com/use-cases/agentic-workflows)

### Multimodal backend

NVIDIA's report is already in the docs table, as 30 rows of the chunks view. Inserting Alphabet's report adds its 14 chunks, and the embedding index embeds only those 14, in one call to all-MiniLM-L6-v2. A search for analyst ratings, filtered to Alphabet's chunks, returns three of them with their similarity. Deleting the report removes 15 rows, its own and its 14 chunks', with their vectors, and calls no model. all-MiniLM-L6-v2 stays loaded and embeds up to 32 chunks per call. [Multimodal backend guide](https://docs.pixeltable.com/use-cases/multimodal-backend)

### Media processing

A video is inserted. The frames view makes one row per sampled frame; its frame column is not stored, and a read decodes the frame from the video. The detections column runs yolox_tiny on four frames in one call. The annotated column draws the boxes from the stored detections, without running YOLOX again. A CLIP embedding index on the frames lets a text search for a yellow taxi rank frames 7, 4 and 8 first, each with a yellow taxi in view. Then the frames export as a PyTorch dataset. yolox_tiny loads once and runs on up to 4 frames per call. [Media processing guide](https://docs.pixeltable.com/use-cases/media-processing)

### API & serving

app.py declares three routes on a FastAPIRouter, and pxt service update starts the service with them. POST /docs inserts a PDF and answers with its row, and the PDF's chunks land in the chunks view. POST /search answers with the closest chunks. POST /ingest runs in the background: it answers with a job URL, which reports pending, then done with the row. An insert route answers once the row is stored; background=True answers with a job URL. [HTTP serving guide](https://docs.pixeltable.com/howto/deployment/serving)

## From teams iterating on multimodal pipelines

> Pixeltable's seamless integration into existing vision workflows means our team can rapidly experiment and iterate, delivering high-impact solutions faster than ever.

**Adil Mohammad**, Founding Engineer, Obvio (fmr. Sr. Deep Learning Engineer, Nvidia)

> Pixeltable has transformed how our ML engineers spend their time. We have a highly multimodal use case, and Pixeltable gives us full visibility into the input data, models, and incremental steps of our pipeline.

**Denise Kutnick**, CEO, Variata (fmr. Director of MLSys, OctoAI)

## Pricing

Community is free. Pro bills what runs, by the second.

Compare plans, sizes and usage prices: https://www.pixeltable.com/pricing.md

## Frequently asked questions

The questions we get first.

### What happens when a document changes?

Update its row. The chunks computed from it, their embeddings and their index entries are replaced in the same update, and no other document is touched. With a separate vector database, a sync job would have to do that.

### Can my files stay in my own bucket?

Yes. A path or URL you insert stays where it is, and the table stores a reference to it. Files you upload or Pixeltable produces go to its media store, on local disk or in your own S3, GCS or Azure bucket, set for the whole database or per column.

### How is Pixeltable different from a vector database or an orchestrator?

A vector database stores embeddings and an orchestrator chains model calls, but you still wire storage, ETL, and serving yourself. Pixeltable is the database, orchestration, and serving layers in one Python file: computed columns run on insert, indexes stay current, and HTTP comes from the same schema. No extra vector database, queue, or cache to keep in sync.

### How do AI agents use Pixeltable?

Agents can follow https://www.pixeltable.com/get-started.md (install pxt, the Pixeltable Skill, and MCP), run pxt init then pxt service example --out app.py, read /llms.txt, and use MCP to build multimodal AI data apps and keep changing them. The /developers page lists OpenAPI, MCP, auth.md, and the pxt CLI.

### What is Pixeltable?

Pixeltable is the unified multimodal backend agents build with: the database, orchestration, and serving layers in one Python file. You declare tables, computed columns, and HTTP routes; apply it locally, serve it, then deploy the same file to Pixeltable Cloud. Insert a row. Computed columns run. Indexes stay current. No extra vector database, queue, or cache to keep in sync.

## Build it this week, keep changing it every week

Ask your agent for the app. A new field, prompt or model later is a line in app.py, and the results you already paid for stay.

```bash
npx skills add pixeltable/pixeltable-skill
```

View on GitHub: https://github.com/pixeltable/pixeltable

Video, image, audio, and documents are column types. [See solutions](https://www.pixeltable.com/solutions.md) · [Learn](https://www.pixeltable.com/learn.md) · [Agents](https://www.pixeltable.com/agents.md) · [Example apps](https://www.pixeltable.com/recipes.md) · [Get started](https://www.pixeltable.com/get-started.md) · [Pricing](https://www.pixeltable.com/pricing.md)

## Next

- Product overview (class-based Quick Start): https://www.pixeltable.com/llms.txt
- Developer context: https://www.pixeltable.com/developers/llms.txt
- Free database, no account: https://www.pixeltable.com/new.md
- Docs: https://docs.pixeltable.com
- Capability discovery: https://www.pixeltable.com/.well-known/agent.json