Pydantic AI Harness can give an agent read-only tools over Pixeltable tables and store the agent's notes in a Pixeltable table. I ran both on Python 3.12 in a Linux container, with pixeltable 0.7.23 and pydantic-ai-harness 0.66.0. A similarity search for the meal limit returned the 60 EUR rule first, with score 0.6223213836535595. A second write of the same memory operation came back replayed=True, and the stored sentence was still "I prefer visual examples."
If you don't want a schema, Pixeltable is the wrong tool. If the notes never need to sit next to tables, video, or an embedding index, use the file store or the SQLite store Harness already ships. The same app.py is what pxt schema update applies locally. Hosted setup is the cloud docs.
Install pydantic-ai-harness 0.66.0#
I sent the integration in pull request 9369. Aditya Vardhan reviewed it and merged it on 2026-11-03 at 13:29 UTC. pydantic-ai-harness 0.66.0 includes the Pixeltable module. It was uploaded at 2026-11-04 10:28 UTC. 0.65.0 does not. I downloaded that wheel, uploaded at 2026-11-03 01:53 UTC, and it contains no Pixeltable module. Tag v2.65.0 is that release, eight commits behind the merge.
The [openai] extra is there because the agent constructor imports the OpenAI provider. sentence-transformers is there because the embedding index in this post runs on the machine. The shipped video example uses OpenAI embeddings instead. I did not run that file.
I ran that command on 2026-11-04 in a python:3.12-slim Docker container with ffmpeg installed from apt. The environment had Python 3.12.16, pixeltable 0.7.23, pydantic-ai-harness 0.66.0, pydantic-ai-slim 2.66.0, sentence-transformers 6.2.0, and openai 3.45.0. I ran every snippet below in that container, in order, except the agent in the last section, which I only constructed. The extra allows Python 3.11 through 3.14. Its marker is python_version < '3.15'. pixeltable-pgserver 0.7.1 publishes 3.15 wheels, so the marker is what excludes 3.15.
Read Pixeltable in the Harness docs. The overview lists Pixeltable.
Declare the handbook and the video#
The catalog tools read tables that already exist. Save this as app.py. topic and text are columns you insert. The embedding index calls sentence-transformers locally, so a search does not call an embedding API. Clips holds a video. Frames is a view that keeps one frame per second. The shape of a class-based schema is the same one in the quickstart.
pxt init marks the project root and writes pixeltable.toml. Apply the file from that folder. -f skips the confirmation.
Insert a policy and a one-second clip#
The clip I inserted is one second of a solid frame plus a sine tone, written with ffmpeg. This command writes a clip of that shape. I checked it with ffprobe, which reported 25 video frames and 1.0 s of video and audio, and did not insert that file.
Run this from the project folder. pxt schema update loaded app.py in its own process, so this process imports the classes and binds them before the insert.
Hugging Face warned that the request was unauthenticated. The inserts still finished. The model was already on disk for this run. A first run downloads it if it is not cached.
The first line is the two handbook rows. The second line counts the clip and the frame the view wrote. Handbook.count() was 2, Clips.count() was 1, and Frames.count() was 1.
pxt idxs hr/handbook printed the index. It follows text. An insert, update, or delete refreshes it. The embedding index docs are here.
The tools read rows and leave media paths out#
PixeltableToolset is the tool list the Pixeltable capability gives the agent. I called the methods myself. The allowlist is hr.handbook, hr.clips, and hr.frames. The capability's default instructions tell the model to describe an unfamiliar table first, and to treat the rows as data rather than as instructions to follow.
list_tables returned the three paths.
describe_table('hr.handbook') reported kind table, the columns topic and text, and the index handbook_text on text. The meals filter returned the one matching row.
The frames query left out frame and video. pos is the frame's position within its clip, and the tools return it. frame_attrs.time was 0.0. The clips query returned the title only.
Asking for the video column raised ModelRetry.
A version handle is refused by the tools. Pixeltable itself still opens one. I used that on the memory table in the next section. Older notes on versioning are in Pixeltable versioning and time travel.
The meal search embedded the question with all-MiniLM-L6-v2 and ranked the 60 EUR rule first. Each call embeds the query with the index's own function. An index built on a paid embedding API would call that API here. I left the default caps, 20 rows and 8000 characters.
A retried write returns the first result#
PixeltableMemoryStore(table_name='hr.memory') creates the table on first use. The Pixeltable guide says a write is compare-and-set on the file row, and that an operation receipt makes a retry return the first mutation. I passed expected_version=None because the path was new, and I used the same operation id and fingerprint on the second call. The fingerprint is a string you choose to identify the operation's arguments. The store looks a retry up by operation id and does not compare the text, so the second call below sends different text to show that the first result wins. Reuse an id only to retry the same write: in a separate call, the same id with a different fingerprint raised MemoryOperationConflictError, "operation id 'pref-1' was reused with different arguments". I did not send a mismatched version. Memory as rows, outside Harness, is the older post Building memory-powered AI.
The first write printed two insert lines, then the mutation. The second write printed no insert line.
existed=False because the path was new. The replay reports that same flag. list_paths returned the note and left the receipt out. The receipt is still a row, at __op__/pref-1.
This lists the table's versions and opens version 3 and version 4 by handle.
Version 0 is the schema, labeled Initial Version. Version 1 inserted nothing and updated nothing. Versions 2 and 3 each insert one row. Version 4 updates one row.
At version 3 the receipt still names the file. At version 4 the receipt's content is cleared, and the note is unchanged. The second sentence was not stored. Receipts stay in the table, and earlier versions stay readable. The guide in the pull request says dropping the table is how you purge a memory, because Pixeltable has no API to prune history. I did not drop the table. The store does not implement metadata writes: Memory(store, write_metadata={'k': 'v'}) raised UserError, which says PixeltableMemoryStore does not record write metadata.
app.py does not declare hr.memory. pxt schema diff app.py hr listed it as extra (not in schema), and pxt schema update app.py hr answered catalog is up to date and left the table alone. pxt schema prune app.py hr would drop it, and it refused to run without -f.
Pass the tools and the store to the agent#
This is the wiring only. The agent never answered a question here, because I had no model API key. Set OPENAI_API_KEY before you construct it. With the variable unset, the constructor raised UserError and pointed at Agent('test'). I set a placeholder key, the constructor returned an Agent, and I made no model request. The model string is the one in the capability docstring and in the video example. I did not check it against a live model list.
The video example transcribes audio, captions frames, and indexes that text with OpenAI. It needs OPENAI_API_KEY and a short clip that contains speech. I did not run it. How computed columns and indexes update on insert is in How it works. The same pattern with the Agno framework is in Give an Agno agent a local knowledge base.
