Summary: In a Pixeltable VideoRAG stack, the “vector index” is an embedding index on a column—not a separate Pinecone/Weaviate deployment. Catalog metadata and index bookkeeping live in embedded Postgres under PIXELTABLE_HOME; frames and derived media live in the media store / cache. This post maps the pieces you create in code to paths on disk.
What People Mean by VideoRAG#
Typical pattern:
- Store videos in a table
- Create a view of frames with
frame_iterator - Add an embedding index (CLIP on frames, sentence embeddings on transcripts)
- Query with
column.similarity(...)for retrieval-augmented answers
Full pipeline recipes: video intelligence tutorial, video similarity search, skill reference video-rag-agents.
Minimal Pipeline#
Where Things Live on Disk#
Default home: ~/.pixeltable (override with PIXELTABLE_HOME). See storage architecture and Docker volumes.
| Artifact | What it is | Where |
|---|---|---|
| Tables / views / schema | Catalog + column defs | $PIXELTABLE_HOME/pgdata (embedded Postgres) |
| Embedding indexes | Vector index metadata + vectors managed by Pixeltable | Tied to the catalog under PIXELTABLE_HOME (not a separate SaaS vector DB) |
| Source videos | Paths/URLs you inserted | Remain at their original location (referenced, not necessarily copied) |
| Extracted frames / generated media | Materialized media Pixeltable produces | $PIXELTABLE_HOME/media (or cloud media destinations) |
| Remote URL downloads | Cached originals | $PIXELTABLE_HOME/file_cache |
| HF / Torch model weights | CLIP etc. downloads | Hugging Face / Torch caches (e.g. ~/.cache/huggingface), not inside the table |
You Do Not Need a Separate Vector Database#
add_embedding_index keeps indexes synchronized as rows are inserted or deleted—same incremental story as computed columns. Comparisons: Pixeltable vs Pinecone, incremental embedding indexes.
Views, FPS, and Storage Growth#
fps=0.5≈ one frame every two seconds—fewer rows, smaller indexes, coarser search.fps=1or higher → denser retrieval, more media and embedding storage.- Views are declarative: Pixeltable stores view definitions and computed results; you do not manually dump every frame to a folder unless you export.
On a Mac, Where Do I Look?#
If you run inside Docker, look inside the mounted volume (often /data/pixeltable), not your host home directory.
FAQ#
What is the vector index in VideoRAG?#
An embedding index on a Pixeltable column (e.g. frames.frame), queried via .similarity().
If I delete PIXELTABLE_HOME, is the index gone?#
Yes—the catalog, indexes, and local media store are gone. Source video files elsewhere on disk remain.
Can indexes live only in S3?#
Generated media can use cloud destinations; the catalog/pgdata still needs a persistent local (or PVC) home. See infrastructure docs.
