GET STARTED

Build Your First AI Workflow

Follow these four essential steps to create powerful multimodal AI applications with Pixeltable's declarative approach.

1
Step 1 / 4

1. Create a Table & Add a Computed Column

The Declarative Foundation

Start by defining a table for any kind of data—structured, unstructured, or multimodal. Then, add computed columns that transform your data using simple Python expressions. Pixeltable's declarative engine automatically orchestrates the computation for all existing and future rows, forming the foundation of your AI pipeline.

  • Unified Schema: Manage diverse data types—images, videos, documents, and structured data—in a single table.
  • Declarative Logic: Define data transformations with simple Python expressions that run automatically.
  • Automated Orchestration: Pixeltable handles the dependency graph and executes computations for new and existing data.
step_1_example.py
1import pixeltable as pxt
2

3# Create a table for films with revenue and budget
4t = pxt.create_table(
5    'films', 
6    {'name': pxt.String, 'revenue': pxt.Float, 'budget': pxt.Float}, 
7    if_exists="replace"
8)
9

10# Insert some data into the table
11t.insert([
12  {'name': 'Inside Out', 'revenue': 800.5, 'budget': 200.0},
13  {'name': 'Toy Story', 'revenue': 1073.4, 'budget': 200.0}
14])
15

16# Add a computed column for profit.
17# Pixeltable calculates this automatically for new and existing rows.
18t.add_computed_column(profit=(t.revenue - t.budget), if_exists="replace")
19

20# Query the results to see the computed profit
21# print(t.select(t.name, t.profit).collect())
22#
23# +------------+--------+
24# | name       | profit |
25# +------------+--------+
26# | Inside Out | 600.5  |
27# | Toy Story  | 873.4  |
28# +------------+--------+
2
Step 2 / 4

2. Run an AI Vision Pipeline

Bring Your Own AI Model

Extend Pixeltable's capabilities by wrapping any Python code—from simple data cleaning to complex AI model inference—in a `@pxt.udf`. Pixeltable seamlessly integrates this User-Defined Function into its declarative framework, automatically managing dependencies, execution, and even model downloads.

  • Seamless Integration: Use the `@pxt.udf` decorator to turn any Python function into a reusable pipeline component.
  • Bring Your Own Model: Integrate any custom logic, from simple data cleaning to complex AI models like YOLOX.
  • Managed Execution: Pixeltable handles dependencies, parallelization, and even downloads required model files.
step_2_example.py
1import PIL
2import pixeltable as pxt
3from yolox.models import Yolox
4from yolox.data.datasets import COCO_CLASSES
5

6# Assumes table 't' exists with an 'image' column
7# t = pxt.get_table('my_images') 
8# t.insert([{'image': 'path/to/cat.jpg'}])
9

10# Wrap any Python code in a UDF
11@pxt.udf
12def detect(image: PIL.Image.Image) -> list[str]:
13    # Load a pre-trained YOLOX model
14    model = Yolox.from_pretrained("yolox_s")
15    # Run inference
16    result = model([image])
17    # Return a list of detected class labels
18    coco_labels = [COCO_CLASSES[label] for label in result[0]["labels"]]
19    return coco_labels
20

21# Apply the UDF as a computed column
22t.add_computed_column(classification=detect(t.image))
23

24# The 'classification' column is now automatically populated
25# with the object detection results for each image.
26#
27# +----------------------+------------------+
28# | image                | classification   |
29# +----------------------+------------------+
30# | <Image: cat.jpg>     | ['cat', 'couch'] |
31# | <Image: birds.png>   | ['bird']         |
32# +----------------------+------------------+
3
Step 3 / 4

3. Perform Multimodal Vector Search

Built-in Vector Search

Add a multimodal embedding index to any column with a single line of code, eliminating the need for a separate vector DB. Pixeltable handles the entire lifecycle: generating embeddings, storing them efficiently, and keeping them automatically in sync with your data. This enables powerful, co-located semantic search across all your data types.

  • No Separate Vector DB: Create a multimodal embedding index with a single line of code, right next to your data.
  • Automatic Syncing: Embeddings are automatically created and updated when your source data changes, ensuring consistency.
  • Powerful Search: Perform text-to-image, image-to-image, and other semantic searches using a simple `.similarity()` method.
step_3_example.py
1import pixeltable as pxt
2from pixeltable.functions.huggingface import clip
3

4# Assumes table 'images' exists with an 'img' column of type Image
5# images = pxt.get_table('my_images')
6

7# 1. Add a CLIP embedding index to the image column
8images.add_embedding_index(
9    'img',
10    embedding=clip.using(model_id='openai/clip-vit-base-patch32')
11)
12

13# 2. Perform text-to-image similarity search
14query_text = "a dog playing fetch"
15sim_text = images.img.similarity(query_text)
16results = images.order_by(sim_text, asc=False).limit(1).collect()
17

18# 3. Perform image-to-image similarity search
19query_image = 'https://path/to/another/dog.jpg'
20sim_image = images.img.similarity(query_image)
21results = images.order_by(sim_image, asc=False).limit(1).collect()
4
Step 4 / 4

4. Build an Incremental RAG Workflow

Incremental & Automated RAG

Build powerful, end-to-end RAG systems with just a few declarative statements. Pixeltable orchestrates the entire workflow, from chunking documents and generating embeddings to retrieving relevant context and prompting an LLM. Because the pipeline is incremental, your RAG application is always efficient and up-to-date.

  • Declarative RAG: Define the entire RAG workflow—chunking, embedding, retrieval, and generation—with a few lines of code.
  • Incremental Processing: Only new or updated documents are processed, making your RAG system highly efficient and cost-effective.
  • Co-located Data: Keep your source documents, chunks, embeddings, and Q&A history unified and perfectly in sync.
step_4_example.py
1import pixeltable as pxt
2from pixeltable.functions import openai, huggingface
3from pixeltable.iterators import DocumentSplitter
4

5# 1. Create tables for documents and Q&A
6docs = pxt.create_table('docs', {'doc': pxt.Document})
7docs.insert([{'doc': 's3://my-data/annual-report.pdf'}])
8qa = pxt.create_table('qa', {'prompt': pxt.String})
9

10# 2. Create a view to chunk documents into sentences
11chunks = pxt.create_view('chunks', docs,
12    iterator=DocumentSplitter.create(document=docs.doc, separators='sentence'))
13

14# 3. Create an embedding index on the text chunks
15embed_model = huggingface.sentence_transformer.using(model_id='all-MiniLM-L6-v2')
16chunks.add_embedding_index('text', string_embed=embed_model)
17

18# 4. Define a query function to retrieve context
19@pxt.query
20def get_context(query_text: str, limit: int = 3):
21    sim = chunks.text.similarity(query_text)
22    return chunks.order_by(sim, asc=False).limit(limit)
23

24# 5. Build the RAG pipeline on the Q&A table
25qa.add_computed_column(context=get_context(qa.prompt))
26qa.add_computed_column(
27    answer=openai.chat_completions(
28        model='gpt-4o-mini',
29        messages=[{
30            'role': 'user',
31            'content': f"Context: {qa.context.text}\nQuestion: {qa.prompt}"
32        }]
33    ).choices[0].message.content
34)
35

36# 6. Ask a question - Pixeltable runs the whole pipeline!
37qa.insert([{'prompt': 'What were the key takeaways from the report?'}])
38# print(qa.select(qa.answer).collect())
INTERACTIVE EXPLORER

Explore Pixeltable Concepts

Dive deeper into how Pixeltable works with this interactive explorer. Learn core concepts, see multimodal data handling, and understand AI integration.

Interactive Pixeltable Explorer

Learn how Pixeltable transforms multimodal AI development

Explore Pixeltable

Progress1 of 6

What is Pixeltable?

Pixeltable is a declarative data infrastructure for building multimodal AI applications. It unifies data storage, transformation, and AI orchestration in a single Python framework.

The Problem It Solves

Traditional Approach
  • • Separate ETL pipelines
  • • Multiple vector databases
  • • Complex orchestration
  • • Brittle data flows
Pixeltable Approach
  • • Unified data tables
  • • Declarative transformations
  • • Built-in orchestration
  • • Incremental computation

Key Benefits

70%
Cost Reduction
90%
Less Code
10x
Faster Iteration
100%
Observability