Your Backend for Multimodal AI Applications

What it covers

• Store: Multimodal Tables
• Orchestrate: AI as Computed Columns
• Insert: One Call Triggers the Full Pipeline
• Retrieve: Semantic Search
• Experiment on Media Data
• Serve: Queries Become API Endpoints
• Version: Automatic History
• Agents: Tool Calling and Memory as Computed Columns

What it uses

Install

terminal

$pip install pixeltable google-genai 'fastapi[standard]' torch torchvision transformers

Environment variables it names

GEMINI_API_KEYGOOGLE_API_KEY

Pixeltable calls

uuid.uuid7()gemini.generate_content()huggingface.detr_for_object_detection()vision.bboxes_draw()anthropic.messages()anthropic.invoke_tools()pxt.create_table()@pxt.querypxt.tools()add_computed_column()

A look at the code

videos.add_computed_column(
    response=gemini.generate_content(
        [videos.video, 'Describe this video in detail.'],
        model='gemini-3-flash-preview',
    )
)

videos.add_computed_column(
    description=videos.response.candidates[0]
    .content.parts[0]
    .text.astype(pxt.String)
)

videos.add_embedding_index(
    'description',
    embedding=gemini.embed_content.using(
        model='gemini-embedding-2-preview'
    ),
)

One cell of the notebook, copied as it is. Open it in Colab for the rest.

• Configure API keys for AI services

  Set up API credentials for OpenAI, Anthropic, and other AI providers so Pixeltable can access them.

  Updated Apr 10, 2026Open in Colab

• Get fast feedback on transformations

  You need to iterate on transformation logic before running it on your entire dataset—especially for expensive operations like API calls or model inference.

  Updated Aug 14, 2026Open in Colab

• Working with the Pixeltable CLI

  You defined a multimodal pipeline in Python and now need to inspect tables, debug computed columns, roll back changes, and expose HTTP endpoints without writing more application code.

  Updated Sep 3, 2026Open in Colab

• Extract fields from LLM JSON responses

  Parse and access specific fields from structured JSON responses returned by language models.

  Updated Apr 10, 2026Open in Colab