PixeltablePixeltable Logo

Pixeltable and Twelve Labs

Twelve Labs is a hosted video API: Marengo for search embeddings, Pegasus for generated text. Pixeltable is the application schema those results can live on. Often and, not or: call Marengo as a computed index, keep the clip in a Video column you own.

pip install 'pixeltable[serve]'
See how it works

Hosted video models vs the table

SidePixeltableTwelve Labs
At a glance
  • Video column plus a video_splitter view; insert indexes new segments
  • First-party twelvelabs.embed.using(model_name='marengo3.0') on video_segment
  • Cross-modal similarity(string=), similarity(image=), similarity(audio=), similarity(video=)
  • Same file can serve FastAPIRouter; Cloud is Limited Beta
  • Marengo 3.0: any-to-video search over a hosted index (visual + audio)
  • Pegasus 1.6: generate text from video or images (Pegasus 1.5 remains for batch analyze)
  • Python SDK: indexes.create, assets.create, indexes.indexed_assets.create, search.query
  • Cloud-only core models; you do not self-host Marengo or Pegasus

What each actually owns

Twelve Labs wins video-native models and a search API. Pixeltable wins the owned catalog, incremental insert, and HTTP beside the clip. Last reviewed: Oct 2026. Sources: docs.twelvelabs.io (v1.3), docs.pixeltable.com/howto/providers/working-with-twelvelabs.

FeaturePixeltableTwelve Labs
What it is
Application schema: store, transform, index, serve
Hosted video understanding API (search + generation)
Video-native models
Calls providers; CLIP, Whisper, YOLOX, or Twelve Labs as columns
Marengo search and Pegasus generation are the product
Where the clip lives
A Video column you query, version, and attach other columns to
An asset in their index; metadata and stream URLs via their API
Insert a new file
Insert a row; video_splitter and the index run for that clip
Upload an asset, poll until ready, then indexed_assets.create
Search API
column.similarity(...) on the segment view you own
search.query against a Marengo index, with rank and time range
Images, PDFs, HTTP
Same schema: Image, Document, FastAPIRouter
Video-first; not a general multimodal application backend
Self-host
pip install pixeltable; Cloud is Limited Beta
No self-host of Marengo or Pegasus

Index a lecture and search a moment

Pixeltable: video_splitter view and EmbeddingIndex with the first-party Twelve Labs UDF. Apply with pxt schema update app.py clips. Twelve Labs SDK (v1.3): create an index, upload an asset, index it, then search.query. Last reviewed: Oct 2026.

Pixeltable

import pixeltable as pxt
from pixeltable.functions.video import video_splitter
from pixeltable.functions import twelvelabs
from pixeltable.serving import FastAPIRouter


TableModel = pxt.model_base()
embed = twelvelabs.embed.using(model_name='marengo3.0')


class Videos(TableModel, name='videos'):
    video: pxt.Video
    title: pxt.String


class Segments(
    TableModel,
    name='segments',
    base=Videos,
    iterator=video_splitter(
        video=Videos.video, duration=5.0, min_segment_duration=4.0
    ),
):
    __indexes__ = [pxt.EmbeddingIndex(video_segment, embedding=embed)]


@pxt.query
def search_clips(query: str, limit: int = 5):
    sim = Segments.video_segment.similarity(string=query)
    return (
        Segments.order_by(sim, asc=False)
        .limit(limit)
        .select(Segments.video_segment, Segments.title, sim)
    )


api = FastAPIRouter(name='clips')
api.add_insert_route(Videos, path='/videos', inputs=[Videos.video, Videos.title])
api.add_query_route(path='/search', query=search_clips)

Twelve Labs

from twelvelabs import TwelveLabs


client = TwelveLabs(api_key='...')
index = client.indexes.create(
    index_name='lectures',
    models=[{'model_name': 'marengo3.0', 'model_options': ['visual', 'audio']}],
)
asset = client.assets.create(
    method='url',
    url='https://example.com/lecture.mp4',
)
# Poll client.assets.retrieve until status == 'ready', then:
client.indexes.indexed_assets.create(
    index_id=index.id, asset_id=asset.id
)
# Poll indexed_assets.retrieve until ready, then:
hits = client.search.query(
    index_id=index.id,
    query_text='person at a whiteboard',
    search_options=['visual', 'audio'],
)
# Pegasus 1.6 is a separate analyze call for generated text.

When each belongs

Use Pixeltable when

  • You need the clip next to other columns

    Transcripts, detections, documents, and HTTP in one file. Twelve Labs can still be the embedding function on the segment view.

  • A coding agent writes the app

    One app.py to review. pip install pixeltable. Pixeltable Cloud is Limited Beta, not GA.

Use Twelve Labs when

  • Hosted video search is the product

    You want Marengo any-to-video search or Pegasus generation without owning a catalog. Pixeltable is extra if the API response is enough.

  • You already pay for a Twelve Labs index

    Keep it. Point Pixeltable at Marengo when you also need an owned table; otherwise stay on their SDK. Docs: https://docs.twelvelabs.io/

Making the right choice

  • And, not or

    • Do not migrate off Twelve Labs just to “complete the stack.” Keep it if Marengo or Pegasus is the contract.
    • Pixeltable’s first-party UDF uses model_name='marengo3.0' as of Oct 2026 (docs.pixeltable.com/howto/providers/working-with-twelvelabs).
    • Twelve Labs v1.3 flow: indexes.create → assets.create → indexed_assets.create → search.query (docs.twelvelabs.io).
    • Pegasus 1.6 generates text; batch analyze may still use Pegasus 1.5. Confirm on the current model pages.

Frequently asked questions

More comparisons

Keep the clip. Call Marengo as a column.

Declare a Video table and a video_splitter view. Apply with pxt schema update. Use Twelve Labs when the hosted model is the product.

pip install 'pixeltable[serve]'
See how it worksGet expert guidance