What is a keyframe, and where is the timestamp?

A keyframe (I-frame) is a complete picture in the compressed stream. frame_iterator(..., keyframes_only=True) emits only those stills. Each row keeps frame_attrs["index"] (stream index) and frame_attrs["time"], the presentation timestamp in seconds (pts times the stream time base). Seconds from the start of the clip was the retired pos_msec value, which subtracted the stream start time.

Can I set fps and keyframes_only together?

No. Only one of fps, num_frames, or keyframes_only can be specified. Pick the sampling contract, then apply the schema.

Yes for decoding. The iterator reads the file; ffmpeg is the decoder. Pixeltable stores the stills as image rows tied to the parent clip.

How to

How do I extract keyframes from video in Python?

Keyframe extraction pulls representative stills from a compressed video so later models run on pictures instead of every encoded frame. You choose I-frames, a fixed frames-per-second rate, or a fixed count. Each still keeps a timestamp on the source clip, which is what a video frame timestamp in Python actually is: a moment, not a filename.

Updated 2026-10-14 · Part of How do I ... in Python?

Before you start

About 15 min.

Python 3.11+
pip install -U 'pixeltable[serve]'
pxt init in the project root
ffmpeg for video decoding

1. Store the clip as a typed video column

Put each file on a table as video, plus any metadata you will filter on. A path string is not enough: later frame extraction needs a typed column.

import pixeltable as pxt

TableModel = pxt.model_base()

class Videos(TableModel, name='videos'):
    title: pxt.String
    video: pxt.Video

2. Walk the clip and emit one row per still

A view iterates the video. keyframes_only=True emits I-frames. fps=1 samples once a second. num_frames=10 emits ten evenly spaced stills. Set only one of those three.

from pixeltable.functions.video import frame_iterator

class Keyframes(
    TableModel,
    name='keyframes',
    base=Videos,
    iterator=frame_iterator(video=Videos.video, keyframes_only=True),
):
    pass

# Fixed rate: frame_iterator(video=Videos.video, fps=1)
# Evenly spaced: frame_iterator(video=Videos.video, num_frames=10)

3. Read the frame timestamp on each row

Each still keeps frame_attrs. index is the stream index. time is the presentation timestamp in seconds, pts times the stream time base. It is not seconds from the start of the clip.

@pxt.query
def list_stills(limit: int = 20):
    return (
        Keyframes.limit(limit)
        .select(Videos.title, Keyframes.frame, Keyframes.frame_attrs)
    )

# frame_attrs["index"] is the stream index
# frame_attrs["time"] is the presentation timestamp in seconds (pts * time_base)

4. Apply the schema

Write the classes in app.py. Apply them with the CLI. Inserting a new clip extracts only that clip; unchanged rows stay as they are.

# pxt init
# pxt schema update app.py keyframes -f

How you sample stills from a clip

How you sample stills from a clip
Method	What you get	When to use it
I-frames (keyframes_only)	Complete pictures in the compressed stream	Long archives, thumbnails, a first pass
Fixed rate (fps)	One still every N seconds	Actions that last about a second
Fixed count (num_frames)	N evenly spaced stills	A short preview of every clip
Every encoded frame	Huge row counts and embedding cost	Almost never for search

Where Pixeltable fits

Pixeltable is the unified multimodal backend for this job: a Video column, a frame_iterator view, and frame_attrs on each still. Insert a clip; the stills follow that row. The same file can add CLIP search or YOLOX later. Docs: extract frames from videos.

import pixeltable as pxt
from pixeltable.functions.video import frame_iterator

TableModel = pxt.model_base()

class Videos(TableModel, name='videos'):
    title: pxt.String
    video: pxt.Video

class Keyframes(
    TableModel,
    name='keyframes',
    base=Videos,
    iterator=frame_iterator(video=Videos.video, keyframes_only=True),
):
    pass

@pxt.query
def list_stills(limit: int = 20):
    return (
        Keyframes.limit(limit)
        .select(Videos.title, Keyframes.frame, Keyframes.frame_attrs)
    )

Questions

What is a keyframe, and where is the timestamp?: A keyframe (I-frame) is a complete picture in the compressed stream. frame_iterator(..., keyframes_only=True) emits only those stills. Each row keeps frame_attrs["index"] (stream index) and frame_attrs["time"], the presentation timestamp in seconds (pts times the stream time base). Seconds from the start of the clip was the retired pos_msec value, which subtracted the stream start time.
Can I set fps and keyframes_only together?: No. Only one of fps, num_frames, or keyframes_only can be specified. Pick the sampling contract, then apply the schema.
Do I need ffmpeg?: Yes for decoding. The iterator reads the file; ffmpeg is the decoder. Pixeltable stores the stills as image rows tied to the parent clip.