ml frameworkmoderateOfficial

FiftyOne Integration

Export a Pixeltable image column to a Voxel51 FiftyOne dataset, including ViT classifications and DETR detections, then open the FiftyOne App.

Get Started Docs

Challenge

FiftyOne is a dataset app: browse stills, overlay detections, compare label sets. Pixeltable is the table that stores those stills and runs models as columns. They meet at export. A folder of JPEGs plus a JSON of boxes is what you would otherwise assemble by hand. Pixeltable already has the image, the ViT dict, and the DETR boxes. The integration is pxt.io.export_images_as_fo_dataset, which builds a FiftyOne Dataset from one image expression plus optional classification and detection expressions. It is not a second store and not a replacement for the FiftyOne App.

Solution

pip install pixeltable fiftyone torch transformers timm. Declare an Image column, optional vit_for_image_classification and detr_for_object_detection columns, then map those JSON blobs into the list-of-dicts FiftyOne expects (label plus confidence; detections also need bounding_box as relative x, y, w, h). Call export_images_as_fo_dataset(table, image_expr, classifications=..., detections=...) and fo.launch_app(dataset). Images already on disk export in their stored format; frames that exist only as iterator output are written first. Multiple DETR checkpoints can be named dict keys so you compare them in one App session. Pixeltable vs Voxel51 is the product split: App for interactive curation, table for the production schema.

Integration Steps

Step 1 of 4

Keep the file, the classification dict, and the detection dict on one row.

import pixeltable as pxt
from pixeltable.functions.huggingface import (
    detr_for_object_detection,
    vit_for_image_classification,
)

TableModel = pxt.model_base()

class Images(TableModel, name='images'):
    image: pxt.Image
    classifications = vit_for_image_classification(
        image, model_id='google/vit-base-patch16-224'
    )
    detections = detr_for_object_detection(
        image, model_id='facebook/detr-resnet-50'
    )

# pxt schema update app.py fo_demo -f

💡 ViT must be a classification checkpoint, not a feature-extraction-only id. DETR boxes are absolute [x1, y1, x2, y2] in pixels. FiftyOne wants relative [x, y, w, h], so the next step converts them.

Use arrow keys to navigate

Key Benefits

No JSON dump of boxes to load FiftyOne by hand

ViT and DETR stay computed columns on the source image

Named detection sets compare two checkpoints in one App session

On-disk stills export in their stored format

The table keeps serving and search; FiftyOne keeps curation

Use Cases

•Review DETR boxes on a production image table

•Compare ResNet-50 and ResNet-101 detections side by side

•Browse ViT classifications in the FiftyOne App

•Export frames from a video iterator once they exist on disk

•Interactive labeling QA without making FiftyOne the serving backend

Technical Requirements

•Python 3.11+

•pip install -U 'pixeltable[serve]' fiftyone torch transformers timm

•A table with an Image column

•pxt schema update before export if you added model columns

Learn More

Working with Voxel51 for visualization Pixeltable vs Voxel51 Hugging Face functions SDK

Build Multimodal AI Applications Faster with Pixeltable Infrastructure Your First Pixeltable Project: Build a Smart Image Organizer in 10 Minutes

Related Integrations

OpenAI
Modality: Model API
Computed column: OpenAI: Setup OpenAI Integration
Docs
Anthropic
Modality: Model API
Computed column: Anthropic: Install and set the API key
Docs
Gemini
Modality: Model API
Computed column: Gemini: Authenticate AI Studio or Vertex
Docs

Ready to Integrate FiftyOne?

Get started with Pixeltable and FiftyOne in minutes.

pip install pixeltable

View on GitHub Starter Kit Get started Try in Notebooks