---
title: "User-uploaded media pipeline reference architecture"
description: "A user-upload media pipeline: HTTP, typed columns, hash checks, models on insert, then search. Named alternatives including S3 and Qdrant."
url: "https://www.pixeltable.com/reference-architecture/user-uploaded-media-pipeline"
updated: "2026-10-28"
doc: "https://docs.pixeltable.com/howto/deployment/serving"
---

# User-uploaded media pipeline reference architecture

User-uploaded media is an HTTP route, a typed row, optional exact-hash and near-duplicate checks, models that run on insert, and an index that stays current. The glue stack is FastAPI plus S3 plus a queue plus a vector database. One TableModel file can cover the same slots.

Updated: 2026-10-28
Docs: https://docs.pixeltable.com/howto/deployment/serving

An upload pipeline has a different failure mode than a batch corpus job. Two requests can race. POST /images does not run a near-duplicate check unless you call that check yourself. The how-to How do I deduplicate a large image dataset states that limit in the code comments.
This architecture covers images, and the same HTTP pattern for video and audio. Image similarity search over HTTP is How do I store images with embeddings and serve similarity search. Audio is How do I transcribe audio uploads and make them searchable. Video insert is on the video search architecture.

## User-uploaded media: HTTP, a typed row, then models and an index {#diagram}

```mermaid
flowchart TB
  client[Browser or app] --> http[POST multipart]
  http --> row[Typed Image / Video / Audio]
  row --> hash[Optional hash lookup]
  hash --> models[Computed columns]
  models --> idx[EmbeddingIndex]
  idx --> search[GET or POST /search]
  http -.->|keep if already the contract| fastapi[FastAPI + S3 + SQS]
  idx -.->|keep if already the serving index| qdrant[Qdrant / pgvector]
```

## Role of each component

- **Upload HTTP.** Multipart insert. FastAPIRouter add_insert_route with uploadfile_inputs. background=True returns a job id for long transcriptions.
- **Typed media column.** Image, Video, or Audio. A path string is not enough for later models. Locally the bytes live under PIXELTABLE_HOME. Cloud hosts the same app file.
- **Exact-hash check.** A UDF that hashes a normalized RGBA PNG. Look up the digest before CLIP so identical pixels never need a vector comparison. There is no built-in image hash function.
- **Near-duplicate check.** CLIP similarity above a threshold you tune. Call it before insert. A computed column on Images that queried Images would be self-referencing.
- **Enrichment.** CLIP, Whisper, YOLOX, or an LLM caption as computed columns. Insert runs the affected expressions. Unchanged rows stay cached.
- **Search HTTP.** A @pxt.query exposed with add_query_route. Image-to-image search uses uploadfile_inputs on the query route; GET cannot take uploads.

## Honest alternatives per slot {#alternatives}

| Slot | Role | Pixeltable | Named alternatives |
| --- | --- | --- | --- |
| Object store | Bytes for uploads | Media columns; PIXELTABLE_HOME locally; Cloud for the hosted file | Amazon S3: The bucket is already the CDN origin or the archive; Cloudflare R2: S3-compatible storage with no egress fee on that vendor; MinIO: An S3 API inside your network |
| API / queue | Accept the upload and run work off the request | FastAPIRouter insert; background=True for long jobs | FastAPI + SQS + Lambda: Those services are already the contract; Celery + Redis: You already run workers next to a custom API |
| Vector index | Search and near-duplicate ranking | EmbeddingIndex on the media or transcript column | Qdrant: You already upsert vectors from a worker; pgvector: Postgres already holds the rest of the app; Pinecone: A hosted ANN namespace is the serving contract |
| Moderation / virus scan | Reject unsafe or malicious files | Not built in. Run a scanner in front of insert, or a UDF you own | ClamAV or a cloud malware API: You must scan bytes before they are stored; A vision moderation API: You must score NSFW or policy labels; Pixeltable can call it as a column |

## Where Pixeltable fits {#where-pixeltable-fits}

Pixeltable is the upload app in this architecture: typed columns, FastAPIRouter insert and search, a hash UDF, and CLIP on insert. Keep S3 plus SQS plus Qdrant when those services are already the contract. POST /images does not call keep_if_new; two uploads at once can both pass. Pixeltable Cloud hosts the same app file. The how-tos for image search and image dedup are the step-by-step versions of this sketch.

```python
import hashlib
from io import BytesIO

import pixeltable as pxt
from pixeltable.functions.huggingface import clip
from pixeltable.serving import FastAPIRouter

TableModel = pxt.model_base()
clip_embed = clip.using(model_id='openai/clip-vit-base-patch32')

def rgba_png_sha256(img) -> str:
    buf = BytesIO()
    img.convert('RGBA').save(buf, format='PNG')
    return hashlib.sha256(buf.getvalue()).hexdigest()

@pxt.udf
def image_sha256(img: pxt.Image) -> str:
    return rgba_png_sha256(img)

class Images(TableModel, name='images'):
    caption: pxt.String | None
    image: pxt.Image
    file_sha256 = image_sha256(image)
    __indexes__ = [
        pxt.EmbeddingIndex(image, embedding=clip_embed),
    ]

@pxt.query
def search_by_text(query: str, limit: int = 5):
    sim = Images.image.similarity(string=query)
    return (
        Images.order_by(sim, asc=False)
        .limit(limit)
        .select(Images.image, Images.caption, sim)
    )

@pxt.query
def search_by_image(img: pxt.Image, limit: int = 5):
    sim = Images.image.similarity(image=img)
    return (
        Images.order_by(sim, asc=False)
        .limit(limit)
        .select(Images.image, Images.caption, sim)
    )

def keep_if_new(candidate, caption: str | None = None, threshold: float = 0.95) -> None:
    images = pxt.get_table('uploads.images')
    digest = rgba_png_sha256(candidate)
    exact = images.where(images.file_sha256 == digest).select(images.file_sha256).collect()
    if len(exact) > 0:
        return
    sim = images.image.similarity(image=candidate)
    hits = (
        images.where(sim > threshold)
        .order_by(sim, asc=False)
        .limit(10)
        .select(images.file_sha256, sim)
        .collect()
    )
    if len(hits) == 0:
        images.insert([{'image': candidate, 'caption': caption}])

api = FastAPIRouter(name='uploads')
api.add_insert_route(
    Images,
    path='/images',
    inputs=[Images.caption],
    uploadfile_inputs=[Images.image],
)
api.add_query_route(path='/search', query=search_by_text)
api.add_query_route(
    path='/search-image',
    query=search_by_image,
    uploadfile_inputs=['img'],
)

# POST /images does not call keep_if_new. Call it yourself after schema update.
# pxt init
# pxt schema update app.py uploads -f
# pxt service update app.py uploads -f
```

To run a sketch, install pixeltable[serve] and the packages its model functions import (such as sentence-transformers, openai, or openai-whisper), export the provider key a model call reads (such as OPENAI_API_KEY), save the file as app.py, and run the commands in its last lines. The tables start empty: insert a row, or POST to an insert route, then select the computed columns. [Get started](https://www.pixeltable.com/get-started) covers the CLI.

## Questions {#questions}

### Does the insert route reject duplicates? {#faq-1}

No. add_insert_route writes the row. keep_if_new is a function you call before insert. Two uploads at once can both pass and both land. The image-dedup how-to states that limit.

### Where should malware scanning run? {#faq-2}

In front of insert, or as a UDF you own. Pixeltable does not scan uploads. A ClamAV sidecar or a cloud malware API is the usual slot.

### Can the same pattern serve video or audio? {#faq-3}

Yes. Swap the column type and the models. Video search and audio transcription architectures on this hub use uploadfile_inputs the same way. background=True fits a long Whisper job.

## Related

- [How do I store images with embeddings and serve similarity search?](https://www.pixeltable.com/learn/how-to/serve-image-similarity-search)
- [How do I deduplicate a large image dataset?](https://www.pixeltable.com/learn/how-to/deduplicate-images-with-embeddings)
- [How do I transcribe audio uploads and search them?](https://www.pixeltable.com/learn/how-to/transcribe-audio-and-search-it)
- [Travel scrapbook example app](https://www.pixeltable.com/recipes/travel-scrapbook)
- [Examples hub: image dedup](https://www.pixeltable.com/examples)
- [Pixeltable vs Pinecone](https://www.pixeltable.com/compare/pixeltable-vs-pinecone)
- [HTTP serving (docs)](https://docs.pixeltable.com/howto/deployment/serving)
- [Find similar images with CLIP (docs)](https://docs.pixeltable.com/howto/cookbooks/search/search-similar-images)
- [Load media from S3 (docs)](https://docs.pixeltable.com/howto/cookbooks/data/data-import-s3)

This is the architecture. Step-by-step Python is on [how-to guides](https://www.pixeltable.com/learn/how-to) and [examples](https://www.pixeltable.com/examples).