How to
How do I store images with embeddings and serve similarity search?
Store each image as a row with its file, compute a CLIP (or similar) embedding on that row, keep an ANN index on the column, and expose two HTTP endpoints: one that inserts images and one that takes text or an image and returns the nearest rows with scores. New images become searchable at insert.
Updated · Part of How do I ... in Python?
Before you start
About 25 min.
- Python 3.11+
- pip install -U 'pixeltable[serve]'
- pxt init in the project root
- torch and transformers for CLIP
1. Store pictures as an image column
Put each file in a table as image, plus any metadata you will filter on. A path string is not enough: the index and the upload route need a typed column.
import pixeltable as pxtfrom pixeltable.functions.huggingface import clipTableModel = pxt.model_base()clip_embed = clip.using(model_id='openai/clip-vit-base-patch32')class Images(TableModel, name='images'):image: pxt.Imagecaption: pxt.String | None
2. Declare CLIP on that column
embedding= registers both CLIP overloads, so a sentence and a still share one index. A new file is embedded when it is inserted. You do not copy vectors into a second database.
class Images(TableModel, name='images'):image: pxt.Imagecaption: pxt.String | None__indexes__ = [pxt.EmbeddingIndex(image, embedding=clip_embed),]
3. Rank by a sentence or by an uploaded still
One query embeds a sentence. The other takes an image. Both read the same index. Return the file, optional caption, and score.
@pxt.querydef search_by_text(query: str, limit: int = 5):sim = Images.image.similarity(string=query)return (Images.order_by(sim, asc=False).limit(limit).select(Images.image, Images.caption, sim))@pxt.querydef search_by_image(img: pxt.Image, limit: int = 5):sim = Images.image.similarity(image=img)return (Images.order_by(sim, asc=False).limit(limit).select(Images.image, Images.caption, sim))
4. Serve insert and both searches over HTTP
Expose an upload route for new files, a JSON search for a sentence, and a multipart search for an image. add_query_route accepts uploadfile_inputs for media-typed query parameters. Apply the schema, then start the service.
from pixeltable.serving import FastAPIRouterapi = FastAPIRouter(name='image_search')api.add_insert_route(Images,path='/images',inputs=[Images.caption],uploadfile_inputs=[Images.image],)api.add_query_route(path='/search', query=search_by_text)api.add_query_route(path='/search-image',query=search_by_image,uploadfile_inputs=['img'],)# pxt init# pxt schema update app.py image_search -f# pxt service update app.py image_search -f
5. Know where the bytes live
Local runs keep catalog, media, and cache under the instance home directory (pgdata, media, file_cache). Cloud hosts the same app file. The infrastructure guide covers volumes and the single-writer pgdata rule.
# Local default: ~/.pixeltable (pgdata, media, file_cache)# Override with PIXELTABLE_HOME.# Infrastructure guide: https://docs.pixeltable.com/howto/deployment/infrastructure
The stack coding agents usually pick vs one schema vs FiftyOne
| Job | Glue stack | One schema | FiftyOne similarity |
|---|---|---|---|
| Files | MinIO or S3 plus a path table | An Image column; media under PIXELTABLE_HOME | Samples in a Dataset; the App browses them |
| Vector index | Qdrant as a second store | EmbeddingIndex on the image column | compute_similarity with a backend you pick |
| Embeddings | OpenCLIP or CLIP in a worker | CLIP on the row at insert | Zoo model, a Model instance, or precomputed vectors |
| Job runner | Prefect between object store, embedder, and index | Computed index on insert | You re-run when the set changes |
| Serving | FastAPI you write against those copies | FastAPIRouter insert, text search, image-upload search | The FiftyOne App and sort_by_similarity, not an app backend |
| New image added | Rerun the flow or upsert by hand | Insert a row; only that file is indexed | Add samples, then update or rebuild the brain index |
| Better choice when | Those services are already the contract | The same file must store, index, and serve | Interactive visual curation and labeling QA |
Where Pixeltable fits
Pixeltable is the serving app for this job: an Image column, CLIP EmbeddingIndex, and FastAPIRouter insert plus text and image-upload search in one file. MinIO or S3 plus Qdrant plus CLIP plus Prefect plus FastAPI is the glue stack coding agents often assemble; keep it when those services are already the contract. FiftyOne compute_similarity (backends: sklearn, qdrant, redis, pinecone, mongodb, elasticsearch, pgvector, mosaic, milvus, lancedb; docs.voxel51.com similarity guide, accessed 2026-10-23) is a dataset curation and exploration tool with the FiftyOne App. It is the better choice for interactive visual curation and labeling QA. Pixeltable vs Voxel51 is that split. Pixeltable Cloud hosts the same app file. The embedding-only walkthrough is How do I embed images for search in Python; this page adds HTTP and image-to-image upload.
import pixeltable as pxtfrom pixeltable.functions.huggingface import clipfrom pixeltable.serving import FastAPIRouterTableModel = pxt.model_base()clip_embed = clip.using(model_id='openai/clip-vit-base-patch32')class Images(TableModel, name='images'):image: pxt.Imagecaption: pxt.String | None__indexes__ = [pxt.EmbeddingIndex(image, embedding=clip_embed),]@pxt.querydef search_by_text(query: str, limit: int = 5):sim = Images.image.similarity(string=query)return (Images.order_by(sim, asc=False).limit(limit).select(Images.image, Images.caption, sim))@pxt.querydef search_by_image(img: pxt.Image, limit: int = 5):sim = Images.image.similarity(image=img)return (Images.order_by(sim, asc=False).limit(limit).select(Images.image, Images.caption, sim))api = FastAPIRouter(name='image_search')api.add_insert_route(Images,path='/images',inputs=[Images.caption],uploadfile_inputs=[Images.image],)api.add_query_route(path='/search', query=search_by_text)api.add_query_route(path='/search-image',query=search_by_image,uploadfile_inputs=['img'],)# pxt init# pxt schema update app.py image_search -f# pxt service update app.py image_search -f
Questions
Do I need Qdrant or Pinecone?
Can users search with an uploaded image?
Where are the image files stored?
What happens when I change the embedding model?
How is this different from FiftyOne similarity?
Next step
Copy the commands into an agent, or run them locally with pip install pixeltable.