Retrieval
What is semantic search?
Semantic search ranks items by meaning, using embeddings, not only by exact keyword match.
Updated
How it works
- The query and the stored pieces are embedded with the same model.
- Nearest neighbors are the hits.
- Keyword search can still sit beside it when exact tokens matter.
What it is not
It is not full-text search, and it is not listing files by name.
semantic search: this, and the thing it is confused with
| This | Not this | |
|---|---|---|
| Matches | Similar meaning | Exact words only |
| Misses | A paraphrase, if you only have keywords | A filename |
| Needs | An embedding model and an index | An inverted word index alone |
Where Pixeltable fits
Pixeltable expresses it as column.similarity(...) on a column that has an embedding index.
Questions
- How does semantic search work?
- The query and the stored pieces are embedded with the same model. Nearest neighbors are the hits. Keyword search can still sit beside it when exact tokens matter.
- What is semantic search often confused with?
- It is not full-text search, and it is not listing files by name.