RAG Operations in Pixeltable

In this tutorial, we'll explore Pixeltable's flexible handling of RAG operations on unstructured text.

Agents · Notebook · Updated

Colab runs it in your browser. If it calls a model provider, it asks for that provider's API key in a cell.

What it covers

  • Creating Tables and Views
  • Data Ingestion
  • Experimenting with Chunking
  • Further Experiments
  • Computing Embeddings

What it uses

Install
terminal
pip install pixeltable sentence-transformers spacy tiktoken
Pixeltable calls
document.document_splitter()huggingface.sentence_transformer()huggingface.clip()pxt.create_table()pxt.create_view()insert()add_computed_column()

A look at the code

rag-operations.ipynb
from pixeltable.functions.huggingface import sentence_transformer
chunks.add_computed_column(
minilm_embed=sentence_transformer(
chunks.text, model_id='paraphrase-MiniLM-L6-v2'
)
)

One cell of the notebook, copied as it is. Open it in Colab for the rest.

Browse all 103