PixeltablePixeltable Logo

Build a RAG pipeline

Create a retrieval-augmented generation system that answers questions using your documents as context.

Agents · Notebook · Updated

Open in ColabRead in the docsView on GitHub

Colab runs it in your browser. If it calls a model provider, it asks for that provider's API key in a cell.

How it works

You build a pipeline that: (1) embeds documents, (2) finds relevant chunks for a query, and (3) generates an answer using those chunks as context.

What it covers

  • Embed and index documents for retrieval
  • Create a query function that retrieves context
  • Generate answers grounded in your documents

What it uses

Install
terminal
pip install pixeltable openai
Environment variables it names
OPENAI_API_KEY
Pixeltable calls
openai.chat_completions()pxt.create_table()@pxt.query@pxt.udfadd_embedding_index()insert()similarity()add_computed_column()

A look at the code

pattern-rag-pipeline.ipynb
# Add embedding index for semantic search
chunks.add_embedding_index(
    column='chunk_text',
    string_embed=embeddings.using(model='text-embedding-3-small'),
)

One cell of the notebook, copied as it is. Open it in Colab for the rest.

More in Agents

Browse all 103

  • LLM as judge

    Score model output with a second model, as a chain of computed columns

    Recipe · Updated

  • Document Indexing and RAG

    In this tutorial, we'll demonstrate how RAG operations can be implemented in Pixeltable. In particular, we'll develop a RAG application that summarizes a collection of PDF documents and uses ChatGPT to answer questions about them.

    Updated Open in Colab