---
title: "Split documents into chunks for RAG"
description: "Break PDFs and documents into searchable chunks for retrieval-augmented generation (RAG) pipelines."
url: "https://www.pixeltable.com/notebooks/doc-chunk-for-rag"
updated: "2026-04-10"
---

# Split documents into chunks for RAG

Break PDFs and documents into searchable chunks for retrieval-augmented generation (RAG) pipelines.

- **Type:** Notebook
- **Group:** Text

## How it works

You create a view with a document_splitter iterator that automatically breaks documents into chunks. Then you add an embedding index for semantic search.

## What it covers

- Split PDFs into sentences with token limits
- Control chunk size with token limits
- Add embeddings for semantic search

## What it uses

- **Install:** `pip install pixeltable sentence-transformers spacy tiktoken`
- **Pixeltable calls:** `document.document_splitter()`, `pxt.create_table()`, `pxt.create_view()`, `insert()`, `add_embedding_index()`, `similarity()`

## A look at the code

```python
# Add embedding index for semantic search
chunks.add_embedding_index(
    column='text',
    string_embed=sentence_transformer.using(model_id='all-MiniLM-L6-v2'),
)
```

## Open it

- **Open in Colab:** https://colab.research.google.com/github/pixeltable/pixeltable/blob/release/docs/release/howto/cookbooks/text/doc-chunk-for-rag.ipynb
- **Docs page:** https://docs.pixeltable.com/howto/cookbooks/text/doc-chunk-for-rag
- **Source on GitHub:** https://github.com/pixeltable/pixeltable/blob/release/docs/release/howto/cookbooks/text/doc-chunk-for-rag.ipynb