---
title: "Extract structured data from images"
description: "Use AI vision to extract JSON data from receipts, forms, documents, and other images."
url: "https://www.pixeltable.com/notebooks/vision-structured-output"
updated: "2026-04-10"
---

# Extract structured data from images

Use AI vision to extract JSON data from receipts, forms, documents, and other images.

- **Type:** Notebook
- **Group:** Images

## How it works

You use Pixeltable's openai.chat_completions() function with multimodal messages that include images directly. Request JSON output via response_format in model_kwargs.

## What it covers

- Extract structured JSON from images using GPT-4o
- Use openai.chat_completions() with multimodal messages
- Access individual fields from the extracted data

## What it uses

- **Install:** `pip install pixeltable openai`
- **Environment variables it names:** `OPENAI_API_KEY`
- **Pixeltable calls:** `openai.chat_completions()`, `pxt.create_table()`, `@pxt.udf`, `insert()`, `add_computed_column()`

## A look at the code

```python
# Add extraction column using openai.chat_completions with multimodal messages
PROMPT = """Analyze this image and extract the following as JSON:
- description: A brief description of the image
- objects: List of objects visible in the image
- dominant_colors: List of dominant colors
- scene_type: Type of scene (indoor, outdoor, etc.)"""

messages = [
    {
        'role': 'user',
        'content': [
            {'type': 'text', 'text': PROMPT},
            {'type': 'image_url', 'image_url': t.image},
        ],
    }
]

t.add_computed_column(
    data=openai.chat_completions(
        messages,
        model='gpt-4o-mini',
        model_kwargs={'response_format': {'type': 'json_object'}},
    )
)
```

## Open it

- **Open in Colab:** https://colab.research.google.com/github/pixeltable/pixeltable/blob/release/docs/release/howto/cookbooks/images/vision-structured-output.ipynb
- **Docs page:** https://docs.pixeltable.com/howto/cookbooks/images/vision-structured-output
- **Source on GitHub:** https://github.com/pixeltable/pixeltable/blob/release/docs/release/howto/cookbooks/images/vision-structured-output.ipynb