Extract structured data from images

How it works

What it covers

• Extract structured JSON from images using GPT-4o
• Use openai.chat_completions() with multimodal messages
• Access individual fields from the extracted data

What it uses

Install

terminal

$pip install pixeltable openai

Environment variables it names

OPENAI_API_KEY

Pixeltable calls

openai.chat_completions()pxt.create_table()@pxt.udfinsert()add_computed_column()

A look at the code

# Add extraction column using openai.chat_completions with multimodal messages
PROMPT = """Analyze this image and extract the following as JSON:
- description: A brief description of the image
- objects: List of objects visible in the image
- dominant_colors: List of dominant colors
- scene_type: Type of scene (indoor, outdoor, etc.)"""

messages = [
    {
        'role': 'user',
        'content': [
            {'type': 'text', 'text': PROMPT},
            {'type': 'image_url', 'image_url': t.image},
        ],
    }
]

t.add_computed_column(
    data=openai.chat_completions(
        messages,
        model='gpt-4o-mini',
        model_kwargs={'response_format': {'type': 'json_object'}},
    )
)

One cell of the notebook, copied as it is. Open it in Colab for the rest.

• Add watermarks to images

  You need to add watermarks to hundreds of different images to protect copyright, add branding, or mark drafts.

  Updated Apr 10, 2026Open in Colab

• Adjust image opacity

  You need to make hundreds of images semi-transparent for backgrounds, overlays, or watermarks.

  Updated Apr 10, 2026Open in Colab

• Apply image filters

  You need to apply filters to hundreds of images—blur, sharpen, edge detection, and other enhancements.

  Updated Apr 10, 2026Open in Colab

• Adjust image brightness and contrast

  You need to fix inconsistent lighting across hundreds of images—adjusting brightness, contrast, and color saturation.

  Updated Apr 10, 2026Open in Colab