monitoringmoderateOfficial

Langfuse Integration

Export Pixeltable OpenTelemetry spans to Langfuse. One insert is one trace; LLM generations nest under UDF spans with tokens and cost.

Get Started Docs

Challenge

Computed columns hide the HTTP call inside insert. That is the point of the table, and it is a problem for an observability product that expects a trace around each generation. If you only patch the OpenAI client, you get generations with no Pixeltable operation, row, or UDF path. If you only log locally, Langfuse never sees the prompt. The bridge is OpenTelemetry: Pixeltable emits spans, Langfuse accepts OTLP/HTTP at /api/public/otel. You choose whether that is enough, or whether you also want the Langfuse SDK for @observe, tags, and one trace around a batch.

Solution

pip install pixeltable[otel] openai langfuse openinference-instrumentation-openai. Set LANGFUSE_PUBLIC_KEY, LANGFUSE_SECRET_KEY, and LANGFUSE_BASE_URL to the region that holds the project (US, EU, Japan, or HIPAA hosts in the Langfuse docs). Call pxt_otel.init() once per process with that host plus /api/public/otel, Basic auth from the two keys, and span_level debug if you want row and UDF spans. Instrument OpenAI so generations carry model, tokens, and cost. Then insert. One Pixeltable operation is one trace. Do not install openinference-instrumentation-anthropic beside the Anthropic SDK 2.x spans: that pair records each call twice. This page matches Pixeltable 0.7.19+, Langfuse SDK 5.x, and openinference-instrumentation-openai as the Langfuse docs state.

Integration Steps

Step 1 of 4

Keys belong to one Langfuse project and one region. A key sent to another host returns 401.

pip install 'pixeltable[otel]' openai langfuse openinference-instrumentation-openai
export LANGFUSE_PUBLIC_KEY=pk-lf-...
export LANGFUSE_SECRET_KEY=sk-lf-...
export LANGFUSE_BASE_URL=https://us.cloud.langfuse.com
export OPENAI_API_KEY=sk-...

💡 pixeltable[otel] installs the Pixeltable bridge, the OpenTelemetry SDK, and OTLP exporters. openai is the client chat_completions() uses. LANGFUSE_BASE_URL is the regional host, not the OTLP path.

Use arrow keys to navigate

Key Benefits

No Langfuse SDK required for operation, row, UDF, and generation traces

One insert is one trace; generations nest under the UDF span

Tokens and cost come from the OpenAI instrumentor, not a hand-rolled logger

span_level info drops per-row spans when a batch would be noisy

Served apps use --otel plus env, so app.py does not call init()

Use Cases

•Inspect which column and UDF produced a completion

•Cost of gpt-4o-mini on a Pixeltable insert

•Tag a batch ingest with the Langfuse SDK

•Debug a served FastAPIRouter that calls computed columns

•Compare hosted OpenAI columns with a second provider without double-counting

Technical Requirements

•Python 3.11+

•pip install 'pixeltable[otel]' openai langfuse openinference-instrumentation-openai

•LANGFUSE_PUBLIC_KEY, LANGFUSE_SECRET_KEY, LANGFUSE_BASE_URL for one region

•OPENAI_API_KEY when tracing chat_completions()

Learn More

Langfuse Observability

AI Agent Architecture: A Practical Guide to Building Agents with State Management Build Multimodal AI Applications Faster with Pixeltable Infrastructure

Related Integrations

Hugging Face
Modality: Model API
Computed column: Hugging Face: Index images with CLIP
Docs
Amazon S3
Modality: Storage
Computed column: Amazon S3: Insert s3:// URIs as media columns
Docs
FiftyOne
Modality: Model API
Computed column: FiftyOne: Store stills and run ViT plus DETR
Docs

Ready to Integrate Langfuse?

Get started with Pixeltable and Langfuse in minutes.

pip install pixeltable

View on GitHub Starter Kit Get started Try in Notebooks