Skip to content
All Stories
2026-10-21•16 min read

OpenTelemetry · Observability · Langfuse · Phoenix · MLflow · Opik · SigNoz · Multimodal AI · Computed Columns

Tracing multimodal AI pipelines with OpenTelemetry: Pixeltable + Langfuse, Phoenix, MLflow, Opik, and SigNoz

Pixeltable exports one OpenTelemetry trace per table operation, with a span tree for rows, files, and UDF calls. Point that same trace at Langfuse, Arize Phoenix, MLflow, Opik, or SigNoz.

Pierre Brunelle

Pierre Brunelle

Pixeltable Team

View as Markdown

Summary: OpenTelemetry data pipeline tracing in Pixeltable is one trace per table operation. insert, collect, update, and the rest each root a span named pixeltable.<operation>. Under it sits the work the engine actually did: catalog and store steps, and, when you ask, a span per row, per file, and per UDF call. Every attribute the runtime records is prefixed pxt.: pxt.table and pxt.num_rows on an insert or update, pxt.tables and pxt.rows on a query such as collect, and pxt.column, pxt.udf_path, pxt.resource_pool, and pxt.retries on the UDF call. To trace multimodal pipelines, the same tree grows a span per downloaded file and, at the finest level, a span per iterator step such as a video frame. Point the exporter at Langfuse, Arize Phoenix, MLflow, Opik, or SigNoz. The catalog does not move with the trace.

This was checked on 2026-10-21 against Pixeltable 0.7.19 and opentelemetry-instrumentation-pixeltable 0.1.0 in that tag of the Pixeltable repo, with the Langfuse SDK 5.0.4, openinference-instrumentation-openai 0.1.65, openai 3.36.0, and the Anthropic Python SDK 2.0.0. The attribute names and span levels below are from that source, not from a vendor diagram.

One operation, one trace#

A declarative pipeline is a schema. The trace is the engine running that schema, not a DAG you assembled by hand. An insert of a row whose computed column calls a model looks like this:

text

Pixeltable's own spans say which operation, row, column, and UDF ran. They do not carry row ids or cell values. A provider UDF such as chat_completions records its token counts as a pixeltable.udf.tokens event (pxt.input_tokens, pxt.output_tokens) on the UDF span. An LLM instrumentor adds the model call as a child span, with the prompt, the completion, and its own token counts. The bridge sets the active OpenTelemetry context for every UDF call, including calls on its thread pools and its async scheduler, so the generation nests under pixeltable.udf.<name>.

The operation span is filled in when the operation finishes. From OpAttrs in pixeltable/telemetry_schemas.py, that includes pxt.table (the table name), pxt.table_id, pxt.path, pxt.version, pxt.num_rows, pxt.num_computed_values, and pxt.num_excs. The UDF span, from UdfCallAttrs, carries:

  • pxt.udf_path, the UDF's fully qualified name, so two functions with the same short name stay distinct
  • pxt.column, the named column this call materializes, when it has one
  • pxt.batch_size, on a batched call
  • pxt.resource_pool and pxt.retries, on a call that went through a resource-pool scheduler. pxt.retries is how many retries this attempt has already used, so the first try is 0

Why a computed column is hard to trace#

A computed column is not a function you call on the request thread. On insert, an expression-evaluation node hands cells to asyncio tasks. A call that names a resource pool waits in a scheduler: the pool that tracks provider usage retries up to 10 times, and the fixed request-rate scheduler retries up to 3. Media downloads and saves run on a thread pool. A video view's frame_iterator yields one row per frame, and each of those rows can call a model of its own.

A patch on openai that only sees the main thread misses the worker. A trace id you set around insert() does not follow a task unless something restores the context on that task. Pixeltable's telemetry does. Worker threads take a capture_context snapshot and restore_context it before the fetch or the save. Async tasks copy a context var at creation, so concurrent UDF calls keep their own ambient span. That is the mechanism that puts the generation under the UDF span instead of beside it.

Turn it on#

The bridge, opentelemetry-instrumentation-pixeltable, is opt-in. Pixeltable itself never imports OpenTelemetry. Install it with the otel extra:

bash

In Pixeltable 0.7.19, the release this post was checked against, pixeltable[otel] pulls opentelemetry-instrumentation-pixeltable==0.1.0, the OpenTelemetry SDK, and the HTTP and gRPC exporters. openai is the client chat_completions calls; neither Pixeltable nor the instrumentor installs it.

Call pxt_otel.init() once per process, before the first table operation. It reads the endpoint from the argument, then from db_exporter_otlp_endpoint on the [[pixeltable.database]] entry in pixeltable.toml, then from OTEL_EXPORTER_OTLP_ENDPOINT, then from the [otel] section of ~/.pixeltable/config.toml. With no endpoint and no provider your application already configured, instrumentation stays inert and exports nothing. A second init() raises, so a kernel that already called it with no endpoint has to be restarted. The headers come from OTEL_EXPORTER_OTLP_HEADERS the same way. The default protocol is http/protobuf. For HTTP, init() appends /v1/traces, and /v1/metrics when metrics export is on, unless the endpoint already ends with that path.

python

Pass metrics=False for Langfuse, Phoenix, MLflow, and Opik. Those four accept traces only. Leave metrics unset for SigNoz. The default exports metrics when an endpoint is set, and SigNoz accepts them at /v1/metrics. Leave logs alone: log export is off unless you pass logs=True. An application file applies the same class with pxt schema update instead of TableModel.create_all(). Creating the table and inserting rows are separate traces. The generation is on the insert, and on an update, and when a new computed column backfills rows that already exist. A later collect() is a pixeltable.collect span and no new model call, because the computed value is stored.

A script that exits should flush. init() uses a BatchSpanProcessor, which ships on a timer while the process stays up:

python

The same trace in Langfuse, Phoenix, MLflow, Opik, and SigNoz#

Nothing in the bridge is specific to a vendor. The endpoint and the headers are the whole difference. The Langfuse, Phoenix, and MLflow paths below are from each product's docs as of 2026-10-21. Opik's row is from Opik's docs on 2026-10-29. SigNoz's row is from SigNoz's docs and Pixeltable's SigNoz page on 2026-11-07. Pass the base URL to init(). For HTTP, init() appends /v1/traces, and /v1/metrics when metrics export is on.

BackendEndpoint for init()Auth header
Langfusehttps://cloud.langfuse.com/api/public/otel (EU) or https://us.cloud.langfuse.com/api/public/otel (US). Traces are posted to /api/public/otel/v1/traces. HTTP only.Basic auth, Authorization=Basic plus the base64 of public_key:secret_key. Keys are per project and per region.
Arize PhoenixSelf-hosted HTTP: http://localhost:6006. The collector is /v1/traces on port 6006, the UI port, not the generic OTLP port 4318. gRPC: http://localhost:4317 with protocol='grpc' and no path. The http:// scheme selects plaintext; without a scheme the gRPC exporter uses TLS.Authorization=Bearer <PHOENIX_API_KEY>. Optional on a local server, required when that server has auth on.
MLflowhttp://localhost:5000. The server exposes /v1/traces. MLflow 3.6.0 or newer, a SQL backend (SQLite, PostgreSQL, MySQL, or MSSQL), HTTP only.x-mlflow-experiment-id=<experiment id>. When workspaces are on, add X-MLFLOW-WORKSPACE=<name>.
OpikCloud: https://www.comet.com/opik/api/v1/private/otel. Self-hosted: http://localhost:5173/api/v1/private/otel. Enterprise: https://<comet-deployment-url>/opik/api/v1/private/otel. HTTP only. Pass the base URL. init() appends /v1/traces.Authorization=<OPIK_API_KEY> sent raw, with no Bearer prefix, plus Comet-Workspace=<workspace>. projectName=<project> is optional. With no projectName, Opik uses Default Project. Self-hosted needs no auth header.
SigNozCloud: https://ingest.<region>.signoz.cloud:443. Self-hosted: http://<host>:4318 (the docs example is http://localhost:4318). Protocol http/protobuf. Pass the base URL, with no path. init() appends /v1/traces and /v1/metrics. The hosts ingest.signoz.io, ingest-in.signoz.io, and ingest-eu.signoz.io are deprecated.signoz-ingestion-key=<ingestion-key>, sent raw, with no Bearer prefix. Do not use the deprecated signoz-access-token header. Self-hosted omits the header.

Sources, checked 2026-10-21: Langfuse OpenTelemetry, Phoenix exporter and the Phoenix endpoint FAQ, and MLflow OTLP ingest. Opik's row, checked 2026-10-29, is from Opik's Pixeltable integration and Opik OpenTelemetry. SigNoz's row, checked 2026-11-07, is from SigNoz's Pixeltable page and SigNoz's OpenTelemetry environment variables. A remote Phoenix host is the hostname on its Settings page, with the same /v1/traces path. Pixeltable's docs publish a page for Langfuse, Arize Phoenix, Pydantic Logfire, and SigNoz. Pixeltable's own walkthrough of the bridge uses Grafana the same way: Observability.

For Opik, pass the endpoint and headers to init() in code. Opik's Pixeltable page says every OTLP exporter in the process reads OTEL_EXPORTER_OTLP_ENDPOINT and OTEL_EXPORTER_OTLP_HEADERS, so another exporter also gets the Opik endpoint and the API key. Send the API key with no Bearer prefix. Opik's troubleshooting for a 401 or 403 tells you to check that.

For Langfuse, build the header in the process and run this before pxt_otel.init(). Do not write it into config.toml; it contains the secret key. A US project key on the EU host returns 401. LANGFUSE_HOST still works and is deprecated; use LANGFUSE_BASE_URL.

python

x-langfuse-ingestion-version=4 makes a direct OTLP export show up in real time. Without it, Langfuse says ingestion can lag by about 10 minutes. The screenshot below was recorded through the Langfuse SDK path in Pitfalls, with the insert wrapped in a Langfuse span named ingest-docs.

Langfuse trace graph for a Pixeltable insert wrapped in an ingest-docs span: pixeltable.insert, its catalog, store, row, and UDF spans, and a ChatCompletion generation. The attributes panel for pixeltable.plan.create shows version 0.7.19, scope.name pixeltable, and pxt.nodes.
One Pixeltable operation in Langfuse: the span tree, and pxt.* attributes on the selected span. Phoenix, MLflow, Opik, and SigNoz take the same OTLP trace. This post has no screenshots of them.

gen_ai and OpenInference, in plain terms#

Two vocabularies describe a model call, and they are not the same attributes.

gen_ai is the OpenTelemetry GenAI semantic convention. A model-call span carries gen_ai.request.model, gen_ai.usage.*, and the prompt and completion under gen_ai.input.messages and gen_ai.output.messages. Older instrumentations write the deprecated gen_ai.prompt and gen_ai.completion. The Anthropic Python SDK 2.0.0 we tested emits these spans itself, as anthropic.messages.create, with no extra instrumentor. It records message content only when its traces content_mode is content; the default is metadata_only. Langfuse showed those spans as generations, with cost.

OpenInference is Arize's vocabulary for LLM apps. A model call's span kind (openinference.span.kind) is LLM. Tokens are llm.token_count.*. The prompt and completion are input.value and output.value. openinference-instrumentation-openai writes that, and it is what we tested under pixeltable.udf.chat_completions.

Who reads which:

  • Langfuse maps both into one generation. Its OTEL attribute table accepts gen_ai.* attributes, including gen_ai.usage.*, OpenInference input.value and llm.token_count.*, and mlflow.spanInputs. Pixeltable's pxt.* spans still appear in the tree; they are ordinary OTLP spans. The Langfuse SDK's default filter would have dropped them. See the pitfalls below.
  • Phoenix is built around OpenInference, so the generation view is that child span. The Pixeltable tree arrives with it, because the collector accepts any OTLP trace.
  • MLflow stores the OTLP spans you send. Its own SDK speaks mlflow.* and can emit gen_ai.* on the way out when MLFLOW_ENABLE_OTEL_GENAI_SEMCONV is set. For a Pixeltable trace, read the span tree in MLflow. The GenAI columns fill in when the child span speaks gen_ai.*, which the Anthropic SDK did in our test. OpenInference attributes stay on the span as attributes.
  • Opik reads both vocabularies. Its OpenTelemetry page maps gen_ai.*, including gen_ai.usage.*, and OpenInference input.value, output.value, llm.model_name, and llm.token_count.* onto the Opik span. A span with a model or token count from either vocabulary gets the span type llm. An attribute the page does not list goes on the span input, under the attribute name. Pixeltable's pxt.* attributes are in that set. The page was checked on 2026-10-29.
  • SigNoz AI Observability reads gen_ai.* on the LLM child span. OpenInference sets llm.model_name, llm.provider, and llm.token_count.* on that span. SigNoz's default attribute mapping copies those onto gen_ai.*. To emit gen_ai.* from the process, set OPENINFERENCE_ENABLE_GENAI_SEMCONV=true. SigNoz's Pixeltable page and Pixeltable's SigNoz page were checked on 2026-11-07.

Do not attach two instrumentors to the same provider. In our Anthropic test, the SDK's native spans plus openinference-instrumentation-anthropic produced a second generation for every call and double-counted cost. OpenAI needs the OpenInference instrumentor for a generation: Pixeltable records token counts as an event on the UDF span, and the backends do not show that event as a generation. Other OpenInference or OpenLLMetry instrumentors that attach to the global provider should nest the same way. We tested OpenAI and Anthropic.

Files and frames#

span_level is a threshold: info, debug, or trace. The docstring on init() matches the call sites.

LevelWhat you get
info (default)The operation span, plus catalog, store, and SQL work: pixeltable.catalog.begin_xact, pixeltable.sa.insert_rows, pixeltable.store.insert_rows, pixeltable.store.write_column, pixeltable.view_load, metadata writes.
debugAdds pixeltable.row, pixeltable.udf.<name>, per-file spans, and the finer catalog and store spans such as pixeltable.plan.create and pixeltable.store.build_rows.
traceAdds one span per iterator step, named pixeltable.iter.<iterator>. A frame_iterator step is one decoded frame.

Per-file spans, all at debug:

  • pixeltable.media.fetch, one per downloaded file, and only on a cache miss. pxt.url is set at the start with userinfo, query, and fragment stripped, because a presigned URL carries credentials. pxt.bytes is attached when the download succeeds.
  • pixeltable.media.encode, one per generated file written to a temp file.
  • pixeltable.media.save, one per persisted file, with pxt.bytes, and pxt.destination when the column uses a configured store. It is left out for the default store.

There is a cap, and it is narrow. ExprEvalNode.MAX_ROW_SPANS is 100. Each expression-evaluation node opens at most 100 pixeltable.row spans per operation, and only for rows whose parent_row is unset. Rows past that still run. A UDF span that nests only under the row span is skipped with it. A call that went through a resource-pool scheduler still emits its pixeltable.udf.* span at debug, beside the row span, with pxt.resource_pool and pxt.retries. Iterator spans have no cap in the source. span_level='trace' on a long video is one span per frame. Use debug unless you are inspecting the iterator itself. info drops the row and UDF spans, so on an insert the generation's parent is pixeltable.store.insert_rows.

python

At debug, each sampled row has a pixeltable.row span and a pixeltable.udf.resize span, and each file that had to be fetched or saved has a media span. At trace, each decoded frame adds pixeltable.iter.frame_iterator around that step. The first 100 rows of the evaluation node get the row span; the iterator spans keep going for every frame.

Lineage stays. The trace is optional.#

Your Agent Needs a Data Plane, Not a Dashboard makes a specific objection: almost every route to a pretty timeline asks you to ship traces, prompts, and user payloads to a vendor. That objection is about the trace, and it is right. It does not describe the catalog. Pixeltable does not turn telemetry on by itself. The bridge is a separate install, and pxt_otel.init() exports nothing until you configure an endpoint or attach it to a provider you set up. Rows, computed values, and column lineage stay under PIXELTABLE_HOME whether or not a span leaves the process. You choose the receiver: Langfuse, a Phoenix you run, MLflow on your machine, Opik, SigNoz, or a collector that redacts before anything crosses the network. The session log and the dependency graph remain the system of record. The span tree answers the question lineage does not: which attempt, which file, which retry, how long.

That split is also the observability layer in The Invisible 80%. Lineage, version history, and reproducibility come from the computed columns. Trace ids, input and output capture, and a latency breakdown are the export you turn on here, with a retention policy you set on the collector, not a second copy of the database.

Pitfalls#

  • The Langfuse SDK drops Pixeltable spans by default. This applies when the Langfuse client is your tracer provider. Since SDK v4, should_export_span keeps Langfuse spans, gen_ai.* spans, and known LLM-instrumentor spans. Pixeltable's scope name is pixeltable, so its spans are filtered out. Create the client first, widen the filter, and attach with PixeltableInstrumentor. pxt_otel.init() is not needed here: it would reuse the client's provider and add no exporter. Flush with langfuse.flush().
python
  • Noise. With the bridge on, create_dir, create_table, drop_dir, and collect are traces too. span_level='info' keeps the operation and the INFO work spans and drops the per-row detail.

What leaves the process#

Prompts and completions travel on the LLM child span, to whatever endpoint you set. Pixeltable's log export is off by default. Leave it off. Log records can carry API keys in error messages, signed URLs, and connection strings. If you do enable logs=True, or if you want prompts redacted before they reach a vendor, do that in an OpenTelemetry Collector in front of the backend. The fetch span already strips credentials from pxt.url. The header that authenticates the exporter contains a secret; keep it in the environment, or on a hosted database run pxt secret set pxt://<org>:<db> OTEL_EXPORTER_OTLP_HEADERS=... and then pxt service restart (see Configuration), not in a file you commit.

See also#

Frequently asked questions

How do you trace an OpenTelemetry data pipeline in Pixeltable?

Install the OpenTelemetry bridge and call pxt_otel.init() once, before the first table operation. Each operation, such as insert or collect, is one trace. An insert or update carries pxt.table and pxt.num_rows; a query such as collect carries pxt.tables and pxt.rows. At span_level debug the trace adds per-row, per-UDF, and per-file spans with pxt.column, pxt.udf_path, and pxt.retries. Configure an OTLP endpoint for Langfuse, Arize Phoenix, MLflow, Opik, SigNoz, or any collector. Export is off until you do.

How do you trace a multimodal pipeline, including video frames?

span_level debug adds one span per downloaded or saved file. span_level trace adds one span per iterator step, named pixeltable.iter.<iterator>, such as one span per decoded video frame. Those iterator spans are not capped. Per-row spans are capped at 100 per expression-evaluation node.

Does Pixeltable send prompts to Langfuse, Phoenix, MLflow, Opik, or SigNoz by default?

No. Pixeltable itself never imports OpenTelemetry. Spans leave only after you install the bridge and either give pxt_otel.init() an endpoint (the endpoint argument, db_exporter_otlp_endpoint on the database entry, OTEL_EXPORTER_OTLP_ENDPOINT, or the [otel] section of config.toml) or attach it to a TracerProvider your application configured. Rows, computed values, and column lineage stay in the catalog. Prompts and completions leave only on the child spans of an LLM instrumentor you enable, and only to the endpoint you configure.

Declarative. Multimodal. Incremental.

One Python file for tables, indexes, and HTTP. Start locally, or sign up for Cloud.

Get started

Docs quickstart · 10-minute tutorial · GitHub · Discord