Storage
What is lakehouse export?
Lakehouse export ships table data, often as Iceberg or Arrow, into an analytics lake without making the lake the system of record for media pipelines.
Updated
How it works
- The pipeline stays on the media table.
- An export writes a snapshot or a stream the warehouse can read.
- Analysts query the lake. The application still inserts into the table.
What it is not
It is not replacing the multimodal table with a warehouse.
lakehouse export: this, and the thing it is confused with
| This | Not this | |
|---|---|---|
| System of record for media | The multimodal table | The warehouse |
| Lake receives | A tabular export | The video bytes and the orchestration |
| Format | Iceberg, Arrow, or similar | A folder of mp4s |
Where Pixeltable fits
Pixeltable can export to Iceberg so a lakehouse reads the rows. The media pipeline stays on the table.
Questions
- How does lakehouse export work?
- The pipeline stays on the media table. An export writes a snapshot or a stream the warehouse can read. Analysts query the lake. The application still inserts into the table.
- What is lakehouse export often confused with?
- It is not replacing the multimodal table with a warehouse.