---
title: "5 Surprising Truths That Will Change How You Build AI Applications"
date: "2026-01-13"
dateModified: "2026-11-07"
author: "Pixeltable Team"
tags:
  - AI Infrastructure
  - Best Practices
  - Data Management
  - Declarative AI
  - Multimodal AI
  - ML Engineering
description: "Discover why so much AI development time goes to infrastructure instead of innovation, and learn five counter-intuitive principles that can transform your approach to building multimodal AI applications."
url: "https://www.pixeltable.com/blog/5-surprising-truths-building-ai-applications"
---

# 5 Surprising Truths That Will Change How You Build AI Applications

The journey of building an AI application often begins with a spark of excitement. The prospect of creating intelligent systems that can analyze video, understand documents, or power sophisticated agents is a powerful motivator. But for many teams, this initial enthusiasm quickly gives way to a frustrating reality: wrestling with a tangled web of infrastructure.
 

 

 Before any real AI work can begin, you're stuck stitching together a chaotic mix of databases, object storage, vector indexes, and pipeline orchestrators. This is the **infrastructure tax on AI innovation**: engineering teams spend much of their time on data plumbing instead of building the valuable AI features that drive business value.
 

 

 The constant need to write custom ETL scripts, manage data synchronization between systems, and manually re-process entire datasets for minor changes becomes a significant drag on productivity and a major source of cost. What if there was a simpler path?
 

 

 By analyzing the architecture of modern AI applications, we've uncovered five counter-intuitive but powerful truths. These truths challenge common assumptions about AI infrastructure and reveal a more streamlined, efficient, and intuitive way to build.
 

 
## 1. The Infrastructure Tax is Real, and Optional

 

 The common approach to building AI infrastructure is a fragmented one. Teams typically find themselves stitching together a collection of specialized, single-purpose tools:
 

 

 - **S3 or other cloud storage** for raw files

 - **Postgres or data warehouses** for structured data

 - **Pinecone or other vector databases** for embeddings

 - **Airflow or Prefect** for orchestration

 - **Custom ETL scripts** to connect everything

 

 

 This fragmented stack is the direct cause of the infrastructure tax. The counter-intuitive truth is that you can [replace this entire complex stack](/blog/unified-multimodal-ai-infrastructure-pixeltable) with a single, unified platform.
 

 

 Instead of writing imperative boilerplate just to ingest multimodal data, a [declarative approach](/blog/declarative-vs-imperative-ai-pipelines) allows you to achieve the same result (ingesting video, image, and audio into a single, unified, and queryable table) with one table definition:
 

 
```python
import pixeltable as pxt

# Create a unified multimodal table
pxt.create_dir('media_analysis', if_exists='ignore')
media = pxt.create_table('media_analysis.content', {
 'source_id': pxt.String,
 'video': pxt.Video,
 'audio': pxt.Audio,
 'image': pxt.Image,
 'document': pxt.Document,
 'metadata': pxt.Json,
})

# That's it. No S3 buckets, no separate databases, 
# no ETL pipelines to connect them.
```

 

 By adopting a unified data platform, you aren't just reducing lines of code; you are eliminating entire categories of work. You stop paying the infrastructure tax and can finally refocus your team's energy on what truly matters: [innovation and building next-generation AI features](/blog/building-multimodal-apps).
 

 
## 2. Your Database Shouldn't Be Dumb About Data

 

 In the traditional database model, tables are simple containers for scalar values. Rich media like images, videos, and audio are treated as opaque binary large objects (BLOBs). The database can store them, but it has no understanding of their content. This forces all the intelligent processing to happen in a separate application layer.
 

 

 The surprising truth is that a modern AI data platform treats the table as a **"universal interface"** where [multimodal data (images, videos, audio, and documents) are first-class citizens](/blog/traditional-tables-to-multimodal-ai). This is more than just storage; it means the data platform itself can compute on and understand the content.
 

 

 You can integrate AI model calls directly into the data layer by adding a computed column. Its expression can call a built-in model function, as below, or a [User-Defined Function (UDF)](/blog/python-udfs-pixeltable) with arbitrary logic, such as calling a vision model on an image or transcribing an audio file:
 

 
```python
import pixeltable as pxt
from pixeltable.functions.openai import chat_completions

# The table from the previous example
analysis = pxt.get_table('media_analysis.content')

# Add an AI-powered computed column: a vision model on competitor screenshots
analysis.add_computed_column(
 competitor_analysis=chat_completions(
 model='gpt-4o',
 messages=[{
 'role': 'user',
 'content': [
 {'type': 'text', 'text': 'Analyze this competitor pricing screenshot. Extract key pricing tiers and features.'},
 {'type': 'image_url', 'image_url': {'url': analysis.image}},
 ],
 }],
 ).choices[0].message.content
)
```

 

 This represents a major paradigm shift. The data platform is no longer a passive container for data; it's an active participant in understanding and enriching it, turning raw media into structured, queryable insights. Learn more about [integrating vision models](/blog/working-with-gemini) and [multimodal AI providers](/blog/claude-anthropic-multimodal-integration-pixeltable).
 

 
## 3. The Most Expensive Compute is the One You Run Twice

 

 A common and incredibly wasteful practice in AI development is re-processing entire datasets when a small change is made. Whether you update a model, tweak a parameter, or fix a bug in your transformation logic, the standard response is often to re-run the entire pipeline, incurring significant costs in both time and API fees.
 

 

 The impactful truth is that with [incremental computation](/blog/dependency-graph-magic-computed-columns), you can avoid redoing the work a change does not touch. Consider two everyday changes to a RAG system with 10,000 documents: adding new documents, and adding a summary step for each chunk:
 

 
| Traditional Approach | Incremental Approach |
| --- | --- |
| Re-run the pipeline over all 10,000 documents to pick up two new ones | Insert the two documents: only their chunks are split and embedded |
| Re-generate every embedding when a summary step is added | Add the summary as a computed column: it computes for each chunk, and the stored embeddings stay |
| Delete the vector index and re-upload everything | The index adds entries for new chunks only |

 

 This is possible because the platform maintains a [dependency graph](/blog/context-graphs-decision-traces-agentic-ai) of all computations, allowing it to precisely trace the impact of any change. Work outside the change is not redone: columns that do not depend on the chunks keep their stored values, and new documents compute only their own rows. Changing the chunking strategy itself is different: recreating the chunks view re-chunks every document and re-embeds every chunk, while columns on the documents table keep their stored values.
 

 

 In video analysis, you can analyze only the [keyframes](/blog/video-keyframe-extraction-pixeltable) (`frame_iterator(video, keyframes_only=True)`), a fraction of all frames that depends on how the video was encoded. This shifts the focus from merely optimizing re-computation to eliminating it entirely, proving that the most expensive compute is the one you never needed to run again.
 

 
```python
import pixeltable as pxt
from pixeltable.functions.document import document_splitter

# The documents table, with a chunks view and an embedding index on it
docs_table = pxt.get_table('rag.docs')

# When you add new documents, only new embeddings are computed
docs_table.insert([
 {'document': '/new/document.pdf'},
 {'document': '/another/document.pdf'},
])

# Existing 10,000 documents? Their embeddings are untouched.
# Only the 2 new documents get processed.

# Update chunking strategy? Recreate the chunks view: every document is
# rechunked, and an embedding index added to the new view embeds every chunk.
# Columns on docs_table keep their stored values.
chunks = pxt.create_view(
 'rag.chunks', docs_table,
 iterator=document_splitter(
 document=docs_table.document,
 separators='token_limit',
 limit=500 # Changed from 300
 ),
 if_exists='replace_force'
)
```

 
## 4. AI Agents Need a Record of Decisions, Not Just Data

 

 As we build more sophisticated AI agents, a high-level truth is emerging: the next generation of enterprise software will be built not on systems of record for objects, but on **"systems of record for decisions."**
 

 
> 
 "Context graphs represent a living record of decision traces stitched across entities and time so precedent becomes searchable."
 Foundation Capital on the trillion-dollar opportunity in agentic AI
 

 

 This concept of a "decision trace" is critical for [building robust and auditable AI agents](/blog/practical-guide-building-agents). It's not enough to have access to data; the agent needs the full context of *why* a decision was made, including the inputs, the model version, the prompt used, and the final response.
 

 

 The surprising implementation detail is that a [dependency graph within an intelligent data platform is a context graph](/blog/context-graphs-decision-traces-agentic-ai). Every computed column is a node in the graph, representing a step in the decision-making process. Every row in the table becomes a complete, versioned, and queryable decision trace.
 

 
```python
# Every row is a complete decision trace
agent_decisions = pxt.create_table('ops.support_routing', {
 'ticket_id': pxt.String,
 'customer_tier': pxt.String,
 'ticket_content': pxt.String,
 'policy_version': pxt.String, # Captured at decision time
})

# Add reasoning steps as computed columns
# (classify_intent, calculate_priority and route_request are your own @pxt.udf functions)
agent_decisions.add_computed_column(
 intent_classification=classify_intent(agent_decisions.ticket_content)
)

agent_decisions.add_computed_column(
 priority_score=calculate_priority(
 agent_decisions.intent_classification,
 agent_decisions.customer_tier
 )
)

agent_decisions.add_computed_column(
 routing_decision=route_request(
 agent_decisions.intent_classification,
 agent_decisions.priority_score
 )
)

# Query decisions with full context: "Why did we route this ticket to tier-3?"
# The dependency graph shows: ticket_content → intent → priority → decision
```

 

 This infrastructure-level approach captures the full lineage of every decision automatically, providing the foundation needed to build [stateful, memory-powered AI agents](/blog/building-memory-powered-ai-stateful-agents-pixeltable). With [time travel capabilities](/blog/pixeltable-versioning-time-travel), you can even replay the exact state at any decision point.
 

 
## 5. You Can Stop Manually Orchestrating Your Pipelines

 

 Data pipelines are typically built using an imperative approach. With tools like [Airflow](/blog/pixeltable-vs-airflow-ml-orchestration), you must manually define every step, manage dependencies between tasks, and specify the exact order of execution. This often results in hundreds of lines of code dedicated to managing the "how" of the pipeline, rather than the "what."
 

 

 The surprising alternative is the [declarative approach](/blog/declarative-ai-pipelines-open-standard). Instead of writing a script that says "first extract frames, then run object detection, then generate embeddings," you simply declare the desired end state: *"this column contains object detections from that frame column."*
 

 
```python
# Declarative: Define WHAT you want, not HOW to get it
from pixeltable.functions.huggingface import clip
from pixeltable.functions.video import frame_iterator
from pixeltable.functions.yolox import yolox

video_analysis = pxt.create_table('analysis.videos', {
 'video': pxt.Video,
})

# Declare frame extraction: a view with one row per frame
frames = pxt.create_view(
 'analysis.frames', video_analysis,
 iterator=frame_iterator(video=video_analysis.video, fps=1)
)

# Declare object detection on frames
frames.add_computed_column(
 detections=yolox(frames.frame, model_id='yolox_s')
)

# Declare an embedding index on frames
frames.add_embedding_index(
 'frame', embedding=clip.using(model_id='openai/clip-vit-base-patch32')
)

# The system automatically:
# ✓ Discovers dependencies
# ✓ Determines execution order
# ✓ Parallelizes independent work
# ✓ Handles updates when data changes
```

 

 The system automatically discovers the dependencies, determines the correct execution order, parallelizes independent work, and handles all updates when the underlying data changes. This is why teams are [switching from complex orchestration tools](/blog/teams-switching-pixeltable-vector-databases) to declarative platforms.
 

 

 When you publish a dataset built this way, you're sharing everything as one complete unit:
 

 

 - **Raw data**: The original videos, images, documents

 - **Computed results**: All transformations and model outputs

 - **Embedding indexes**: Ready for similarity search

 - **Schema and metadata**: Complete reproducibility

 

 

 This declarative model frees developers from the tedious, error-prone work of pipeline orchestration. It allows them to focus their expertise on the core AI logic. See [data sharing in action](/blog/pixeltable-data-sharing-launch) for more details.
 

 
## Conclusion: The Future of AI is Declarative and Unified

 

 The five truths we've explored point to a clear and powerful conclusion: the chaotic, fragmented era of AI infrastructure is giving way to a new paradigm. This future is centered on **unified, declarative, and intelligent data platforms** that manage complexity automatically, so teams spend their time on the application instead of the plumbing.
 

 

 Let's recap what we've learned:
 

 

 - **The infrastructure tax is optional:** unified platforms replace the glue code between storage, database, vector index and orchestrator

 - **Databases should understand your data:** multimodal types and computed columns enable AI-native workflows

 - **Incremental computation avoids rerunning work:** dependency graphs eliminate redundant processing

 - **Decision traces enable auditable AI:** context graphs capture the "why" behind every decision

 - **Declarative beats imperative:** focus on what you want, not how to orchestrate it

 

 

 By eliminating the manual, imperative work of data plumbing, we can finally focus on the true goal of AI development: building applications that deliver real value.
 

 

 **If the data plumbing were gone, what AI features would you build next?**
 

 

 Ready to experience these truths firsthand? Start with our [10-minute tutorial](/blog/your-first-pixeltable-project) or explore the [interactive playground](/playground) to see declarative AI infrastructure in action.