---
title: "How do I turn a script into a talking-head video with an API?"
description: "Turn scripts into talking-head videos in Python: text-to-speech, a lip-sync model on fal, then transcribe the result to check it and search every clip."
url: "https://www.pixeltable.com/learn/how-to/build-a-talking-head-video-pipeline"
updated: "2026-11-02"
doc: "https://docs.pixeltable.com/howto/cookbooks/video/video-generate-ai"
---

# How do I turn a script into a talking-head video with an API?

Voice the script with a text-to-speech model, then send a portrait and that audio to a lip-sync model, which returns a video of the face speaking the audio. Keep every script, voice track, and rendered clip in one table, transcribe each render, and index the transcripts so you can find a clip later.

Updated: 2026-11-02
Part of [How do I ... in Python?](https://www.pixeltable.com/learn/how-to).

## Before you start

About 30 min.

- Python 3.11+
- pip install -U 'pixeltable[serve]'
- pxt init in the project root
- OpenAI API key
- fal.ai API key
- a portrait image URL
- ffmpeg

## On this page

- [Before you start](https://www.pixeltable.com/learn/how-to/build-a-talking-head-video-pipeline#before-you-start)
- [Store each request as a row](https://www.pixeltable.com/learn/how-to/build-a-talking-head-video-pipeline#step-1)
- [Voice the script](https://www.pixeltable.com/learn/how-to/build-a-talking-head-video-pipeline#step-2)
- [Render the talking head](https://www.pixeltable.com/learn/how-to/build-a-talking-head-video-pipeline#step-3)
- [Check what the clip says](https://www.pixeltable.com/learn/how-to/build-a-talking-head-video-pipeline#step-4)
- [Make the clips searchable and serve them](https://www.pixeltable.com/learn/how-to/build-a-talking-head-video-pipeline#step-5)
- [Comparison](https://www.pixeltable.com/learn/how-to/build-a-talking-head-video-pipeline#comparison)
- [Where Pixeltable fits](https://www.pixeltable.com/learn/how-to/build-a-talking-head-video-pipeline#where-pixeltable-fits)
- [Questions](https://www.pixeltable.com/learn/how-to/build-a-talking-head-video-pipeline#questions)

## 1. Store each request as a row {#step-1}

Each row holds a script and the URL of a portrait. Everything after that is a computed column, so inserting a row starts the work. When you add a script, only that row runs.

```python
import os

import fal_client
import pixeltable as pxt
import pixeltable.functions as pxtf
from pixeltable.functions import fal, openai
from pixeltable.serving import FastAPIRouter

TableModel = pxt.model_base()
embed = openai.embeddings.using(model='text-embedding-3-small')

@pxt.udf
def hosted_url(audio: pxt.Audio) -> str:
    # The avatar model fetches its inputs by URL, so put the voice track on fal's storage.
    return fal_client.SyncClient(key=os.environ['FAL_API_KEY']).upload_file(audio)

class Clips(TableModel, name='clips'):
    id = pxt.Column(value=pxtf.uuid.uuid7(), primary_key=True)
    script: pxt.String
    portrait_url: pxt.String
```

## 2. Voice the script {#step-2}

Send the script to a text-to-speech model and store the audio file. Pick a voice per brand or per presenter and keep it in the column definition, so every clip sounds the same.

```python
    voice = openai.speech(script, model='tts-1', voice='alloy')
```

## 3. Render the talking head {#step-3}

Lip-sync models read their inputs by URL. Upload the voice track to the provider storage, then call the model with the portrait URL and the audio URL. This example uses VEED Fabric 1.0 on fal, which takes image_url, audio_url, and resolution (720p or 480p) and returns video.url. Other image-plus-audio models on fal take similar inputs; change the app name and the input keys.

```python
    voice_url = hosted_url(voice)
    render = fal.run(
        {'image_url': portrait_url, 'audio_url': voice_url, 'resolution': '480p'},
        app='veed/fabric-1.0',
    )
    video = render['video']['url'].astype(pxt.Video)
```

## 4. Check what the clip says {#step-4}

Pull the audio track out of the rendered video and transcribe it. Compare the transcript with the script to catch truncated or garbled renders before anyone publishes them.

```python
    clip_audio = pxtf.video.extract_audio(video, format='mp3')
    heard = openai.transcriptions(clip_audio, model='whisper-1')
    transcript = heard['text'].astype(pxt.String | None)
```

## 5. Make the clips searchable and serve them {#step-5}

Add an embedding index on the transcript, and a query route that returns the closest clips for a phrase. POST a script and portrait URL to /clips. Search with /clips/search. A failed render stops the insert and returns the error. From Python, insert(..., on_error='ignore') keeps the row, and pxt recompute avatar_app/clips render --errors-only retries only the failed renders.

```python
    __indexes__ = [pxt.EmbeddingIndex(transcript, embedding=embed)]

@pxt.query
def find_clips(query: str):
    sim = Clips.transcript.similarity(string=query)
    return Clips.order_by(sim, asc=False).limit(5).select(Clips.id, Clips.script, Clips.video, Clips.transcript)

api = FastAPIRouter(name='avatar')
api.add_insert_route(Clips, path='/clips', inputs=[Clips.script, Clips.portrait_url], outputs=[Clips.id, Clips.video, Clips.transcript])
api.add_query_route(path='/clips/search', query=find_clips)

# pxt init
# pxt schema update app.py avatar_app -f
# pxt service update app.py avatar_app -f
```

## Batch API or live session? {#comparison}

|  | Batch render (this guide) | Real-time session |
| --- | --- | --- |
| Output | An MP4 you store and reuse | A live WebRTC stream |
| Typical use | Course clips, product updates, personalized video messages | Support agents, tutors, interview practice |
| Example providers | VEED Fabric and other lip-sync models on fal, HeyGen video API, Hedra Character-3 | LiveAvatar, Tavus, Simli, Anam, D-ID agents |
| Billing | Per second of rendered video | Per minute of session |

## Where Pixeltable fits {#where-pixeltable-fits}

Pixeltable does not generate faces. It runs the pipeline around the model you choose: the script and portrait go in, the voice, render, transcript, and index are computed columns, and each new row runs only its own steps. Swapping the lip-sync model is a change to one column, and pxt recompute re-renders the rows you choose. A failed render stops the insert and returns the error. From Python, insert(..., on_error='ignore') keeps the row, and pxt recompute avatar_app/clips render --errors-only retries only the failed renders.

```python
import os

import fal_client
import pixeltable as pxt
import pixeltable.functions as pxtf
from pixeltable.functions import fal, openai
from pixeltable.serving import FastAPIRouter

TableModel = pxt.model_base()
embed = openai.embeddings.using(model='text-embedding-3-small')


@pxt.udf
def hosted_url(audio: pxt.Audio) -> str:
    # The avatar model fetches its inputs by URL, so put the voice track on fal's storage.
    return fal_client.SyncClient(key=os.environ['FAL_API_KEY']).upload_file(audio)


class Clips(TableModel, name='clips'):
    id = pxt.Column(value=pxtf.uuid.uuid7(), primary_key=True)
    script: pxt.String
    portrait_url: pxt.String

    # 1. Voice the script.
    voice = openai.speech(script, model='tts-1', voice='alloy')
    voice_url = hosted_url(voice)

    # 2. Render a talking-head clip from the portrait and the voice (VEED Fabric 1.0 on fal).
    render = fal.run(
        {'image_url': portrait_url, 'audio_url': voice_url, 'resolution': '480p'},
        app='veed/fabric-1.0',
    )
    video = render['video']['url'].astype(pxt.Video)

    # 3. Check what the clip actually says, and make it searchable.
    clip_audio = pxtf.video.extract_audio(video, format='mp3')
    heard = openai.transcriptions(clip_audio, model='whisper-1')
    transcript = heard['text'].astype(pxt.String | None)

    __indexes__ = [pxt.EmbeddingIndex(transcript, embedding=embed)]


@pxt.query
def find_clips(query: str):
    sim = Clips.transcript.similarity(string=query)
    return Clips.order_by(sim, asc=False).limit(5).select(Clips.id, Clips.script, Clips.video, Clips.transcript)


api = FastAPIRouter(name='avatar')
api.add_insert_route(Clips, path='/clips', inputs=[Clips.script, Clips.portrait_url], outputs=[Clips.id, Clips.video, Clips.transcript])
api.add_query_route(path='/clips/search', query=find_clips)

# pip install -U 'pixeltable[serve]' fal-client openai
# export OPENAI_API_KEY=... FAL_API_KEY=...
# pxt init
# pxt schema update app.py avatar_app -f
# pxt service update app.py avatar_app -f
```

## Concepts

- [How does video search work?](https://www.pixeltable.com/learn/how-video-search-works)
- [What is speech-to-text?](https://www.pixeltable.com/learn/what-is-speech-to-text)

## Use case

- [Video Intelligence Pipeline: Extract, Enrich, and Search Video at Scale](https://www.pixeltable.com/use-cases/video-content-analysis)

## Questions {#questions}

### Can I use HeyGen or Hedra instead of fal? {#faq-1}

Yes. Write a @pxt.udf that calls their API and returns the video URL or a local file path, and use it in place of fal.run. The voice, transcript, and index columns stay the same. HeyGen video API pricing moved to pay-as-you-go credits in late October 2026; check the current HeyGen API pricing page rather than an old subscription table.

### Can I use an open-source lip-sync model? {#faq-2}

Yes, if it runs behind an HTTP endpoint you can call from a UDF, whether you host it or use a hosted model page.

### What does a render cost? {#faq-3}

The lip-sync provider bills per second of output video; check the model pricing page. Text-to-speech and transcription are billed by OpenAI.

### How do I stop a bad render from shipping? {#faq-4}

Compare transcript with script in another computed column (for example a word error rate UDF) and filter on it before publishing.

## Related

- [Real-time AI avatar APIs compared](https://www.pixeltable.com/learn/real-time-ai-avatar-apis-compared)
- [How do I give a real-time AI avatar memory of past conversations?](https://www.pixeltable.com/learn/how-to/give-an-ai-avatar-memory)
- [How do I transcribe audio uploads and make them searchable?](https://www.pixeltable.com/learn/how-to/transcribe-audio-and-search-it)
- [How does video search work?](https://www.pixeltable.com/learn/how-video-search-works)
- [What is speech to text?](https://www.pixeltable.com/learn/what-is-speech-to-text)
- [Video intelligence pipeline](https://www.pixeltable.com/use-cases/video-content-analysis)
- [Generate videos with AI](https://docs.pixeltable.com/howto/cookbooks/video/video-generate-ai)
- [HTTP serving and background jobs](https://docs.pixeltable.com/howto/deployment/serving)