How to

How do I turn a script into a talking-head video with an API?

Voice the script with a text-to-speech model, then send a portrait and that audio to a lip-sync model, which returns a video of the face speaking the audio. Keep every script, voice track, and rendered clip in one table, transcribe each render, and index the transcripts so you can find a clip later.

Updated 2026-11-02 · Part of How do I ... in Python?

Before you start

About 30 min.

Python 3.11+
pip install -U 'pixeltable[serve]'
pxt init in the project root
OpenAI API key
fal.ai API key
a portrait image URL
ffmpeg

1. Store each request as a row

Each row holds a script and the URL of a portrait. Everything after that is a computed column, so inserting a row starts the work. When you add a script, only that row runs.

import os

import fal_client
import pixeltable as pxt
import pixeltable.functions as pxtf
from pixeltable.functions import fal, openai
from pixeltable.serving import FastAPIRouter

TableModel = pxt.model_base()
embed = openai.embeddings.using(model='text-embedding-3-small')

@pxt.udf
def hosted_url(audio: pxt.Audio) -> str:
    # The avatar model fetches its inputs by URL, so put the voice track on fal's storage.
    return fal_client.SyncClient(key=os.environ['FAL_API_KEY']).upload_file(audio)

class Clips(TableModel, name='clips'):
    id = pxt.Column(value=pxtf.uuid.uuid7(), primary_key=True)
    script: pxt.String
    portrait_url: pxt.String

2. Voice the script

Send the script to a text-to-speech model and store the audio file. Pick a voice per brand or per presenter and keep it in the column definition, so every clip sounds the same.

voice = openai.speech(script, model='tts-1', voice='alloy')

3. Render the talking head

Lip-sync models read their inputs by URL. Upload the voice track to the provider storage, then call the model with the portrait URL and the audio URL. This example uses VEED Fabric 1.0 on fal, which takes image_url, audio_url, and resolution (720p or 480p) and returns video.url. Other image-plus-audio models on fal take similar inputs; change the app name and the input keys.

voice_url = hosted_url(voice)
    render = fal.run(
        {'image_url': portrait_url, 'audio_url': voice_url, 'resolution': '480p'},
        app='veed/fabric-1.0',
    )
    video = render['video']['url'].astype(pxt.Video)

4. Check what the clip says

Pull the audio track out of the rendered video and transcribe it. Compare the transcript with the script to catch truncated or garbled renders before anyone publishes them.

clip_audio = pxtf.video.extract_audio(video, format='mp3')
    heard = openai.transcriptions(clip_audio, model='whisper-1')
    transcript = heard['text'].astype(pxt.String | None)

5. Make the clips searchable and serve them

Add an embedding index on the transcript, and a query route that returns the closest clips for a phrase. POST a script and portrait URL to /clips. Search with /clips/search. A failed render stops the insert and returns the error. From Python, insert(..., on_error='ignore') keeps the row, and pxt recompute avatar_app/clips render --errors-only retries only the failed renders.

__indexes__ = [pxt.EmbeddingIndex(transcript, embedding=embed)]

@pxt.query
def find_clips(query: str):
    sim = Clips.transcript.similarity(string=query)
    return Clips.order_by(sim, asc=False).limit(5).select(Clips.id, Clips.script, Clips.video, Clips.transcript)

api = FastAPIRouter(name='avatar')
api.add_insert_route(Clips, path='/clips', inputs=[Clips.script, Clips.portrait_url], outputs=[Clips.id, Clips.video, Clips.transcript])
api.add_query_route(path='/clips/search', query=find_clips)

# pxt init
# pxt schema update app.py avatar_app -f
# pxt service update app.py avatar_app -f

Batch API or live session?

Batch API or live session?
	Batch render (this guide)	Real-time session
Output	An MP4 you store and reuse	A live WebRTC stream
Typical use	Course clips, product updates, personalized video messages	Support agents, tutors, interview practice
Example providers	VEED Fabric and other lip-sync models on fal, HeyGen video API, Hedra Character-3	LiveAvatar, Tavus, Simli, Anam, D-ID agents
Billing	Per second of rendered video	Per minute of session

Where Pixeltable fits

Pixeltable does not generate faces. It runs the pipeline around the model you choose: the script and portrait go in, the voice, render, transcript, and index are computed columns, and each new row runs only its own steps. Swapping the lip-sync model is a change to one column, and pxt recompute re-renders the rows you choose. A failed render stops the insert and returns the error. From Python, insert(..., on_error='ignore') keeps the row, and pxt recompute avatar_app/clips render --errors-only retries only the failed renders.

import os

import fal_client
import pixeltable as pxt
import pixeltable.functions as pxtf
from pixeltable.functions import fal, openai
from pixeltable.serving import FastAPIRouter

TableModel = pxt.model_base()
embed = openai.embeddings.using(model='text-embedding-3-small')


@pxt.udf
def hosted_url(audio: pxt.Audio) -> str:
    # The avatar model fetches its inputs by URL, so put the voice track on fal's storage.
    return fal_client.SyncClient(key=os.environ['FAL_API_KEY']).upload_file(audio)


class Clips(TableModel, name='clips'):
    id = pxt.Column(value=pxtf.uuid.uuid7(), primary_key=True)
    script: pxt.String
    portrait_url: pxt.String

    # 1. Voice the script.
    voice = openai.speech(script, model='tts-1', voice='alloy')
    voice_url = hosted_url(voice)

    # 2. Render a talking-head clip from the portrait and the voice (VEED Fabric 1.0 on fal).
    render = fal.run(
        {'image_url': portrait_url, 'audio_url': voice_url, 'resolution': '480p'},
        app='veed/fabric-1.0',
    )
    video = render['video']['url'].astype(pxt.Video)

    # 3. Check what the clip actually says, and make it searchable.
    clip_audio = pxtf.video.extract_audio(video, format='mp3')
    heard = openai.transcriptions(clip_audio, model='whisper-1')
    transcript = heard['text'].astype(pxt.String | None)

    __indexes__ = [pxt.EmbeddingIndex(transcript, embedding=embed)]


@pxt.query
def find_clips(query: str):
    sim = Clips.transcript.similarity(string=query)
    return Clips.order_by(sim, asc=False).limit(5).select(Clips.id, Clips.script, Clips.video, Clips.transcript)


api = FastAPIRouter(name='avatar')
api.add_insert_route(Clips, path='/clips', inputs=[Clips.script, Clips.portrait_url], outputs=[Clips.id, Clips.video, Clips.transcript])
api.add_query_route(path='/clips/search', query=find_clips)

# pip install -U 'pixeltable[serve]' fal-client openai
# export OPENAI_API_KEY=... FAL_API_KEY=...
# pxt init
# pxt schema update app.py avatar_app -f
# pxt service update app.py avatar_app -f

Questions

Can I use HeyGen or Hedra instead of fal?

Yes. Write a @pxt.udf that calls their API and returns the video URL or a local file path, and use it in place of fal.run. The voice, transcript, and index columns stay the same. HeyGen video API pricing moved to pay-as-you-go credits in late October 2026; check the current HeyGen API pricing page rather than an old subscription table.

Can I use an open-source lip-sync model?

Yes, if it runs behind an HTTP endpoint you can call from a UDF, whether you host it or use a hosted model page.

What does a render cost?

The lip-sync provider bills per second of output video; check the model pricing page. Text-to-speech and transcription are billed by OpenAI.

How do I stop a bad render from shipping?

Compare transcript with script in another computed column (for example a word error rate UDF) and filter on it before publishing.

Next step

Copy the commands into an agent, or run them locally with pip install pixeltable.

Get started Quick Start