Summary: LectureSync is the student loop: a lecture video and a slide deck become searchable pages and sampled frames, then an answer cites the slide and the transcript. You do not stitch a PDF splitter, a frame farm, and two vector databases. You declare Lectures, a Slides view, and a Frames view. Retrieve first. Generate second. pip install pixeltable.
The Product#
- Ingest
lecture_video+slides_pdf - One row per slide page (text + page image)
- One row per sampled second of video
- Ask “What does backpropagation compute?” and land on a page plus nearby speech
Passage RAG on a prospectus is DocuVision. Visual moments in a clip library are ClipFinder. This post is both media types on one lecture.
The Stitch You Delete#
PyMuPDF for pages. FFmpeg for JPEGs. Whisper in a GPU box. Chroma for slide text. CLIP in Pinecone for frames. A notebook that hopes the timestamps line up. Delete the PDF; the chunks in Chroma are your problem. In a catalog, delete the lecture; both views follow.
The Receipt#
Retrieval is the index. Generation is one Gemini call over those strings — not a shipped slide-aligner. Frames is the visual audit trail at 1 fps, same iterator as ClipFinder. Same app.py on Cloud: pxt db update → pxt schema update → pxt service update.
What This Is Not#
Not an LMS. Not Zoom. Not “we forced every slide to a timestamp with a private model.” You retrieve the page, you retrieve nearby speech, you generate. Auth and grades stay where they are.
People Also Ask#
Why separators='page'? Decks are pages, not 300-token passages. DocuVision uses token_limit for prospectuses. Change the iterator kwargs; do not change the table.
Do I need two CLIP indexes? Slide images and video frames are different rows. Each view gets its own index. Do not mix embedding= and image_embed= on one snippet.
How do I go to Cloud? Same file. PIXELTABLE_API_KEY, pxt://org:db, then db → schema → service.
Ground the Answer in the Deck#
Declare the lecture. Split the PDF. Sample the video. Retrieve, then generate. Keep the citation. Delete the aligner service.
- Pixeltable on GitHub:
pip install pixeltable - ReelForge — podcast chapters
- MedDossier — clinical intake row
- AdRadar — creative fatigue
- DocuVision — PDF Q&A
- ClipFinder — NL video moments
- Pixeltable documentation
