Abhinav Raj wrote a good piece on XDA this morning: My self-hosted AI models research, code, and generate images, and all they cost is a GPU I already own. He runs one job for research, one for code and one for images, all on a single 16GB card he already owned. I like the setup. This post runs the same three jobs from one Pixeltable app file, so the research notes, the generated code and the images land in tables you can search, version and re-run instead of in three different UIs.
Everything below ran on the current PyPI release (0.7.20), but not on Abhinav's hardware. I used a 0.5B stand-in model behind llama-server, on CPU, and a stub ComfyUI that returns a fixed PNG. The wiring, tool calls, indexes and outputs are real. The preset values, the model-swap behavior and the Klein workflow's node ids are not. The model quality obviously isn't Abhinav's.
What the XDA setup runs#
Abhinav's machine is a 16GB RTX 4070 Ti Super with a Ryzen 5 7600X. Per his article, the three jobs map to three lead models:
| Job | Model | How he runs it | Notes from the article |
|---|---|---|---|
| Deep research | BigBang-v1, a fine-tune of Qwen3.6-35B-A3B (Apache 2.0), bartowski's IQ4_XS GGUF | llama.cpp, with --n-cpu-moe moving about 7GB of expert weights to system RAM, 128K context | llama.cpp's web UI connected to Exa search over MCP with a free API key. He reports 68 to 78 tokens per second. |
| Coding | Qwen3.8-27B (dense), Unsloth's UD-IQ4_XS quant (14.3GB) | llama.cpp, 32K context, 6 of 64 layers end up in system RAM | He reports 15 to 17 tokens per second and uses it for Pygame projects. Its SWE-bench Pro score is 61.7 against 54.2 on BigBang-v1's card, both self-reported. |
| Image generation and editing | FLUX 3 Image Klein, distilled (5.6B), 16 steps | ComfyUI, with Qwen3.5 9B as prompt upsampler and Qwen3-VL 4B as text encoder, INT8 builds of all three | About 16 seconds per 1024x1024 image in his tests. FLUX Kommunity License, not Apache 2.0. |
The benchmark numbers he quotes (BrowseComp, SWE-bench Pro) are self-reported by the model makers, and he says so. He also says BigBang-v1 added a KB number and fix to one answer that he couldn't trace back to Microsoft, so he still checks its sources. Keep that in mind; nothing below fixes it.
Why wire it into tables#
A chat UI is the right tool for one question. It gets awkward on the twentieth: the answers sit in browser history, the code is in a scratch folder, the images are in ComfyUI's output/ directory, and when a better model shows up next month you can't re-run last month's prompts against it without pasting them back in by hand.
In Pixeltable each job is a table. You insert a question, a spec or a prompt, and computed columns call the local model and store the result next to the input. That buys you a few concrete things:
- Every output is stored with the exact expression that produced it, and every insert or schema change is a new table version.
- Change a prompt or swap a model, then
pxt recomputere-runs that column for the old rows, and the columns that depend on it follow. - New rows only compute new rows. Re-running the file doesn't call the models again for work already done.
- An embedding index on the answers and one on the images lets you search everything you generated, by text.
I'd keep the llama.cpp web UI for quick chats. The tables pay off once the same kind of request repeats.
One llama.cpp server for both language models#
Pixeltable's OpenAI integration reads OPENAI_BASE_URL, so it talks to any OpenAI-compatible server (configuration docs). llama-server is one. Recent llama.cpp builds also have a router mode: start the server without a model, give it a preset file, and the model field of each request picks which GGUF to load. With --models-max 1 only one is loaded at a time, which is what you want on 16GB.
The per-model flags from the article go in the preset. Paths are yours, and the n-cpu-moe value is a starting point to tune until BigBang-v1 fits your VRAM; the article gives the size it moved (about 7GB), not the flag value.
Set OPENAI_READ_TIMEOUT before Python starts. Pixeltable builds the OpenAI client once per process and keeps the timeout it saw. The coder asks for up to 4096 tokens. At the 15 tokens per second Abhinav reports, that reply takes about four and a half minutes, so the 120 second default aborts it.
If you'd rather run a small model inside the Python process, pixeltable.functions.llama_cpp.create_chat_completion does that (llama.cpp guide). For these two models I'd use the server: the in-process function offloads every layer to the GPU and doesn't expose --n-cpu-moe or the context size, and the article depends on both.
Install the Python side and create the project:
Research: BigBang-v1 plus Exa search over MCP#
Abhinav connected Exa to the llama.cpp web UI over MCP. Pixeltable can load an MCP server's tools as functions with pxt.mcp_udfs(url), then hand them to the model with pxt.tools(...) and run whatever the model asks for with openai.invoke_tools (tool calling and MCP guide). Exa's hosted server at https://mcp.exa.ai/mcp answers without a key, rate-limited.
The research table is three computed columns: the model plans a search, the tools run, and a second call answers from what came back.
with_sources is a five-line UDF that puts the question and the search results into the second prompt. It lives in local_tools.py with the other helpers, shown in full in the image section.
The system prompt line about objective is there for a reason. Exa's web_search_exa tool requires both query and objective. When my test model left objective out, the call returned nothing and the model answered from memory. Look at the results column when an answer seems off; it's stored right there.
Here is what the CPU run stored for "What is the latest stable release of the Python programming language? Use web search." The plan column holds the tool call the model made:
results holds Exa's response, starting with python.org's release page for 3.15.1, and answer says 3.15.1. On an earlier run, the same 0.5B model got the version right and then made up a commit count to go with it. Abhinav's advice to check the sources applies at every model size.
One limit to know: this is one round of tool calls per row. His Windows question took five searches in an interactive loop. If you need more rounds, add another plan and results pair, or see the agentic patterns page.
Code: the dense Qwen model, with a syntax check#
The coding table sends the spec to the coder preset, pulls the Python block out of the reply and checks that it parses. ok is a cheap filter. It won't tell you the game is fun.
The request uses model='coder', so router mode should unload BigBang-v1 and load Qwen3.8-27B. I pointed both preset names at one small model, so I did not see that swap. Insert a batch of specs in one call. If the router swaps, it swaps once for the batch, not once per spec.
Images: FLUX 3 Image Klein in ComfyUI, through a small UDF#
Pixeltable has no ComfyUI integration. ComfyUI does have an HTTP API, so the pattern is a UDF: export your Klein workflow with Workflow > Export (API), and the UDF fills in the prompt and seed, queues the job, waits for it, and returns the image. The prompt upsampler and the text encoder stay inside the workflow, the way Abhinav runs them.
Nodes 6 and 3 are placeholders. The stub server never checked them. Open your exported JSON and use the prompt and sampler ids from that file.
The table is short. The CLIP index takes both images and text, so you can search the images with a sentence later.
The seed is stored, and the workflow is not. comfyui() rereads klein_t2i_api.json on every call, so the same seed regenerates the same image only while that file and the weights stay put. After you change the workflow, pxt recompute stores the new images. Klein also edits from the same weights. An edit workflow works the same way: upload the source through ComfyUI's /upload/image route and set its LoadImage node. I left that out to keep the UDF short.
If your image model runs in Hugging Face diffusers, huggingface.text_to_image and huggingface.image_to_image run it in-process with no ComfyUI. I didn't check whether Klein loads that way, so for this stack I'd stay with ComfyUI.
Create the tables and use them#
Search works across everything you generated, by meaning:
All three jobs in one row#
Separate tables are fine. Since these are the same three jobs working on one idea, I'd rather chain them: you insert an idea, the research model reads up on it, the coder writes a first version from those notes, and ComfyUI draws a title screen.
Add that to app.py, then apply the schema again. The first pxt schema update ran before Projects existed, so it did not create this table. In a new Python session:
Pixeltable runs the columns in dependency order, so reply waits for notes. You don't write that orchestration. I ran a longer chain on CPU, with the 0.5B stand-in and the stub ComfyUI, and it stored ok True and a 256x256 PNG. That run also asked the researcher for an image prompt, which loads the language model again before ComfyUI. The class above skips that call, and I have not re-run this shorter one.
Changing a prompt next month#
Say a better coding model shows up, or you want a lower temperature. Edit the class, apply it, then re-run the column for the rows you already have:
schema update names the columns whose expressions changed and leaves the stored values in place. It also prints the pxt recompute line to run. recompute asks before it runs (add -f in scripts), then re-runs reply and cascades to code and ok. pxt history gpu/projects prints one tab-separated row per version, with how many rows that version inserted or updated. The old outputs stay in the earlier versions (version control docs), so you can compare the new coder against the old one on the same specs.
What it costs, and the rough edges#
The bill is the same as Abhinav's: the models are free to download, the GPU is the one you have, and you pay for electricity. Pixeltable is open source and runs on the same machine. Exa's hosted MCP server has a keyless tier with rate limits. Abhinav used a free Exa API key; if you pass one in the MCP URL, know that Pixeltable stores the URL with the column definition, so I'd stay keyless for a personal setup.
Some trade-offs I'd rather say up front:
- VRAM is shared.
--models-max 1keeps a single llama.cpp model loaded, and it leaves that model in memory when the batch ends. ComfyUI has its own process and does not get that memory back. On a 16GB card, stopllama-serverbefore an image batch and start it again after. I have not measured that handoff on a 4070 Ti Super. - One tool round per row, as mentioned in the research section.
- The ComfyUI UDF is generic code, not a supported integration. If ComfyUI changes its API, you fix twenty lines.
- If you only ever ask one question at a time, the llama.cpp web UI is simpler, and I'd just use it.
Try it#
Start with the Quickstart, then the tool calling and MCP guide and the llama.cpp guide. We have older posts on Ollama, llama.cpp and vLLM if you run a different local server. And thanks to Abhinav for the write-up; his article has the model-by-model details and is worth reading first.
