A field guide

Zero → Hero:
your own AI cloud
on one box.

No subscriptions. No sending your data anywhere. Ten steps to a full local AI stack — chat, images, agents, code, search and scraping — all running on the machine under your desk.

01

The iron

Get a box with a giant shared memory pool.

The whole game is unified GPU memory: 128 GB or bigger, on a current NVIDIA chip with the new 4-bit floating point (NVFP4). More memory means bigger models you can actually load — not faster clocks. This is my starting point:

02

The keys

Make a Hugging Face account. Grab a token.

This is where the models live. Create a free account, generate a read access token, and accept the licenses for any gated models. Drop the token in your environment — every tool in the stack uses it to pull weights.

03

The brain

Serve a model with vLLM — sized to your box.

vLLM turns your GPU into an OpenAI-compatible endpoint. Scope the model to what fits — not every model will, and that's fine. I run a ~120B Qwen at 60% of the memory, which leaves real headroom for the image side.

04

The eyes

Stand up ComfyUI for images.

The 35% you held back runs visual workflows. Download the models for two graphs: one image generator and one image editor. Wire each up as a ComfyUI workflow you can call on demand.

05

The face

Put a face on it: Open WebUI.

Your chat interface and the front door to everything. Point it at vLLM for text and at ComfyUI for images, and you've got one clean window into the whole brain.

06

The hands

Code with it from VS Code using Cline.

Cline is an agentic coder that lives inside VS Code. Point it at your local vLLM endpoint and it writes, runs and fixes code against your own model — agentic development with no API meter running.

07

The sandbox

Give it a place to run code: Jupyter.

A Jupyter container is the safe sandbox where agents actually execute code — isolated, disposable and reproducible. When the model wants to run something, it runs here, not on your host.

08

The memory

Add retrieval with a FastAPI RAG service.

A small FastAPI service indexes your documents and feeds the relevant chunks back to the model. This is how the stack answers from your files — not just what the model was trained on.

09

The lookout

Search the live web — privately — with SearXNG.

SearXNG is a self-hosted metasearch engine. Your stack queries the live web through it, so you get fresh results without handing your searches to anyone else.

10

The scout

Scrape and browse with Playwright in Docker.

A headless Playwright container drives a real browser, so the stack can open pages, click through and pull the content SearXNG only points at. Search finds it; Playwright reads it.

The whole stack

One machine. Many ports.

Ship it

Hand someone the whole thing.

It all lives in one docker compose. Clone it, set your HF token, docker compose up. Skeleton below — tune the image tags for arm64 + the NVIDIA runtime on DGX OS.

vllm:
  image: vllm/vllm-openai   # arm64 + CUDA
  command: --model Qwen/…120B
           --gpu-memory-utilization 0.60
  ports: ["8000:8000"]

comfyui:   ports: ["8188:8188"]
open-webui: ports: ["3000:8080"]
jupyter:   ports: ["8888:8888"]
rag:       ports: ["8002:8002"]  # FastAPI
searxng:   ports: ["8080:8080"]
playwright:ports: ["3001:3001"]
# cline runs in VS Code → vllm:8000

Zero → Hero. Built on a Dell GB10.
Joshua Talbott · joshuaallentalbott.com