A language model,
running in this tab.
The model is downloaded once, cached by your browser, and executed on your GPU through WebGPU. There is no API key, no account and no metered token — hosting this on a server per visitor was the cost I did not want to pay, so the model ships to the reader instead.
The cheapest inference is the one you do not serve.
A chat demo hosted on a GPU is a bill that scales with strangers. Shipping a 0.5B model to the visitor's GPU costs one cached download and nothing afterwards — and it makes a privacy claim that is structurally true rather than a promise in a policy page.
The trade is real: ~300 MB before the first token, a WebGPU requirement, and a 0.5B model that is fast and not clever. The 1.5B swap is in the selector when your GPU can carry it.
- Runtime
- WebLLM 0.2.82
- Backend
- WebGPU
- Models
- Qwen2.5 · SmolLM2
- Server calls
- weights only
- Code
- GitHub
If the status line reads “Not supported”, your browser has no WebGPU: the demo is honest about it rather than silently degrading to a worse path.