colibri
an experiment in running a frontier-scale mixture-of-experts model on hardware i already own — no cloud, no datacenter. most of a 744B model's parameters sit idle on any given token, so instead of loading everything, colibri keeps ~10 GB resident and streams the experts it needs in from disk on demand.
how it works
- mixture-of-experts — only the active experts are ever in memory
- disk streaming — page experts in on demand, ~10 GB resident footprint
- commodity target — runs on a $500 second-hand rig, not an accelerator
hardware
Ryzen 7 5700G · EVGA GTX 1080 8GB (2016) · 48GB RAM — the same machine i still run today.
the honest part
it runs ~1–2 tokens/sec. not fast. but it runs, on a decade-old card, and that was the whole point: not speed, but proof the approach is possible on what you've got. HBM and modern GPUs are priced out of reach — i'd rather squeeze the hardware i have.