Edge0
Run large mixture-of-experts models on a Mac by streaming experts from SSD
Emerged finds new products worth a look, daily — what’s new today
If you want a frontier-sized open model on your own desk without renting GPUs, this puts one behind a local API your existing tools already speak.
Strata is an open-source inference engine that splits a large mixture-of-experts model across GPU, RAM and SSD: hot experts stay in VRAM, the rest wait in memory, and a small draft model speeds decoding by roughly 1.6-1.8x. The README reports 60-95 tokens per second on an RTX 5070. It needs an RTX 20-series or newer card, 32-64 GB of RAM and about 80 GB of disk, and the first load can freeze the machine for a few minutes. MIT licensed; the model weights carry Qwen's own licence.
More like this
Strata
Shared memory for Claude Code and Codex sessions, with an LLM judge at the door
InstinctFlash
A serving runtime that speeds robot policy models up for real control loops
MAPLE
Gemma 4 26B cut to 48 experts per layer so it fits a 6 GB laptop GPU
Phobos
A Rust kernel language that compiles its own GPU code and serves LLMs with it
PicoLM
A C99 LLM engine that streams weights from disk, from Tru64 to Raspberry Pi