Edge0

Run large mixture-of-experts models on a Mac by streaming experts from SSD

GHSeen on GitHub

It lets a 35B-parameter MoE model run locally on Apple Silicon without holding every expert in memory.

Edge0 is an open-source inference framework for mixture-of-experts language models on Apple Silicon. Expert weights stream from the SSD on demand, so peak memory stays bounded; a prerouter predicts which experts come next, which the team says lifts decode speed by up to 59%. Trained LoRA adapters recover quality lost to 4-bit quantisation. It ships two ready models, Edge0-35B (about 23GB) and Edge0-8B (about 4.2GB), on Hugging Face, and serves an OpenAI-compatible API.

More like this

Jinfer

An AI inference engine that runs inside the JVM, with no Python sidecar

MAPLE

Gemma 4 26B cut to 48 experts per layer so it fits a 6 GB laptop GPU

Strata

Runs a 125B mixture-of-experts model on a single gaming PC, fully offline

InstinctFlash

A serving runtime that speeds robot policy models up for real control loops

PicoLM

A C99 LLM engine that streams weights from disk, from Tru64 to Raspberry Pi

Warp

Run trillion-parameter models on a laptop by streaming weights from NVMe