Edge0
Run large mixture-of-experts models on a Mac by streaming experts from SSD
Emerged finds new products worth a look, daily — what’s new today
If you only need a local model for Python and STEM work, cutting the experts you never use can buy a sixfold speed-up on hardware you already own.
MAPLE profiles which experts Gemma 4 26B-A4B actually activates on coding and STEM prompts, keeps 48 of 128 per layer, and ships the result as a 4.75 GB GGUF that loads fully into 6 GB of VRAM. On an RTX 4050 laptop the maker measured 45 tokens a second against 6.9 for the offloaded original. The pipeline scripts are included, so you can prune for your own domain. The README is candid: four self-run tasks, one GPU, and general chat and multilingual skill are given up on purpose.
More like this
Strata
Runs a 125B mixture-of-experts model on a single gaming PC, fully offline
Laya
Small encoder models that answer routing and triage questions in one pass
qwen38-27b-rtx3090
A tuned vLLM stack that serves Qwen3.8-27B fast on one RTX 3090
Warp
Run trillion-parameter models on a laptop by streaming weights from NVMe
Flide
Exact, shared-context inference server for small Jev-style decision models