Phobos
A Rust kernel language that compiles its own GPU code and serves LLMs with it
Emerged finds new products worth a look, daily — what’s new today
PicoLM
A C99 LLM engine that streams weights from disk, from Tru64 to Raspberry Pi
It shows how far down the hardware ladder local inference can go, which matters if you ship to old, odd or memory-starved machines.
PicoLM is an inference engine written in plain C99 that streams model layers from disk instead of loading them, so it runs in roughly 45MB of RAM. It reads GGUF files for Llama 2, Mistral Nemo, GPT-2, Gemma-3n and Qwen, with SIMD on CPU plus CUDA and HIP, and a Vulkan backend in progress. It builds on Linux, Windows, macOS back to 10.4, FreeBSD, Android, RISC-V, PowerPC, OSF/1 Tru64, the first iPhone and MS-DOS. By Gabucino, formerly of MPlayer. MIT, 896 commits.
More like this
Edge0
Run large mixture-of-experts models on a Mac by streaming experts from SSD
bootgpt
512 bytes of assembly that boot a PC straight into a neural net
Warp
Run trillion-parameter models on a laptop by streaming weights from NVMe
jdBasic
A modern BASIC where you pause a running program, change the code and carry on
Strata
Runs a 125B mixture-of-experts model on a single gaming PC, fully offline