drumih/turbo-fieldfare
A Swift + Metal runtime running a 26B model on any Apple Silicon Mac
Run a 26B model on the 8GB Mac you already own.
Clone it. Build it. It's running in minutes.
Clone it, run one build command, launch the app. First run downloads the model — about fifteen gigabytes — then you're generating text locally, no account needed.
The model is 14GB. It only loads 2.
The full model is fourteen gigabytes on disk. But it keeps only a small shared core in memory, then streams in just the expert weights each token needs — so RAM stays around two gigabytes.
One decision made it 8 times faster.
The obvious approach — just mapping the file — ran at half a token a second. Switching to explicit parallel reads pushed that to almost four. One decision, eight times the speed.
3.88 billion active. 26 billion total.
Twenty six billion parameters total, but it's a mixture of experts — for any single token, only about three point nine billion actually fire. That's the second trick keeping this fast.
One engineer. Andrey Mikhaylov. Free to run.
This isn't a company, it's one engineer — Andrey Mikhaylov, a Metal and iOS engineer by trade. Free, Apache licensed. Clone it this weekend.