Run Qwen 3.8 Flash Next (125B) on consumer hardware (RTX 4090) at 100T/s

Strata is an MIT-licensed engine by Niko Veit. It keeps the model's experts in system RAM, caches the busiest ones on the GPU, and puts the n-gram table on the SSD. The README reports 53 to 94 tokens per second on a 12 GB RTX 5070, depending on quantization. The 100 figure is closer to a community number: one HN user measured 124 tokens per second on a 4090 with 128 GB of DDR5. All of the thread's speed reports are self-reported and unverified. The approach works because of the model's design. Qwen released Flash-Next on August 26, 2026. It has 125B main parameters, 6B active per token, a 51B n-gram table and a 4B multi-token-prediction head. A separate benchmark that capped the GPU at about 12 GiB found Strata decoding 62.9 tokens per second against 23.4 for llama.cpp. For founders, this lowers the cost of running a competitive open model for local agents, prototyping and privacy-sensitive workloads. It does not remove the hardware bill: system RAM and bandwidth become the constraint. Treat the "Coder" variant with care. It drops half the experts, and the 91% SWE-bench Verified figure is the authors' own measurement.