Model

The VQ(Vector Quantization) models for single DGX-Spark

@baekpicaby Sunghoon Baek · Independent

0

Extreme Vector Quantization (VQ) for running massive open-weight MoE models on a single 128GB NVIDIA DGX Spark. From compressing a 501B model from ~1TB to ~100GB, to DeepSeek V4.1 Flash hVQ achieving up to 1,157 tok/s prefill and 32–36 tok/s decode on GB10. Exploring the limits of model compression, quality preservation, and efficient local inference with ds4-dfm-rs.

See what other members are building and what each project is looking for.