← Aagam Sachin Bothara's projects

Measures which LLM serving settings pay off on your GPU and your traffic, then serves the winner. Benchmarks vLLM, SGLang and llama.cpp configurations; one OpenAI-compatible AP

Built withPythonsglanggpuinfeinferencequnatization

Comments · 0

No comments yet. Tried it? Tell Aagam what worked, what didn't, or what you'd want next.
Sign in to comment

More from members

All projects

See what other members are building and what each project is looking for.