@aagam-bothara3
AI inference, UIC
53m ago · AI Engineering, Agents, Infra, Models · Chicago

Auto Tuner for Local LLM's- is your local llm really running at its best?

Well, I noticed a problem: there is no fixed setting for GPUs and models to achieve maximum throughput, as it can be optimized depending on the specific GPU model and engine. So, I tried to build this auto-tuner. Honestly, this is just the start of the main idea I have, and I will keep posting updates :)

Please try it on your machine and let me know how it works for you.
Thanks!

Github: github.com/Aagam-Bothara/rigtune

LLMInference

Comments

No comments yet
Start the conversation.
Log in to join the thread.