@aagam-bothara3
AI inference, UIC
57m ago · New project · Agents · Chicago

Shipped: Rigtune

Well I saw this problem that there is no fixed setting for gpu and models to work as maxium throughput and it can be optimzied depending on gpu models and engines so I tried to build this auto tuner honestly this is just the start of main Idea I have will keep posting updates :),Please try it on your machine and do let me know how it works for you.

Thanks.
give me copy paste verision by fixing grammer

Well, I noticed a problem: there is no fixed setting for GPUs and models to achieve maximum throughput, as it can be optimized depending on the specific GPU model and engine. So, I tried to build this auto-tuner. Honestly, this is just the start of the main idea I have, and I will keep posting updates :) Please try it on your machine and let me know how it works for you.

Thanks!

Rigtune logoRigtune@aagam-bothara/auto-tuner-for-gpu · github.com/Aagam-Bothara/rigtune
LLM

Comments

No comments yet
Start the conversation.
Log in to join the thread.