$git clone https://github.com/algorithmicsuperintelligence/optillm
Installs into the current project.
Summary
Optimizing inference proxy for LLMs