fast-litellm Rust + Python Drop-in Rust acceleration for LiteLLM. Faster connection pooling, rate limiting, and dramatically lower memory use.
A PyO3 acceleration layer for LiteLLM. Replaces the hottest Python paths — HTTP connection pooling, rate limiting, response normalisation — with thread-safe Rust. No API changes; one import.
Install
$ pip install fast-litellmFrequently asked questions
What is fast-litellm?
fast-litellm is Drop-in Rust acceleration for LiteLLM. Faster connection pooling, rate limiting, and dramatically lower memory use. It is part of the Neul Labs open-source portfolio (36 MIT-licensed projects for AI agent infrastructure). A PyO3 acceleration layer for LiteLLM. Replaces the hottest Python paths — HTTP connection pooling, rate limiting, response normalisation — with thread-safe Rust. No API changes; one import.
How do I install fast-litellm?
Install with: `pip install fast-litellm`. fast-litellm is MIT licensed and ships prebuilt binaries where applicable.
What language is fast-litellm written in?
fast-litellm is written in Rust + Python.
Is fast-litellm open source?
Yes. fast-litellm is MIT licensed and publicly available at https://github.com/neul-labs/fast-litellm. All 36 Neul Labs projects are MIT licensed.
How does fast-litellm compare to alternatives?
See the detailed comparison at https://www.neullabs.com/compare/fast-litellm-vs-litellm.