fast-litellm
Rust + Python Drop-in Rust acceleration for LiteLLM. Faster connection pooling, rate limiting, and dramatically lower memory use.
A PyO3 acceleration layer for LiteLLM. Replaces the hottest Python paths — HTTP connection pooling, rate limiting, response normalisation — with thread-safe Rust. No API changes; one import.
Install
$ pip install fast-litellmFrequently asked questions
What is fast-litellm?
fast-litellm is an entry in the Neul Labs project catalogue. Drop-in Rust acceleration for LiteLLM. Faster connection pooling, rate limiting, and dramatically lower memory use. A PyO3 acceleration layer for LiteLLM. Replaces the hottest Python paths — HTTP connection pooling, rate limiting, response normalisation — with thread-safe Rust. No API changes; one import. Current source, licence, release and documentation status should be verified at the linked project surfaces.
How do I install fast-litellm?
The catalogue records this install command: `pip install fast-litellm`. Verify the linked repository, package, supported version and current instructions before use.
What language is fast-litellm written in?
fast-litellm is written in Rust + Python.
Is fast-litellm open source?
Source is linked at https://github.com/neul-labs/fast-litellm. Inspect the repository licence and release metadata for the exact version; Neul Labs does not apply one blanket licence claim to the whole catalogue.
How does fast-litellm compare to alternatives?
See the detailed comparison at https://www.neullabs.com/compare/fast-litellm-vs-litellm.