The Complete LiteLLM Guide (2026)

Have you run into any of these? Your project runs on OpenAI, and one day you want to switch to Claude — only to find every SDK call needs rewriting. Or you're using three different providers at once, API keys scattered everywhere, and your bill is a black box you can't map back to any single service. Or your production system starts hitting rate limits on one model constantly, and you end up writing a pile of boilerplate retry logic by hand.

LiteLLM exists to solve exactly these problems. 53k stars on GitHub, currently at v1.91.0 (2026-07-04), supporting 100+ LLM providers, 8ms P95 latency, and throughput up to 1.5k+ RPS. It ships in two flavors: the Python SDK (embedded directly in your code) and the Proxy Server (a self-hosted AI gateway). You can use either on its own, or combine them.

This is the most complete LiteLLM guide available, covering: SDK basics, cross-model switching, Router strategies, standing up and configuring a Proxy, Claude Code integration, cost tracking, connecting relay providers, and production deployment. It's a long read — bookmark it and work through it in sections.

1. What LiteLLM Is, and What Problem It Solves

LiteLLM's core value can be summed up in one line: call any LLM with the same code, without learning each provider's own SDK format.

100+ providers each use their own API format — OpenAI has one, Anthropic has another, Google Vertex has its own, and AWS Bedrock is different again. LiteLLM wraps all of them in a single OpenAI-compatible interface: change one model parameter, and the request routes to whichever provider you specify.

Beyond a unified interface, LiteLLM also provides:

• Smart routing: automatic load balancing, failover, and distributing requests by latency, cost, or weight
• Cost tracking: automatically computes token cost per call and rolls up billing across providers
• Observability: one-line integration with monitoring platforms like Langfuse, MLflow, and Helicone
• Proxy mode: a self-hosted, OpenAI-compatible server — any tool built on the OpenAI SDK (including Claude Code) can point at it with no code changes
• Virtual keys and budgets: issue separate virtual API keys to different team members, each with its own spending cap

Comparing the two usage modes:

Mode	Best for	Pros	Cons
Python SDK	Your own Python project, calling an LLM directly in code	Lightweight, no extra infrastructure, easy to debug	Python-only, can't be shared with other languages/tools
Proxy Server	A shared team gateway, or a backend for Claude Code / Cursor and similar tools	Language-agnostic, centralized key and cost management, production-grade HA	Requires running and maintaining an extra process/container

2. Installation

2.1 Installing the Python SDK

# uv is recommended (faster)
uv add litellm

# or pip
pip install litellm

2.2 Installing the Proxy Server

# Install the full version with Proxy support
uv tool install 'litellm[proxy]'

# or pip
pip install 'litellm[proxy]'

After installing, run litellm --version to confirm — the current latest release is v1.91.0.

3. Python SDK: The Basics

3.1 The simplest call

LiteLLM's completion() function has the exact same interface as OpenAI's chat.completions.create(), with one extra model prefix to specify the provider:

from litellm import completion
import os

# OpenAI
os.environ["OPENAI_API_KEY"] = "sk-..."
response = completion(
    model="openai/gpt-4o",
    messages=[{"role": "user", "content": "Hello, write a quick sort in Python"}]
)
print(response.choices[0].message.content)

Switching to Anthropic Claude only requires changing the model and the corresponding key:

os.environ["ANTHROPIC_API_KEY"] = "sk-ant-..."
response = completion(
    model="anthropic/claude-sonnet-5",   # note the anthropic/ prefix
    messages=[{"role": "user", "content": "Hello, write a quick sort in Python"}]
)
print(response.choices[0].message.content)

Switching to Google Gemini:

os.environ["GEMINI_API_KEY"] = "AIza..."
response = completion(
    model="gemini/gemini-3.5-flash",
    messages=[{"role": "user", "content": "Hello, write a quick sort in Python"}]
)
print(response.choices[0].message.content)

Everything except the model parameter stays exactly the same. That's LiteLLM's core value proposition: switching models doesn't require touching your business logic.

3.2 Model string format by provider

Provider	Example model string	Environment variable
OpenAI	openai/gpt-4o	OPENAI_API_KEY
Anthropic	anthropic/claude-sonnet-5	ANTHROPIC_API_KEY
Google Gemini	gemini/gemini-3.5-flash	GEMINI_API_KEY
AWS Bedrock	bedrock/anthropic.claude-3-sonnet	AWS_ACCESS_KEY_ID, etc.
Azure OpenAI	azure/<deployment-name>	AZURE_API_KEY, etc.
Ollama (local)	ollama/llama3.2	No key required
DeepSeek	deepseek/deepseek-chat	DEEPSEEK_API_KEY
OpenAI-compatible (relay)	openai/model-name + api_base	Custom

3.3 Streaming output

response = completion(
    model="openai/gpt-4o",
    messages=[{"role": "user", "content": "Write a short essay about AI"}],
    stream=True
)

for chunk in response:
    content = chunk.choices[0].delta.content
    if content:
        print(content, end='', flush=True)
print()  # newline

3.4 Async calls

import asyncio
from litellm import acompletion

async def main():
    response = await acompletion(
        model="anthropic/claude-sonnet-5",
        messages=[{"role": "user", "content": "Explain quantum entanglement"}]
    )
    print(response.choices[0].message.content)

asyncio.run(main())

3.5 Function calling (tool use)

import json
from litellm import completion

tools = [
    {
        "type": "function",
        "function": {
            "name": "get_weather",
            "description": "Get the weather for a given city",
            "parameters": {
                "type": "object",
                "properties": {
                    "city": {"type": "string", "description": "City name"},
                    "unit": {"type": "string", "enum": ["celsius", "fahrenheit"]}
                },
                "required": ["city"]
            }
        }
    }
]

response = completion(
    model="openai/gpt-4o",
    messages=[{"role": "user", "content": "What's the weather like in Beijing today?"}],
    tools=tools,
    tool_choice="auto"
)

# Parse the tool call
if response.choices[0].message.tool_calls:
    tool_call = response.choices[0].message.tool_calls[0]
    print(f"Calling tool: {tool_call.function.name}")
    print(f"Arguments: {json.loads(tool_call.function.arguments)}")

4. Python SDK: Advanced Techniques

4.1 Unified error handling

LiteLLM maps every provider's errors onto OpenAI's exception types, so you only need to handle one set of exceptions:

from litellm import completion
from litellm.exceptions import (
    AuthenticationError,
    RateLimitError,
    APIError,
    BadRequestError
)

try:
    response = completion(
        model="anthropic/claude-sonnet-5",
        messages=[{"role": "user", "content": "Hello"}]
    )
except AuthenticationError as e:
    print(f"Invalid API key: {e}")
except RateLimitError as e:
    print(f"Rate limited, retry later: {e}")
except BadRequestError as e:
    print(f"Bad request parameters: {e}")
except APIError as e:
    print(f"Server-side error: {e}")

4.2 Cost tracking (per call)

import litellm

response = completion(
    model="openai/gpt-4o",
    messages=[{"role": "user", "content": "Write a 100-word blurb"}]
)

# Get the cost of this call (USD)
cost = litellm.completion_cost(completion_response=response)
print(f"Call cost: ${cost:.6f}")
print(f"Input tokens: {response.usage.prompt_tokens}")
print(f"Output tokens: {response.usage.completion_tokens}")

4.3 Observability integration (Langfuse)

import litellm

# One line to wire up Langfuse — every LLM call gets logged automatically
litellm.success_callback = ["langfuse"]
os.environ["LANGFUSE_PUBLIC_KEY"] = "pk-..."
os.environ["LANGFUSE_SECRET_KEY"] = "sk-..."

response = completion(
    model="openai/gpt-4o",
    messages=[{"role": "user", "content": "Hello"}]
)
# This call is automatically logged to the Langfuse dashboard

Other supported monitoring platforms include MLflow, Helicone, Lunary, Arize, and Weights & Biases — configuration works the same way across all of them.

4.4 Connecting an OpenAI-compatible relay (custom api_base)

Relay providers (like SiliconFlow or AiHubMix) typically expose an OpenAI-compatible interface, and LiteLLM can connect to them directly:

response = completion(
    model="openai/deepseek-v4-flash",   # use whatever model name your relay supports
    api_base="https://api.siliconflow.cn/v1",  # the relay's endpoint
    api_key="sk-...",   # your API key from the relay
    messages=[{"role": "user", "content": "Hello"}]
)

5. Router: Load Balancing and Failover

When your system needs to call multiple deployments of the same model (say, three Azure GPT-4o deployments spread across different regions), or needs to automatically fail over to a backup model when the primary one fails, that's what litellm.Router is for.

5.1 Basic routing setup

from litellm import Router

model_list = [
    {
        "model_name": "gpt-4o",   # the unified name exposed externally
        "litellm_params": {
            "model": "openai/gpt-4o",
            "api_key": "sk-openai-primary",
            "weight": 7   # 70% of traffic
        }
    },
    {
        "model_name": "gpt-4o",
        "litellm_params": {
            "model": "azure/gpt-4o-eastus",
            "api_base": "https://my-eastus.openai.azure.com/",
            "api_key": "azure-key",
            "api_version": "2024-08-01-preview",
            "weight": 3   # 30% of traffic
        }
    }
]

router = Router(model_list=model_list)

# Called exactly the same way as litellm.completion()
response = router.completion(
    model="gpt-4o",   # use the unified name
    messages=[{"role": "user", "content": "Hello"}]
)

5.2 The seven routing strategies, explained

Strategy	Best for	Characteristics
simple-shuffle (default)	General-purpose production use	Weighted random by RPM/TPM or weight, lowest overhead — the recommended default
rate-limit-aware-v2	Multi-account rate-limit avoidance	Tracks per-deployment TPM in real time, filters out over-limit deployments, runs asynchronously
latency-based-routing	Latency-sensitive real-time systems	Dynamically caches each deployment's response time, picks the fastest
least-busy	Controlling concurrency, preventing overload on a single deployment	Picks the deployment with the fewest in-flight requests
usage-based-routing	Multi-instance distributed deployments	Routes to the lowest-TPM deployment; requires Redis
cost-based-routing	Cost-sensitive workloads	Picks the cheapest available deployment for each request
custom	Custom business logic	Subclass CustomRoutingStrategyBase to implement your own logic

5.3 Failover and retry policy

from litellm.router import RetryPolicy, AllowedFailsPolicy

retry_policy = RetryPolicy(
    RateLimitErrorRetries=3,        # retry rate-limit errors 3 times
    TimeoutErrorRetries=2,          # retry timeouts twice
    AuthenticationErrorRetries=0,   # never retry auth errors
    BadRequestErrorRetries=1,
    ContentPolicyViolationErrorRetries=3
)

allowed_fails_policy = AllowedFailsPolicy(
    RateLimitErrorAllowedFails=100,  # tolerate 100 rate-limit failures before cooling down
    ContentPolicyViolationErrorAllowedFails=1000
)

router = Router(
    model_list=model_list,
    retry_policy=retry_policy,
    allowed_fails_policy=allowed_fails_policy,
    allowed_fails=3,      # cool down a deployment after 3+ failures within 1 minute
    cooldown_time=30      # cooldown duration in seconds
)

5.4 Priority ordering (the order parameter)

model_list = [
    {
        "model_name": "claude-sonnet",
        "litellm_params": {
            "model": "anthropic/claude-sonnet-5",
            "api_key": "primary-key",
            "order": 1   # highest priority, tried first
        }
    },
    {
        "model_name": "claude-sonnet",
        "litellm_params": {
            "model": "anthropic/claude-sonnet-5",
            "api_base": "https://your-relay-address/v1",
            "api_key": "relay-key",
            "order": 2   # fallback: used if the primary route fails
        }
    }
]
router = Router(model_list=model_list)

6. Proxy Server: A Self-Hosted AI Gateway

LiteLLM Proxy is an independently deployable HTTP server that exposes a fully OpenAI-compatible interface. Everyone on your team, code in any language, and any AI tool (Claude Code, Cursor, Open WebUI, and so on) can just point their base_url at this Proxy to manage all LLM access from one place.

6.1 The simplest way to start it

# Start directly, routing to a specific model
litellm --model openai/gpt-4o

# Proxy runs at http://0.0.0.0:4000

6.2 Core config.yaml settings

For production, it's best to manage everything through a config.yaml:

model_list:
  # Primary model: Claude Sonnet 5
  - model_name: claude-sonnet
    litellm_params:
      model: anthropic/claude-sonnet-5
      api_key: os.environ/ANTHROPIC_API_KEY

  # Fallback: DeepSeek via SiliconFlow (direct-connect relay)
  - model_name: claude-sonnet
    litellm_params:
      model: openai/deepseek-v4-flash
      api_base: https://api.siliconflow.cn/v1
      api_key: os.environ/SILICONFLOW_API_KEY
      order: 2   # fallback

  # Free coding model: GLM-4.7-Flash (direct-connect relay)
  - model_name: glm-flash
    litellm_params:
      model: openai/glm-4.7-flash
      api_base: https://open.bigmodel.cn/api/paas/v4/
      api_key: os.environ/GLM_API_KEY

  # Local Ollama
  - model_name: local-llama
    litellm_params:
      model: ollama/llama3.2
      api_base: http://localhost:11434

litellm_settings:
  drop_params: true          # automatically drop params the target model doesn't support
  num_retries: 3             # global retry count
  request_timeout: 60        # timeout in seconds
  success_callback: ["langfuse"]  # monitoring callback

general_settings:
  master_key: sk-my-secret-key   # key required to access the Proxy
  alerting: ["slack"]            # alerts for slow requests/errors

router_settings:
  routing_strategy: simple-shuffle
  allowed_fails: 3
  cooldown_time: 30

# Start with the config file
litellm --config config.yaml

# Verify it's up
curl http://localhost:4000/health

6.3 Calling the Proxy through the OpenAI SDK

import openai

# Point at your local Proxy
client = openai.OpenAI(
    api_key="sk-my-secret-key",    # the master_key from config.yaml
    base_url="http://localhost:4000"
)

response = client.chat.completions.create(
    model="claude-sonnet",    # matches the model_name in config.yaml
    messages=[{"role": "user", "content": "Hello"}]
)
print(response.choices[0].message.content)

6.4 Model discovery (listing all available models)

curl http://localhost:4000/models \
  -H "Authorization: Bearer sk-my-secret-key"

7. Integrating with Claude Code

This is one of LiteLLM's most popular use cases: routing Claude Code's requests to other models (GPT-5.6, Gemini, DeepSeek, GLM), or through a relay to work around direct-access reliability issues.

7.1 How it works

Claude Code internally uses the Anthropic Messages API format. LiteLLM Proxy will:

1. Receive the Anthropic-format request Claude Code sends
2. Automatically convert it into the target provider's format (OpenAI/Gemini/DeepSeek/etc.)
3. Forward it to the target provider
4. Convert the response back into Anthropic format and return it to Claude Code

7.2 Setup steps

Step 1: Create a config.yaml (using GPT-4o and a relay provider as an example)

model_list:
  - model_name: gpt-4o
    litellm_params:
      model: openai/gpt-4o
      api_key: os.environ/OPENAI_API_KEY

  - model_name: deepseek-v4-flash
    litellm_params:
      model: openai/deepseek-v4-flash
      api_base: https://api.siliconflow.cn/v1
      api_key: os.environ/SILICONFLOW_API_KEY

  - model_name: glm-flash
    litellm_params:
      model: openai/glm-4.7-flash
      api_base: https://open.bigmodel.cn/api/paas/v4/
      api_key: os.environ/GLM_API_KEY

general_settings:
  master_key: sk-claude-proxy

Step 2: Start the Proxy

litellm --config config.yaml
# Listening on http://0.0.0.0:4000

Step 3: Set Claude Code's environment variables

export ANTHROPIC_BASE_URL="http://0.0.0.0:4000"
export ANTHROPIC_AUTH_TOKEN="sk-claude-proxy"

Step 4: Launch Claude Code with a specific model

# Use GPT-4o
claude --model gpt-4o

# Use the direct-connect DeepSeek relay
claude --model deepseek-v4-flash

# Use the free GLM-4.7-Flash
claude --model glm-flash

# Enable gateway model discovery (switch models inside Claude Code with /model)
export CLAUDE_CODE_ENABLE_GATEWAY_MODEL_DISCOVERY=1
claude   # once running, use /model to list and switch models

7.3 How it compares to claude-code-router

Many readers may already be using claude-code-router (CCR) — so how does LiteLLM Proxy differ from it?

Comparison	LiteLLM Proxy	claude-code-router
Supported providers	100+	A handful of major ones
Routing strategies	7, highly configurable	Routes based on request type
Claude Code integration	✓	✓ (purpose-built for this)
Admin dashboard	✓ (built-in UI)	✗
Multi-user teams	✓ (virtual keys)	✗
Cost tracking	✓ (fine-grained)	✗
Deployment complexity	Moderate (needs ops work)	Simple (single process)

In short: CCR is lighter for individuals, and LiteLLM Proxy is more complete for teams. You can also stack them: CCR handles Claude Code's smart routing, while LiteLLM handles unified multi-provider access for your other Python services.

8. Cost Tracking and Budget Controls

8.1 A custom cost callback

import litellm
from litellm.integrations.custom_logger import CustomLogger

class CostTracker(CustomLogger):
    def __init__(self):
        self.total_cost = 0.0
        self.calls = []

    def log_success_event(self, kwargs, response_obj, start_time, end_time):
        cost = kwargs.get("response_cost", 0)
        self.total_cost += cost
        self.calls.append({
            "model": kwargs.get("model"),
            "cost": cost,
            "tokens": response_obj.usage.total_tokens,
            "latency": (end_time - start_time).total_seconds()
        })
        print(f"[Cost] {kwargs.get('model')}: ${cost:.6f}")

tracker = CostTracker()
litellm.callbacks = [tracker]

# Check the running total after a call
response = litellm.completion(
    model="openai/gpt-4o",
    messages=[{"role": "user", "content": "Hello"}]
)
print(f"Total cost so far: ${tracker.total_cost:.4f}")

8.2 Proxy virtual keys and budgets

LiteLLM Proxy supports issuing virtual API keys per user or project, each with its own independent spending cap:

# Create a virtual key (via the Admin API)
curl -X POST http://localhost:4000/key/generate \
  -H "Authorization: Bearer sk-my-secret-key" \
  -H "Content-Type: application/json" \
  -d '{
    "team_id": "frontend-team",
    "max_budget": 10.0,       # $10 spending cap
    "budget_duration": "30d", # resets every 30 days
    "models": ["claude-sonnet", "glm-flash"],  # only these models are allowed
    "metadata": {"user": "alice@example.com"}
  }'

# Returns: {"key": "sk-virtual-abc123", ...}

Alice calls the Proxy with sk-virtual-abc123, and any call over $10 is automatically rejected — without ever needing to share the master key.

9. Connecting Relay Providers

Mainstream relay providers generally expose an OpenAI-compatible interface, so connecting them to LiteLLM is straightforward. Here are configs for a few common ones:

model_list:
  # SiliconFlow (direct-connect, full DeepSeek/Qwen/GLM lineup)
  - model_name: deepseek-v4-flash
    litellm_params:
      model: openai/deepseek-v4-flash
      api_base: https://api.siliconflow.cn/v1
      api_key: os.environ/SILICONFLOW_API_KEY

  # AiHubMix (direct-connect Claude/GPT relay, supports prompt caching)
  - model_name: claude-sonnet-via-relay
    litellm_params:
      model: openai/claude-sonnet-5-20261001
      api_base: https://aihubmix.com/v1
      api_key: os.environ/AIHUBMIX_API_KEY

  # Zhipu AI GLM (direct-connect, free-tier models available)
  - model_name: glm-flash-free
    litellm_params:
      model: openai/glm-4.7-flash
      api_base: https://open.bigmodel.cn/api/paas/v4/
      api_key: os.environ/GLM_API_KEY

  # OpenRouter (unified interface to 400+ global models)
  - model_name: openrouter-sonnet
    litellm_params:
      model: openai/anthropic/claude-sonnet-5
      api_base: https://openrouter.ai/api/v1
      api_key: os.environ/OPENROUTER_API_KEY

Set the environment variables, then start:

export SILICONFLOW_API_KEY="sk-sf-..."
export AIHUBMIX_API_KEY="sk-ahm-..."
export GLM_API_KEY="your-glm-key"

litellm --config config.yaml

With this, a single Proxy now unifies four different relay providers — your code only needs to connect to http://localhost:4000, and different model_name values route to different relay providers, all managed in one place.

10. Production Deployment: Docker + Redis

10.1 Single-container Docker deployment

# docker-compose.yml
version: '3.8'
services:
  litellm:
    image: ghcr.io/berriai/litellm:main-latest
    ports:
      - "4000:4000"
    volumes:
      - ./config.yaml:/app/config.yaml
    environment:
      - ANTHROPIC_API_KEY=${'{ANTHROPIC_API_KEY}'}
      - SILICONFLOW_API_KEY=${'{SILICONFLOW_API_KEY}'}
      - GLM_API_KEY=${'{GLM_API_KEY}'}
      - LITELLM_MASTER_KEY=sk-production-key
    command: --config /app/config.yaml --port 4000 --num_workers 8
    restart: unless-stopped

docker-compose up -d

10.2 Production-grade deployment: adding Redis for shared distributed state

# docker-compose-production.yml
version: '3.8'
services:
  redis:
    image: redis:7-alpine
    ports:
      - "6379:6379"
    volumes:
      - redis-data:/data

  litellm:
    image: ghcr.io/berriai/litellm:main-latest
    ports:
      - "4000:4000"
    volumes:
      - ./config.yaml:/app/config.yaml
    environment:
      - ANTHROPIC_API_KEY=${'{ANTHROPIC_API_KEY}'}
      - REDIS_HOST=redis
      - REDIS_PORT=6379
      - LITELLM_MASTER_KEY=sk-production-key
    depends_on:
      - redis
    command: --config /app/config.yaml --port 4000 --num_workers 8

volumes:
  redis-data:

Enable Redis in config.yaml:

router_settings:
  routing_strategy: usage-based-routing   # distributed-aware routing
  redis_host: redis
  redis_port: 6379
  cache_responses: true   # enable response caching to cut cost

10.3 Performance reference numbers

Setup	RPS	P95 latency
Single instance, 8 workers	~400	<10ms (proxy layer)
Single instance, 16 workers	~800	<10ms
3 instances + Redis	~1500+	<8ms (P95)

11. Security Notes

⚠️ Important security warning: LiteLLM PyPI package versions v1.82.7 and v1.82.8 have been confirmed compromised in a supply-chain attack that planted credential-stealing malware, sending your API keys and environment variables to an external server. If you installed either of these versions, immediately: ① upgrade to the latest version; ② rotate every API key; ③ check your system for unusual processes.

Other security recommendations:

• Never hardcode API keys in config.yaml — always reference environment variables using the os.environ/VAR_NAME format
• Don't expose the Proxy to the public internet unless you've configured virtual keys and an IP allowlist; for local development, bind to 127.0.0.1:4000 rather than 0.0.0.0:4000
• Rotate the master_key regularly; virtual keys can be set to expire
• When enabling request logging, make sure to scrub PII so sensitive user input isn't recorded to an external monitoring system

12. FAQ

Q: I'm already using the OpenAI SDK — how much code do I have to change to adopt LiteLLM?

A: In Proxy mode, just two lines: change api_key to the Proxy's master_key, and base_url to the Proxy's address. Your business logic doesn't change at all. In Python SDK mode, swap openai.ChatCompletion.create() for litellm.completion() — the parameter format is identical.

Q: How much latency does LiteLLM add?

A: Officially benchmarked at roughly 8ms P95 (the proxy layer itself, not counting the LLM's own response time). For most LLM workloads, the model's own response already takes hundreds of milliseconds to several seconds, so an extra 8ms is negligible.

Q: Can I use LiteLLM in a Node.js / Go / Java project?

A: Yes, via Proxy mode. The Proxy exposes a standard, OpenAI-compatible HTTP REST interface, so any language can call it directly with an HTTP client, or point that language's own OpenAI SDK's base_url at the Proxy.

Q: What does LiteLLM's Anthropic-compatible endpoint (`/anthropic`) support?

A: It supports the core features of the Messages API, including streaming, tool_use, vision (image input), and system prompts — plus routing Anthropic SDK-based tools like Claude Code, through the Proxy, to other providers (GPT, Gemini, DeepSeek, etc.).

Q: How do I debug whether routing is behaving as expected?

A: Add the --detailed_debug flag on startup, or set set_verbose=True when initializing the Router — this prints the routing decision, retry log, and actual deployment used for every request.

Q: Does LiteLLM support embeddings and image generation?

A: Yes. LiteLLM's unified interface covers /embeddings (text vectorization), /images/generations (image generation), /audio/transcriptions (speech recognition), and more — all called similarly to completion().

13. Conclusion

LiteLLM solves a very real pain point in LLM engineering: multi-provider fragmentation. When your system depends on OpenAI, Anthropic, local models, and relay providers all at once, LiteLLM is currently the most mature unification layer available.

Recommendations by use case:

• Writing Python scripts / personal projects: use SDK mode — pip install and call completion() directly, the lightest option
• Wiring multi-model access into Claude Code / Cursor: use Proxy mode — a 5-minute setup, point ANTHROPIC_BASE_URL at your local Proxy
• Sharing LLM access across a team: Proxy + virtual keys + budget controls, full enterprise-grade management
• High-availability production systems: Proxy + Redis + multiple instances, 1500+ RPS with P95 latency under 8ms

For developers outside the US: pairing LiteLLM with relay providers (SiliconFlow, AiHubMix, GLM, etc.) is currently the most flexible way to access LLMs — the relay solves the direct-access problem, and LiteLLM solves unified multi-provider management, and the two complement each other well.