Self-Hosted AI Gateway with 2server: Pool Accounts, Avoid 429s
A single 429 rate limit can bring down an entire AI pipeline. 2server's cliproxy template runs CLIProxyAPI as an OpenAI-compatible gateway on your VM: pooling OAuth accounts, round-robin routing, and automatic 429 failover.
The problem: one account hits 429 and the entire app breaks
Developers self-hosting AI agents or AI-backed applications face two recurring problems:
- Cascading failures from rate limits. An agent calls Claude or Codex via a single provider account. When that account hits
429 Too Many Requestsor runs out of quota, there is no fallback path, and the entire pipeline halts. - Scattered credentials. Multiple applications call the same provider. For convenience, OAuth tokens are copied directly into each service. The more places holding tokens, the greater the surface for credential leaks.
The standard fix is an AI gateway running locally on your VM: applications target the gateway, while the gateway manages credentials, routing, and failovers. 2server provides a dedicated template for this: cliproxy.
What is cliproxy
According to 2server documentation, the cliproxy template runs CLIProxyAPI as an OpenAI-compatible AI gateway directly on your VM. It executes as an unprivileged Go binary with a read-only root filesystem.
What the gateway handles
- Pools OAuth accounts. A single gateway consolidates accounts across Codex, Claude, Antigravity, xAI, and Kimi.
- Dedicated client keys per application. Each consuming application gets its own client key and never handles provider tokens directly.
- Round-robin load balancing. Multiple accounts for the same provider are balanced cyclically.
- Priority-based failover. When an account encounters quota exhaustion or a
429error, the gateway automatically falls back to alternative accounts in priority order. - Provider tokens remain isolated. The core design goal: applications only ever touch the local client key.
[App A] → client key A ─┐
[App B] → client key B ─┤ CLIProxyAPI → Claude accounts
[Agent C] → client key C ─┘ (on VM) → Codex accounts
→ xAI accounts ...When to use it
A good fit when:
- You run two or more agents or apps calling AI providers and want to pool quota without sharing raw provider tokens.
- You maintain multiple accounts for the same provider and need to unify them behind a single endpoint.
Unnecessary when:
- You run a single app against a single provider and have never encountered rate limits.
Where to start
Configuration details (schema keys, adding accounts, issuing client keys) are covered in the template documentation. Review the guide before deploying; this overview covers architectural behavior rather than configuration references.
Full documentation: 2server cliproxy
