Unified LLM Gateway

Teams want to use the best model for each task without juggling several provider accounts, SDKs, keys and invoices — and without rewriting code that already speaks the OpenAI or Anthropic API.

1. OpenAI / Anthropic SDK
2. Auth & plan limits
3. Router
4. Provider pool
5. Metering
6. Admin console

1. Drop-in endpoints compatible with both the OpenAI and Anthropic SDKs, including streaming and tool calls, so existing clients only change a base URL.
2. A routing layer over a pool of upstream providers with health monitoring and fallback.
3. Per-key usage metering, plan limits and a checkout flow, with an admin console for users, plans, keys and pool health.
4. Provider and payment secrets encrypted at rest; bot protection on sign-up.
5. Ships as Docker with an optional Cloudflare Tunnel, or to Workers / Vercel; covered by Vitest.