llmbridge

llmbridge is a unified Go interface to every major LLM provider. Switch between OpenAI, Anthropic, Gemini, Bedrock, Azure, Cohere, Ollama, Groq, or any OpenAI-compatible endpoint by changing one line, and application code never changes. One API covers chat, streaming, tool use, embeddings, TTS, and image generation. A router adds multi-provider failover with weighted routing, circuit breaking, and typed fallback for context-window and content-policy errors. Ships with an OpenAI-compatible HTTP proxy, four cache backends (in-memory, disk, Redis, and semantic cosine-similarity), per-key budget and spend tracking, guardrails for PII and prompt injection, and observability via Langfuse tracing and Prometheus metrics. Multi-tenant auth with SSO/OIDC, deployed via Docker, docker-compose, or a Helm chart.

Technologies Used

Go

Redis

Prometheus

SQLite

Docker

Kubernetes

GitHub Actions