Hermes-router puts provider failover behind an OpenAI-compatible endpoint

Hermes-router is a single-file Python proxy, served via Flask/Waitress, that sits between an application and a pool of model providers. Compatible clients point their base URL at the router — the README uses localhost:8319 — and keep using OpenAI-style chat, embeddings, and models calls, or Anthropic’s Messages endpoint. The practical appeal is straightforward: when one provider returns a rate limit, the router tries the next configured option instead of surfacing that error to the app. The routing layer is the core of the project. According to the README, each request passes an auth check against configured proxy keys, an exact-match cache lookup, a 1–5 complexity rating, and then provider ordering based on fit and health. Keys for a provider are pooled from a structured auth.json and .env, selected with synchronized round-robin rotation, and a rate-limited key is temporarily sidelined. A circuit breaker is intended to stop repeated attempts against unhealthy providers. The README is explicit about the limit of this design: requests can still fail when every configured provider is unavailable or exhausted. Who it fits, and what else is bundled This targets developers and small teams stretching free tiers or mixing free and paid accounts across Gemini, OpenRouter, Groq, Mistral, Cohere, NVIDIA, Codex, Kimi, and local Ollama or LM Studio models. Smart routing claims to pick the cheapest model that can handle a request, with local models able to fall back to cloud providers. Tool calling and embeddings are supported, alongside optional semantic caching in addition to exact-match caching. Operational features are unusually broad for a README of this size: per-key budgets and rate limits, usage analytics at /v1/usage