Notes from building a multi-model inference gateway — cost, caching, infrastructure, and the economics of running frontier open-weight models.
July 16, 2026
Two backends, the same open-weight model, the same OpenAI-compatible API — and an 8× gap in time-to-first-token, with one of them faking streaming. What we learned benchmarking LLM backends for agentic traffic, and why the provider's spec sheet won't tell you.
June 26, 2026
Multi-provider routers load-balance the same model across providers, which quietly kills your prompt cache. On a cache-heavy agent workload that's the difference between the cached rate card and 32x it. Here's the mechanism — and why cache locality should be the default.
June 21, 2026
If you already call LLMs through OpenRouter with the OpenAI SDK, switching to Dynoyard is a one-line change. Same request shape, same streaming, same tool calls, same usage accounting.
June 15, 2026
An anonymized look at a real multi-tenant AI platform — why its workload shape and model choice, not the headline per-token rate, decide the bill, and how that lands it at roughly a third of frontier-lab pricing.
June 8, 2026
A leaderboard score is a poor predictor of how a model performs on your task. Here's a more reliable way to choose — cost-led, measured on your own traffic.