Regulated fintech cuts AI spend by ~55% after routing all traffic through an LLM gateway
Financial services / fintech · ~350 employees, regulated · EU / Switzerland · Gateway live in 2 weeks; ~55% AI cost reduction within one quarter
Short answer
A ~350-person regulated European fintech reduced its LLM spend by roughly 55% within one quarter by routing all model traffic through an LLM gateway with task-based model routing, semantic caching and per-team budgets — with under 10ms of added routing overhead.
~55%
reduction in monthly LLM spend
<10ms
added routing overhead
11 → 11
teams, each now with its own visible budget
2 weeks
from kickoff to all traffic behind the gateway
The challenge
- Eleven product teams shared a handful of provider API keys, so spend could not be attributed to anyone.
- Every task ran on the most capable (and most expensive) model, including simple classification and extraction.
- A single misbehaving agent could burn a month's budget overnight with nothing to stop it.
- Compliance could not evidence which model saw which data — a blocker for the next audit.
What bluebill did
- Placed an LLM gateway in front of all model traffic; applications changed a base URL, not their code.
- Introduced task-based routing so cheap and self-hosted models handle simple work and frontier models handle the rest.
- Enabled semantic caching for repeated prompts across support and internal-knowledge workloads.
- Set hard per-team budget ceilings and per-key rate limits, with anomaly detection on agent traffic.
- Turned on PII redaction and prompt-injection guardrails, with full request logging exported to the existing SIEM.
“We expected a cost project. What we got first was an answer to 'who is using what, with which data'.”
— Head of Platform, anonymised European fintech
What changed afterwards
- AI spend is attributed per team and per application, and reviewed on the same cadence as cloud spend.
- A runaway agent now costs minutes of budget instead of weeks, because ceilings stop it automatically.
- The audit evidence — model inventory, guardrail decisions, retention-bounded logs — is produced by normal operation.
Environment: Multiple LLM providers · Self-hosted open models · Kubernetes · SIEM export
Questions this engagement answers
How much can an LLM gateway realistically save?
This fintech cut LLM spend by roughly 55% in a quarter. Most of the saving came from routing simple tasks to cheaper or self-hosted models and from caching repeated prompts, not from reducing how much teams used AI.
Does a gateway add latency?
Routing overhead here stayed under 10ms. Caching made a meaningful share of requests faster than before, because cached responses skip the model call entirely.
How much application change is required?
Applications pointed at a new base URL. No SDK rewrites were needed, which is why all traffic was behind the gateway within two weeks.
Client names are withheld by agreement. Engagement profiles are anonymised; figures are the measured results of the engagement described and are consistent with bluebill's aggregate programme results. Individual results vary with environment, scale and starting maturity.