One route's prompts are far larger than the rest
One part of your app sends much bigger prompts than everywhere else — usually old context or examples that pile up and never get trimmed. You pay for every extra word, on every call.
Route 'rag-bloated' /v1/rag sends prompts far larger than comparable traffic producing similar-length responses (p90 6000 tokens vs corpus norm). Trim static instructions, deduplicate context, and move stable content behind prompt caching. Savings assume only half the excess is removable (0.5 safety factor).
Evidence sample (20 of max 20 calls — token counts only, never text)
| When (UTC) | Model | Tokens | Note |
|---|---|---|---|
| 2026-06-11T16:00:00+00:00 | claude-haiku-4-5 | 6350 | prompt above corpus norm |
| 2026-06-11T16:03:20+00:00 | claude-haiku-4-5 | 6350 | prompt above corpus norm |
| 2026-06-11T16:06:40+00:00 | claude-haiku-4-5 | 6350 | prompt above corpus norm |
| 2026-06-11T16:10:00+00:00 | claude-haiku-4-5 | 6350 | prompt above corpus norm |
| 2026-06-11T16:13:20+00:00 | claude-haiku-4-5 | 6350 | prompt above corpus norm |
| 2026-06-11T16:16:40+00:00 | claude-haiku-4-5 | 6350 | prompt above corpus norm |
| 2026-06-11T16:20:00+00:00 | claude-haiku-4-5 | 6350 | prompt above corpus norm |
| 2026-06-11T16:23:20+00:00 | claude-haiku-4-5 | 6350 | prompt above corpus norm |
| 2026-06-11T16:26:40+00:00 | claude-haiku-4-5 | 6350 | prompt above corpus norm |
| 2026-06-11T16:30:00+00:00 | claude-haiku-4-5 | 6350 | prompt above corpus norm |
| 2026-06-11T16:33:20+00:00 | claude-haiku-4-5 | 6350 | prompt above corpus norm |
| 2026-06-11T16:36:40+00:00 | claude-haiku-4-5 | 6350 | prompt above corpus norm |
| 2026-06-11T16:40:00+00:00 | claude-haiku-4-5 | 6350 | prompt above corpus norm |
| 2026-06-11T16:43:20+00:00 | claude-haiku-4-5 | 6350 | prompt above corpus norm |
| 2026-06-11T16:46:40+00:00 | claude-haiku-4-5 | 6350 | prompt above corpus norm |
| 2026-06-11T16:50:00+00:00 | claude-haiku-4-5 | 6350 | prompt above corpus norm |
| 2026-06-11T16:53:20+00:00 | claude-haiku-4-5 | 6350 | prompt above corpus norm |
| 2026-06-11T16:56:40+00:00 | claude-haiku-4-5 | 6350 | prompt above corpus norm |
| 2026-06-11T17:00:00+00:00 | claude-haiku-4-5 | 6350 | prompt above corpus norm |
| 2026-06-11T17:03:20+00:00 | claude-haiku-4-5 | 6350 | prompt above corpus norm |