Optimize findings

The Optimize page runs a recurring pass over the last 7 days of request_logs and produces concrete findings — each with an estimated monthly $ impact and a one-click fix path.

Rendering diagram…

Finding types

How findings are classified

The analyzer uses a combination of structural and statistical signals:

• Oversized prompt — flags requests where the system prompt exceeds the model-specific token threshold (e.g. 4,000 tokens for gpt-4o-mini, 8,000 for gpt-4o) and the response quality metrics (completion length, stop reason) don't correlate with the prompt size. A 12,000-token system prompt that produces the same output as a 2,000-token version is a trim candidate.
• Duplicate requests — hashes the prompt content (system + user messages) and flags clusters of 3+ identical hashes within a 60-second window from the same virtual key. Near-duplicates (Jaccard similarity > 0.95 on the message array) are also flagged.
• Model mismatch — classifies each request's task type from the message structure (single-turn chat, multi-turn conversation, tool-use/agentic, code generation) and compares against the model's intended use case. A single-turn "what's the capital of France?" running on o3 is a mismatch.
• Stuck in retry — tracks per-session retry rates and flags sessions where more than half the attempts hit the same model with the same error class (typically 429 or 500).

How estimates are calculated

Each finding ships with a projected monthly $ savings, computed from:

• The volume of matching requests in the last 7 days.
• The cost delta between the current model and the suggested target (priced from the LiteLLM catalog).
• A 30-day projection assuming current traffic patterns hold.

Estimates are conservative — RouteShift floors negatives to zero and ignores findings under $1/month so the page only surfaces work worth doing.

Acting on a finding

Each card has an Apply fix button that drops you into the right surface:

• Oversized prompt → links to the offending requests so you can see which prompt template needs trimming.
• Duplicate requests → opens the rule editor pre-filled to add a deduplication tag.
• Model mismatch → opens /models/compare with both models pre-selected, then opens a rule draft.
• Stuck in retry → same as model mismatch.

Findings auto-resolve when the underlying behavior disappears from the trailing 7-day window.

Best practices

• Start with the highest-$ finding. The page ranks by projected savings, so the top card is always the best ROI.
• Fix model mismatches with rules, not client changes. A routing rule that re-maps chat tasks from o3 to gpt-4o-mini fixes every affected key at once — no client deploy needed.
• Pair with budgets. After fixing findings, set a budget on the affected keys so the same waste pattern triggers an alert if it recurs.
• Review after deploys. New prompt templates or agent deployments are the most common source of new findings. Check Optimize after any significant prompt change.