Plugins

Plugins are per-request augmentation steps that run inside the proxy before it dispatches to an upstream provider. They let a single request pull in extra context — live web-search results, extracted PDF content — without the client changing SDKs, wiring a second API, or shipping the enrichment logic itself.

Two plugins are available today:

• web — runs a live web search and injects the results into the system prompt as labeled untrusted context.
• file-parser — accepts inline or URL-backed PDF file parts, applies bounded extraction (or fail-closed native PDF passthrough on providers that support it), and forwards the augmented request upstream.

Plugin runs are non-cacheable, so a fresh search or a fresh extraction happens on every request.

Enabling plugins

There are three ways to enable a plugin on a request:

• Model suffix — append :online to the model name (gpt-4o:online, claude-3-5-sonnet:online) to enable the web plugin with default settings.
• Explicit plugins array — pass one or more plugin objects on the request body to enable web, file-parser, or both, and to tune their options.
• Content parts — the file-parser plugin is enabled automatically whenever a request contains a file or input_file content part; no plugins entry is needed.

{
  "model": "gpt-4o",
  "messages": [
    {
      "role": "user",
      "content": [
        { "type": "text", "text": "What shipped in the OpenAI DevDay 2026 keynote? Compare against the attached agenda." },
        { "type": "file", "filename": "agenda.pdf", "file": { "file_url": "https://example.com/devday-agenda.pdf" } }
      ]
    }
  ],
  "plugins": [
    { "id": "web", "max_results": 5, "search_prompt": "OpenAI DevDay 2026 keynote announcements" },
    { "id": "file-parser" }
  ]
}

The plugins array is stripped before the request reaches the upstream provider — providers never see RouteShift-specific fields. The :online model suffix is stripped for the same reason.

Web search

The web plugin runs a live search against a configured backend and injects the results into the system prompt as a clearly-delimited untrusted-context block. See Web search for the full options, request/response shape, warning codes, and per-request surcharge.

File parser

The file-parser plugin turns PDF file parts (base64 inline, or an https:// URL) into either a bounded text extraction or, on providers with a tested native adapter, a validated native PDF part. It enforces per-file and per-request byte, page, and text-length budgets, and runs URL fetches through an SSRF-safe, DNS-pinned transport.

See File parser for supported input shapes, budget defaults and environment variables, native-vs-text passthrough rules, and the full list of warning codes.

Plugin runs, billing, and audit

Every plugin execution — whether it ran successfully, degraded to a warning, or errored — writes a sanitized audit row keyed by (request_id, plugin_id). Rows record the plugin id, status (ok / warning / error / skipped), latency, cost, and a stable detail code (never a URL, filename, or backend body).

Plugin surcharges are tracked separately from provider model cost, so plugin fees never get silently relabeled as routing spend. Today only the web plugin adds a per-invocation surcharge; the file-parser plugin does not. Requests that ran a plugin are always marked non-cacheable — the same prompt run twice will pay the surcharge twice. See Savings and pricing and Observability for how plugin cost and audit rows appear in your usage feed.

Streaming responses can't carry a JSON warnings envelope, so the proxy sends the same information as X-RouteShift-Plugin-Warning (comma-separated codes) and X-RouteShift-Plugin-Skip-Reason (human-readable reasons) headers.

Using plugins from the SDK

@routeshift/sdk re-exports the wire types (PluginId, PluginSpec, PluginWarning, PluginWarningResponseMetadata) so requests are typed end-to-end. Non-streaming responses expose warnings on res.warnings; streaming responses expose the header equivalents on stream.metadata:

import { ProxyClient } from "@routeshift/sdk";

const client = new ProxyClient({
  baseUrl: "https://api.routeshift.io",
  apiKey: process.env.ROUTESHIFT_KEY!,
});

// Buffered call — warnings arrive in the response body.
const res = await client.chat({
  model: "gpt-4o",
  plugins: [{ id: "web", max_results: 5 }],
  messages: [{ role: "user", content: "What shipped this week?" }],
});
for (const warning of res.warnings ?? []) {
  console.warn(`${warning.plugin}: ${warning.code}`);
}

// Streaming call — warnings arrive as headers, surfaced through metadata.
const stream = client.chatStream({
  model: "gpt-4o:online",
  plugins: [{ id: "web" }],
  messages: [{ role: "user", content: "Summarize today's filings." }],
});
const { pluginWarning, pluginSkipReason } = await stream.metadata;
if (pluginWarning) console.warn(pluginWarning, pluginSkipReason);
for await (const event of stream) {
  process.stdout.write(event.content ?? "");
}

Awaiting stream.metadata before iterating is safe — the SDK starts the underlying request once and shares it with the event iterator. If the request fails before headers arrive, stream.metadata resolves to an empty object and iterating the stream throws the underlying error.

Billing

Every plugin defaults to optional: a backend outage, misconfiguration, or per-file failure attaches a warning to the response and lets the underlying model call proceed. Set "required": true on a plugin entry when a degraded answer is worse than a failed request — RouteShift then short-circuits with 502 plugin_required_failed before dispatching upstream.