Skip to content

Latest commit

 

History

History
837 lines (687 loc) · 54.7 KB

batch-quickstart.mdx

File metadata and controls

837 lines (687 loc) · 54.7 KB
publish true
title Batch API Quickstart
sidebarTitle Quickstart
description Submit a batch, get notified when it finishes, read the results, and clean up
slug batch-quickstart
seoTitle Batch API Quickstart | OpenRouter Documentation
og:site_name OpenRouter Documentation
og:title Batch API Quickstart
og:description Submit multiple inference requests with OpenRouter and retrieve their results asynchronously.
og:image https://openrouter.ai/dynamic-og?pathname=batch-quickstart&title=Batch%20API%20Quickstart&description=Submit%20and%20retrieve%20asynchronous%20batches%20of%20inference%20requests
og:image:width 1200
og:image:height 630
twitter:card summary_large_image
twitter:site @OpenRouter
noindex false
nofollow false
icon rocket

This guide submits a batch, waits for it with a webhook or by polling, reads the results, and deletes the batch. For what batch is and when to use it, see the Batch API overview.

Before you start, you need:

  • An OpenRouter API key.
  • Credits for the estimated cost. OpenRouter checks the estimate against your balance before it accepts the batch. Batches on your own provider key (BYOK) and invoiced accounts skip this check.
  • A model from the batch models list. Use its normal slug, for example anthropic/claude-haiku-5.5. A model with no batch variant returns 400.

Build the batch

A batch has one model, one endpoint, and a list of requests. Each request body is the same body you send to the sync endpoint.

Field Required Description
endpoint Yes /v1/chat/completions, /v1/responses, /v1/messages, or /v1/embeddings. Every request uses this endpoint.
model Yes The model for every request. A request body can leave out model. If it sets model, the value must match.
requests Yes, unless input is set A non-empty array of { custom_id, body }. custom_id must be unique in the batch.
input No { "uri": "gs://bucket/object" } or { "uri": "bq://project.dataset.table" }: an input file in your own Cloud Storage bucket, or an input table in your own BigQuery dataset, sent in place of requests. Enabled per account, on Google Vertex with your own provider key only. See BYOK.
callback_url No A public HTTPS URL that receives a webhook when the batch finishes. See Wait with a webhook.
provider No only and allow_fallbacks. See Choose a provider.
completion_window No 24h, which is the default and the only accepted value.
{
  "endpoint": "/v1/responses",
  "model": "anthropic/claude-haiku-5.5",
  "callback_url": "https://example.com/webhooks/openrouter-batch",
  "requests": [
    {
      "custom_id": "review-1",
      "body": {
        "input": "Classify: The fit is perfect."
      }
    },
    {
      "custom_id": "review-2",
      "body": {
        "input": "Classify: It arrived broken."
      }
    }
  ]
}
Put `endpoint`, `model`, `provider`, `completion_window`, and `callback_url` before `requests`. OpenRouter reads the body as a stream so it can accept large batches without buffering them. If `requests` comes first, the submit returns `400`.

Add images and files

Batch accepts text, public image URLs, and some file URLs. It returns text and embeddings only. What works depends on the provider that runs the batch, so pin a provider with provider.only when you send images. A batch with file URLs, inline files, or YouTube URLs that lands on a provider that can't take them moves to another provider for the model that can. This happens only for batches billed through OpenRouter, not ones sent with your own provider key, when provider.allow_fallbacks is not false and, if you set provider.only, the new provider is in that list.

Provider Image URL File URL
Anthropic Yes Yes
DeepInfra (/v1/chat/completions only) Yes Yes
Mistral Yes Yes
OpenAI Yes Models served on the Responses API only
Google Vertex Yes, and gs:// images with BYOK gs:// PDFs and text files with BYOK only, and gs:// images, audio, and video for enabled accounts
Google AI Studio Yes No. Public YouTube URLs work as video_url parts.
xAI, Together, Sail Research Yes No
Alibaba (/v1/chat/completions only) Yes No
Fireworks, Morph No No

File parts use the endpoint's own type: input_file on /v1/responses, file on /v1/chat/completions, and document on /v1/messages. The entries below use the Responses shape so they drop into the batch above; on a /v1/chat/completions batch, send messages with image_url and file parts instead. The YouTube entry is the exception: it is a /v1/chat/completions line, so use it in a chat completions batch with a model that takes video input, such as google/gemini-2.5-flash.

On Google Vertex, image URLs must use HTTPS, end in .png, .jpg, .jpeg, .webp, .heic, or .heif, and not point at Google API, Googleusercontent, or Cloud Storage HTTPS hosts. A provider can also turn off image URLs for one model. File URLs work only on models that list file input. If the provider chosen at submit can't take file URLs, the batch moves to the cheapest other provider for the model that can, under the same conditions, as long as it costs at most twice the original provider's price. It fails with No batch provider for model '<model>' that this batch can route to accepts file URLs; use the sync API. when no provider qualifies. Public YouTube URLs work only on Google AI Studio, as video_url parts on /v1/chat/completions. For a model that takes video input, a YouTube batch that lands on Google Vertex moves to Google AI Studio for the same model under the same conditions and price limit, and fails with No batch provider for model '<model>' that this batch can route to accepts YouTube URLs; use the sync API. when no provider qualifies.

{
  "custom_id": "photo-1",
  "body": {
    "input": [
      {
        "role": "user",
        "content": [
          { "type": "input_text", "text": "Describe this product photo." },
          { "type": "input_image", "image_url": "https://example.com/shoe.jpg", "detail": "auto" }
        ]
      }
    ]
  }
}
{
  "custom_id": "contract-1",
  "body": {
    "input": [
      {
        "role": "user",
        "content": [
          { "type": "input_text", "text": "Summarize this contract." },
          { "type": "input_file", "filename": "contract.pdf", "file_url": "https://example.com/contract.pdf" }
        ]
      }
    ]
  }
}
{
  "custom_id": "video-1",
  "body": {
    "messages": [
      {
        "role": "user",
        "content": [
          { "type": "text", "text": "Summarize this video in one sentence." },
          { "type": "video_url", "video_url": { "url": "https://www.youtube.com/watch?v=jNQXAC9IVRw" } }
        ]
      }
    ]
  }
}
Not supported What to do instead
Base64 or data-URI images Host the image and send a public https URL. Inline base64 images (image/png, image/jpeg, image/webp, image/gif in image_url.url data URIs) are allowlist-only and also need a provider whose batch API has been verified to read them. Anthropic reads all four, up to 5 MiB decoded per image. OpenAI reads all four, animated GIF included, on both /v1/chat/completions and /v1/responses, up to 20 MiB decoded per image and per request (inline files and images in one request share that limit). DeepInfra is verified for image/png and image/jpeg (up to 20 MiB decoded per image and across all inline media in one request); it rejects image/webp and image/gif with its accepted types, and the MIME type must be lowercase with no parameters. Sail Research reads all four, static and animated GIF included, up to 20 MiB decoded per image and per request, and up to 20 images per request counting public URL and inline images together. Alibaba reads all four, up to 4 MiB decoded per image and per request, and refuses any request whose native line would exceed 6,291,456 bytes, because one oversized line fails the whole Alibaba batch. Mistral reads image/png, image/jpeg, and image/webp, up to 10 MiB decoded per image and 20 MiB of inline media per request; OpenRouter refuses inline image/gif for Mistral. xAI reads PNG, JPEG, and WebP but not GIF, up to 17 MiB decoded per image and per request (inline files and images in one request share that limit). Google Vertex: OpenRouter accepts image/png, image/jpeg, and image/webp, up to 7 MiB decoded per image and 20 MiB of inline media per request (inline files and images in one request share that limit), and sends them as inline image data; a native Vertex batch reading an inline image has not been verified yet. OpenRouter refuses inline image/gif for Vertex. Every other provider still rejects them with the provider's name and its accepted types. Pin the provider for the batch that contains these lines (for example provider: { only: ["sail-research"] } at the batch top level, next to model; see Choose a provider): an unpinned batch can route to a provider that refuses inline images, and that refusal does not fall back to another provider. Accounts on the explicit allowlist bypass known-CSAM screening until it is wired into batch submit; accounts admitted only by a rollout to all accounts get Inline base64 images need known-CSAM screening, which is not active for this batch; use a public http(s) image URL. Other accounts get Only public http(s) image URLs are supported in batch; base64/data-URI images are rejected. Inline images must be content parts of a user message: inline images in messages[].images, on assistant or system turns, and in tool results aren't supported in batch.
Inline file bytes and provider file IDs Send a public file URL, or an inline base64 data URI (data:<media type>;base64,...) of a type the provider's batch API has been verified to read. For accounts with inline files enabled, either individually or through a rollout to all accounts: PDFs work on Anthropic, DeepInfra, Google AI Studio, Google Vertex, Mistral, and OpenAI (up to 20 MiB decoded per file and per request) and on xAI (up to 17 MiB decoded per file and per request), so two 12 MiB files in one request fail admission even though each fits on its own; plain text, CSV, Markdown, and HTML also work on Google AI Studio and on OpenAI models served on the Responses wire; JSON and Word (.docx) also work on OpenAI models served on the Responses wire. OpenAI models still served on Chat Completions (for example gpt-audio) take PDFs only. Audio and video stay rejected as inline bytes; inline images follow the image row above. Each file must be what its media type says: a PDF starts with %PDF-, text types are valid UTF-8, JSON parses, and a .docx is a Word (OOXML) package, not just any ZIP archive. A type the chosen provider can't take fails the batch with the provider's name and its accepted types. Other accounts get Inline base64 files are not enabled for this account in batch. If the provider chosen at submit can't take inline files at all, the batch moves to another provider for the model that can (unless provider.allow_fallbacks is false); a type the chosen provider can't take is not re-routed. Inline bytes count toward the request size and are reserved against your balance as text at submit, so a large text file reserves more than its final token cost until the batch finalizes.
Audio input, and video other than YouTube Use the sync API. On Google Vertex with your own key (BYOK), accounts OpenRouter has enabled for batch audio or video can instead reference that media in Cloud Storage as gs:// files (see the Google Vertex section). A YouTube video on Google AI Studio holds the model's full input limit against your balance at submit, because its length is unknown. You are billed for the tokens the provider reports.
Image or audio output (modalities, audio, image_config) Use the sync API.
OpenRouter web search (:online, the web plugin, web_search_options) Use the provider's own search tool, for example OpenAI web_search on /v1/responses or Anthropic web_search_20250305 on /v1/messages. Google Vertex rejects web search tools.
stream: true, speed, deferred_tools, Anthropic beta features Remove them, or use the sync API.
access_programs.cyber other than standard (OpenAI Daybreak programs) Supported only on BYOK OpenAI requests served on the Responses API, using your own OpenAI key enrolled in that program. Remove the field on every other batch; the line is rejected before anything reaches the provider.
A request with no input, or a max output token cap below 1 Fix the request.
A web search tool with an engine other than auto or native Remove engine, or set it to native.

Unknown parameters are dropped before the request reaches the provider, the same as the sync API.

The provider fetches each URL when it runs the batch, which can be hours after you submit. Keep the URL public until the batch finishes. A request that fails these checks moves the batch to `failed`, and `error.message` names the request's `custom_id`.

Submit the batch

POST https://openrouter.ai/api/v1/batches
curl https://openrouter.ai/api/v1/batches \
  -H "Authorization: Bearer $OPENROUTER_API_KEY" \
  -H "Content-Type: application/json" \
  -d @batch.json
const headers = {
  Authorization: `Bearer ${process.env.OPENROUTER_API_KEY}`,
  'Content-Type': 'application/json',
};

const response = await fetch('https://openrouter.ai/api/v1/batches', {
  method: 'POST',
  headers,
  body: JSON.stringify({
    endpoint: '/v1/responses',
    model: 'anthropic/claude-haiku-5.5',
    callback_url: 'https://example.com/webhooks/openrouter-batch',
    requests: reviews.map((review) => ({
      custom_id: review.id,
      body: { input: `Classify: ${review.text}` },
    })),
  }),
});

if (response.status !== 202) {
  throw new Error((await response.json()).error.message);
}
const batch = await response.json();
import os
import requests

response = requests.post(
    "https://openrouter.ai/api/v1/batches",
    headers={"Authorization": f"Bearer {os.environ['OPENROUTER_API_KEY']}"},
    json={
        "endpoint": "/v1/responses",
        "model": "anthropic/claude-haiku-5.5",
        "callback_url": "https://example.com/webhooks/openrouter-batch",
        "requests": [
            {
                "custom_id": review["id"],
                "body": {"input": f"Classify: {review['text']}"},
            }
            for review in reviews
        ],
    },
)
response.raise_for_status()
batch = response.json()
{
  "id": "batch_123",
  "object": "batch",
  "endpoint": "/v1/responses",
  "model": "anthropic/claude-haiku-5.5",
  "completion_window": "24h",
  "status": "validating",
  "created_at": 1782097200,
  "finalized_at": null,
  "provider": "Anthropic",
  "expires_at": 1782183600,
  "request_counts": { "total": 2, "completed": 0, "failed": 0 },
  "usage": null,
  "results": null,
  "error": null
}

Save the id. A 202 means OpenRouter stored the batch. It checks every request next. If a check fails, the batch moves to failed and error.message says why.

provider is the provider that runs the batch. expires_at is the time after which that provider stops working on the batch, or null if the provider has no hard deadline. Most providers allow 24 hours. Google AI Studio allows 48 hours, and Google Vertex allows 96 hours.

If OpenRouter moves the batch to another provider for capacity before it starts, later reads show the new provider. expires_at changes to the new provider's window, still counted from created_at.


Wait for the batch to finish

Most batches finish in minutes, and nearly all finish within 24 hours. Use a webhook to find out when a batch finishes, and keep a slow poll as a backup.

Google Vertex can queue a batch for up to 72 hours while it waits for capacity, then process it for up to 24 hours. A Vertex batch that is still `validating` or `in_progress` after 24 hours is healthy.

Option A: Wait with a webhook

Set callback_url on the batch or Default batch webhook URL in your workspace settings. A batch-level URL overrides the workspace default and must use public HTTPS, and its hostname must resolve in DNS to public addresses only. A private or internal address, or a hostname that does not resolve, makes the submit fail with 400. If OpenRouter's DNS resolver fails while checking the hostname, the submit returns 503; retry it. OpenRouter checks the hostname again before it hands the event to its webhook delivery service, and skips the webhook if the hostname now resolves to a private address. That service handles delivery retries; these checks are not repeated on each retry or redirect. Webhooks are delivered over IPv4, so the hostname needs at least one public A record; a host with only AAAA records never receives deliveries.

OpenRouter sends one POST when the batch reaches a final status:

Event Sent when
batch.completed The batch finished. Some requests in it can still have errors.
batch.failed A check failed, or the provider rejected the batch.
batch.expired The provider did not finish before expires_at.
batch.cancelled The provider cancelled the batch.
{
  "id": "evt_51504673caddb4daeaed47e7de1dde8e",
  "object": "event",
  "type": "batch.completed",
  "created_at": 1792445705,
  "data": { "id": "batch_123" }
}
Header Value
X-OpenRouter-Signature t=<unix seconds>,v1=<hex HMAC-SHA256>. Sent only when the workspace has a webhook signing secret.
X-OpenRouter-Idempotency-Key <batch id>-<status>, for example batch_123-completed.

The event holds only the batch ID. Fetch the batch to get the status and results. A repeated delivery of the same event has the same id and idempotency key.

To get a signing secret, open your workspace settings and select Generate signing secret under Webhook signing secret. Video generation webhooks use the same secret.

import crypto from 'node:crypto';
import express from 'express';

const app = express();
const signingSecret = process.env.OPENROUTER_WEBHOOK_SECRET;
if (!signingSecret) throw new Error('Set OPENROUTER_WEBHOOK_SECRET');

function isValidSignature(rawBody: Buffer, header: string): boolean {
  const parts = header.split(',').map((part) => part.trim());
  const timestamp = parts.find((part) => part.startsWith('t='))?.slice(2);
  const signature = parts.find((part) => part.startsWith('v1='))?.slice(3);
  if (!timestamp || !signature) return false;

  const age = Math.floor(Date.now() / 1000) - Number(timestamp);
  if (Number.isNaN(age) || Math.abs(age) > 300) return false;

  const expected = crypto
    .createHmac('sha256', signingSecret)
    .update(Buffer.concat([Buffer.from(`${timestamp},`, 'utf8'), rawBody]))
    .digest('hex');
  if (expected.length !== signature.length) return false;
  return crypto.timingSafeEqual(Buffer.from(expected), Buffer.from(signature));
}

app.post(
  '/webhooks/openrouter-batch',
  express.raw({ type: 'application/json' }),
  async (req, res) => {
    if (!isValidSignature(req.body, req.header('X-OpenRouter-Signature') ?? '')) {
      return res.sendStatus(401);
    }

    const event = JSON.parse(req.body.toString('utf8'));
    try {
      // A repeated delivery carries the same event id, so this adds no second job.
      await queue.addIfAbsent(event.id, { batchId: event.data.id });
    } catch {
      return res.sendStatus(500); // OpenRouter retries the delivery
    }
    return res.sendStatus(200);
  },
);
  1. Verify the signature against the raw body, before you parse the JSON.
  2. Put the batch ID on a durable queue in one write, keyed by the event id. The id is part of the signed body, and a repeated delivery has the same id, so a repeat adds no second job.
  3. If the queue write fails, return a non-2xx response so OpenRouter retries. A 2xx tells OpenRouter the delivery succeeded.
  4. From the queue, fetch the batch. The webhook can arrive a few seconds before GET shows the final status. If the status does not match the event yet, wait and retry.
To see a real delivery before you write a receiver, set `callback_url` to a temporary [Webhook.site](https://webhook.site) URL. Webhook.site shows the body and headers. It does not verify the signature.

Option B: Poll

Call GET /api/v1/batches/{id} until the status is final: completed, failed, expired, or cancelled. Once a minute is enough. Back off if you get 429.

If a batch the provider ran ends failed, expired, or cancelled and the provider reports fewer requests than OpenRouter accepted, request_counts.total is the accepted number and the requests the provider did not report count as failed. Older batches that have no stored accepted count keep the provider's counts.

const FINAL = new Set(['completed', 'failed', 'expired', 'cancelled']);

async function waitForBatch(id: string) {
  for (;;) {
    const response = await fetch(`https://openrouter.ai/api/v1/batches/${id}`, { headers });
    if (response.ok) {
      const batch = await response.json();
      const resultsReady = batch.status !== 'completed' || batch.results !== null;
      if (FINAL.has(batch.status) && resultsReady) return batch;
    } else if (response.status !== 429 && response.status < 500) {
      // 402, 404, and 410 do not clear by retrying.
      const detail = await response.json().then((b) => b?.error?.message).catch(() => null);
      throw new Error(detail ?? `Batch poll failed with ${response.status}`);
    } else {
      await response.body?.cancel();
    }
    const delayMs = response.status === 429 ? 120_000 : 60_000;
    await new Promise((resolve) => setTimeout(resolve, delayMs));
  }
}

The status moves through validating, in_progress, and finalizing before it reaches a final status. If you use both a webhook and polling, process each batch once.

This example waits for results on a completed batch. Do not use it as written for a Google Vertex BYOK batch submitted with an input file or table: its results stay null, so the loop never ends. For that kind of batch, stop as soon as the status is final. See BYOK.

While a batch is in_progress, request_counts.completed and request_counts.failed can rise as the provider reports finished requests. Treat them as an estimate: not every provider reports progress, the numbers can lag by several minutes, and they can briefly return to 0. The counts are final once the batch reaches a final status.

To check many batches at once, list them. List items have metadata only, and results is always null.

curl 'https://openrouter.ai/api/v1/batches?status=validating&status=in_progress&limit=100' \
  -H "Authorization: Bearer $OPENROUTER_API_KEY"
Parameter Description
limit 1 to 100. Defaults to 20.
after The previous page's last_id. Use it while has_more is true.
status Repeat to include more than one status. finalizing and cancelling are not accepted.
created_after, created_before Unix seconds, or an ISO 8601 date or datetime with an offset, for example 2026-08-20T00:00:00Z. When you set both, created_after must be earlier than created_before, or the request returns 400.
{
  "object": "list",
  "data": [
    {
      "id": "batch_123",
      "object": "batch",
      "status": "in_progress",
      "model": "anthropic/claude-haiku-5.5",
      "provider": "Anthropic",
      "request_counts": { "total": 2, "completed": 1, "failed": 0 },
      "results": null
    }
  ],
  "first_id": "batch_123",
  "last_id": "batch_123",
  "has_more": false
}

Batches are listed newest first. Every API key in a workspace sees the same batches, including batches submitted with other keys. The exception is a key created by a member who is limited to their own activity in that workspace, or by someone who has since left the organization: that key lists, retrieves, and deletes only the batches attributed to its creator, and other batches in the workspace return 404. If you pass one of those other batches as after, the list returns 400; start again from the first page.

Page with `after` only; offset and `before` are unsupported. Use the ID cursor rather than timestamps: `created_at` has whole-second precision, while `created_after` preserves subsecond precision. A batch created later in the same second can still match `created_after=`.

Read the results

When the status is completed, results holds one entry per request. There is no separate download endpoint. The one exception is a Google Vertex BYOK batch submitted with an input file or table: its results stay null and, when Vertex reported an output location, output.uri names the output folder in your bucket or the output table in your dataset. See BYOK. Each entry has either a response or an error. Results are not in input order, so match them on custom_id.

const batch = await waitForBatch(batchId);
if (batch.status !== 'completed') {
  throw new Error(batch.error?.message ?? `Batch ended as ${batch.status}`);
}

const retry: string[] = [];
for (const result of batch.results) {
  if (result.error) {
    retry.push(result.custom_id);
    continue;
  }
  if (result.response.body.status !== 'completed') {
    retry.push(result.custom_id);
    continue;
  }
  const message = result.response.body.output.find(
    (item: { type: string }) => item.type === 'message',
  );
  const text = message?.content.find(
    (part: { type: string; text?: string }) => part.type === 'output_text',
  )?.text;
  if (text === undefined) {
    retry.push(result.custom_id);
    continue;
  }
  save(result.custom_id, text);
}
console.log(`Cost $${batch.usage.cost}. ${retry.length} requests to retry.`);
{
  "id": "batch_123",
  "object": "batch",
  "status": "completed",
  "provider": "Anthropic",
  "request_counts": { "total": 2, "completed": 1, "failed": 1 },
  "usage": {
    "prompt_tokens": 40,
    "completion_tokens": 12,
    "total_tokens": 52,
    "cost": 0.000005,
    "is_byok": false
  },
  "results": [
    {
      "id": "batch_req_1",
      "custom_id": "review-1",
      "response": {
        "status_code": 200,
        "request_id": "request_1",
        "body": {
          "id": "gen-batch-1782097200-a1b2c3d4e5f6a7b8c9d0",
          "object": "response",
          "model": "anthropic/claude-haiku-5.5",
          "status": "completed",
          "output": [
            {
              "type": "message",
              "id": "msg_abc123",
              "status": "completed",
              "role": "assistant",
              "content": [{ "type": "output_text", "text": "positive", "annotations": [] }]
            }
          ]
        }
      },
      "error": null
    },
    {
      "id": "batch_req_2",
      "custom_id": "review-2",
      "response": null,
      "error": {
        "type": "invalid_request_error",
        "message": "max_tokens is greater than the model limit",
        "param": "max_tokens",
        "error_type": "invalid_request"
      }
    }
  ],
  "error": null
}
{
  "id": "batch_456",
  "object": "batch",
  "status": "failed",
  "request_counts": { "total": 2, "completed": 0, "failed": 0 },
  "usage": null,
  "results": null,
  "error": {
    "message": "Batch request 'review-2' contains unsupported 'input_audio' content."
  }
}

Each response.body has the same shape as the sync endpoint's response. Its id is the generation ID. To report a bad generation, submit that ID through Report Feedback.

OpenRouter deletes requests and results 30 days after you submit. After that, `GET` on the batch returns `410 Gone`. Save the results you need before then. A Google Vertex BYOK batch submitted with an `input` file or table is the exception: OpenRouter serves no results for it, so `GET` keeps returning the batch and, when Vertex reported one, its `output` location. See [BYOK](#byok).

Delete the batch

To remove the stored requests and results sooner, delete the batch after you save the results. You can only delete a finished batch. A batch that is still running returns 409. There is no cancel endpoint yet.

curl -X DELETE https://openrouter.ai/api/v1/batches/batch_123 \
  -H "Authorization: Bearer $OPENROUTER_API_KEY"
{
  "id": "batch_123",
  "object": "batch",
  "deletion": {
    "openrouter": "deleted",
    "upstream": { "provider": "Anthropic", "status": "deleted" }
  }
}
deletion.upstream.status Meaning
deleted The provider's batch record was deleted too. Anthropic, Fireworks, Google AI Studio, Google Vertex, and Mistral support this.
unsupported The provider has no batch-delete API.
not_applicable The batch never reached the provider.

OpenRouter also deletes provider-side files, whatever upstream.status says:

Provider Files deleted
OpenAI, Mistral, DeepInfra, Morph, Alibaba, xAI Uploaded input, output, and error files
Fireworks Input and output datasets
Google AI Studio Uploaded input and output files
Together Uploaded input. Together does not allow deleting generated output.
Google Vertex Output in OpenRouter-owned storage. Files in your own GCP bucket (BYOK) stay under your control.
Anthropic, Sail Research None. These providers take inline requests, so there are no files.

This includes input left behind by a submission that failed.

upstream is left out when no provider was assigned. A 200 means every cleanup step finished, and a later GET returns 404. A 5xx means cleanup stopped part-way. Send the same request again to resume.

Deleting a batch does not refund it. Billing, generation, and audit records stay. To delete a BYOK batch, the provider key it ran on must still be enabled. Otherwise the request returns 409.


Choose a provider

A batch runs on one provider. By default, OpenRouter picks the cheapest provider that serves the model in batch mode, after your account's provider allowlist, model author settings, data policy, guardrails, and BYOK keys apply. A model whose author your account blocks is rejected at submit with 404, the same as the sync API. When your account has a nonempty list of allowed authors, a model whose author is not on that list is rejected the same way; an empty list allows every author. Guardrail model lists match a batch model by its base model, so allowing openai/gpt-5-nano also allows openai/gpt-5-nano:batch. Providers at the same price share traffic, and a BYOK key is preferred over cheaper platform providers. If the chosen provider has no capacity, OpenRouter moves the batch to the next cheapest allowed provider and bills that provider's price.

OpenRouter can also spread batches across the providers priced close to the cheapest one, so that one provider does not take every batch. By default, a provider qualifies when its price is at most twice the cheapest allowed provider's price, compared on a typical batch mix of five prompt tokens to one completion token, and when each other price that both providers charge (such as per request, per image, or for cached tokens) is inside the same limit at its highest tier. Among those providers, a cheaper one usually gets more batches, and a provider that already holds many requests in batches in progress usually gets fewer. You pay the price of the provider that runs the batch, shown as provider on the batch. The limit applies to that typical mix, not to each batch, so a batch with far more completion tokens than the mix can cost more than twice the cheapest provider's price. This never applies to batches on your own provider key (BYOK), and a batch whose provider.only leaves one provider stays on it. A model with one provider in that price range starts on that provider, and the share counts the requests a provider holds, not how its batches ended.

OpenRouter can instead keep every batch on the cheapest provider in that price range while that provider is healthy, and move new batches to the next cheapest one in the range only while it is not. A provider counts as unhealthy when the oldest batch it still holds was accepted several hours ago, or when a large share of the requests it recently finished failed. The failure check applies only after enough of the provider's requests have recently finished, whether they came from one batch or several. The move is gradual, each more expensive provider gets the same check, and when no provider in the range passes, the batch starts on the one that is least behind: the provider whose oldest batch still in progress is the newest, and the cheapest of those when several are equal. When that age is not known for a provider in the range, the batch starts on the cheapest one. A batch that a provider has already accepted is not moved. Spreading and this health-based routing are both off by default. OpenRouter turns on one of them in its routing configuration, and you can't select either on a request.

Option Batch What it does
provider.only Supported Use only these provider slugs, for example ["google-vertex"].
provider.allow_fallbacks Supported Set to false to keep the batch on the provider chosen at submit. If that provider has no capacity, or can't take the batch's inline files, the batch fails.
provider.order Rejected Put only the provider you want in only.
provider.sort Rejected Batch always routes by price, as described above.
Other sync preferences Rejected Remove them.
{
  "endpoint": "/v1/chat/completions",
  "model": "google/gemini-3.8-flash",
  "provider": { "only": ["google-vertex"], "allow_fallbacks": false },
  "requests": [
    {
      "custom_id": "ticket-1",
      "body": {
        "messages": [{ "role": "user", "content": "Summarize this ticket in one sentence." }]
      }
    }
  ]
}

Set provider at the top level of the batch, next to model. It applies to every request. A provider field inside a request body does not pin the batch, so a chat batch with provider only in its bodies runs on the default provider.

If none of the providers in only serves the model in batch mode, the submit returns 404. Embeddings batches do not accept provider.


Use other endpoints

Set endpoint to choose the request shape for every body in the batch. To mix shapes, submit separate batches.

{
  "endpoint": "/v1/messages",
  "model": "anthropic/claude-haiku-5.5",
  "requests": [
    {
      "custom_id": "msg-1",
      "body": {
        "max_tokens": 64,
        "messages": [{ "role": "user", "content": "Say hello." }]
      }
    }
  ]
}
{
  "endpoint": "/v1/chat/completions",
  "model": "anthropic/claude-haiku-5.5",
  "requests": [
    {
      "custom_id": "chat-1",
      "body": {
        "messages": [{ "role": "user", "content": "Say hello." }]
      }
    }
  ]
}
{
  "endpoint": "/v1/embeddings",
  "model": "openai/text-embedding-3-small",
  "requests": [
    {
      "custom_id": "emb-1",
      "body": {
        "input": [
          "The quick brown fox jumped over the lazy dog.",
          "Pack my box with five dozen liquor jugs."
        ]
      }
    }
  ]
}

For embeddings, input is a string, an array of strings, a token array, or an array of token arrays. Multimodal input, input_type, and provider are not supported on embeddings batches.

Each result body is a standard embeddings response, with one embedding per input string, ordered by index:

{
  "id": "batch_req_emb_1",
  "custom_id": "emb-1",
  "response": {
    "status_code": 200,
    "request_id": "request_456",
    "body": {
      "object": "list",
      "data": [
        { "object": "embedding", "embedding": [0.0023064255, -0.009327292, 0.015797347], "index": 0 },
        { "object": "embedding", "embedding": [-0.012282, 0.0034567, -0.0089123], "index": 1 }
      ],
      "model": "openai/text-embedding-3-small",
      "usage": { "prompt_tokens": 18, "total_tokens": 18 }
    }
  },
  "error": null
}

BYOK

If you have a provider key, batches route through it the same as sync requests. The provider bills you for inference, and OpenRouter charges only the BYOK fee. Completed batches report usage.is_byok: true and the provider's inference estimate in usage.cost_details.upstream_inference_cost.

When the BYOK-only switch is on for your account or workspace, batches run only on your own provider keys:

  • Keys marked as fallback-only count, the same as sync requests.
  • If none of your keys covers the model, the submit fails with 404 instead of 202 Accepted.
  • Keys for providers that can't run a batch on your own key (Fireworks, Mistral, Sail Research, and Claude Platform on AWS) don't count.
Vertex batches use a bucket in your GCP project for input and output. Leave out `bucket`, and OpenRouter creates a private bucket on the first batch in each region. You can also give an existing bucket. See [Google Vertex API keys](/guides/overview/auth/byok#google-vertex-api-keys) for permissions.
Accounts that OpenRouter has enabled for it can name an input file that is already in their own bucket, in place of `requests`: `"input": { "uri": "gs://your-bucket/batches/input.jsonl" }`. Send exactly one of `input` or `requests`. The URI names one object: no wildcard and no folder. Vertex reads the file directly as the job input. OpenRouter does not copy, convert, or read its contents, so the file must meet these rules:

- It is in Vertex's own Gemini batch input format (one JSON object per line with a `request`), not the `{ custom_id, body }` format of `requests`.
- Every line has a unique `custom_id` that does not look like a number. Vertex rewrites numeric-looking values (`"000042"` comes back as `42`), and OpenRouter cannot prefix IDs in a file it does not write.
- Your project's Vertex AI service agent (`service-<PROJECT_NUMBER>@gcp-sa-aiplatform.iam.gserviceaccount.com`) can read the object. Google's batch guide names that agent as the reader of a Cloud Storage input file, and it is the same one that needs access to your batch bucket. If Vertex fails the job with a permission error on the input file, OpenRouter reports only the error code, not Google's message. Read the job's error in your own Google Cloud project to see which identity needs read access.

- The service account of your provider key can read the object's metadata (`storage.objects.get`, for example through `roles/storage.objectViewer` on the bucket). OpenRouter uses it to check, before the job is created, that the object exists and is within Google's size limit for a Cloud Storage batch input file (1 GB when this page was written).
- The batch uses the chat completions, responses, or messages endpoint. Other endpoints return `400` at submit.
- The object is not under `openrouter-batch/input/` in your batch bucket. OpenRouter writes its own input files there, and a batch that names an object under it ends `failed` without creating a Vertex job.
- The object stays unchanged and readable until the batch ends. OpenRouter checks only its size and passes the URI on, so Vertex reads whatever the object holds when the job starts. An auto-provisioned batch bucket deletes objects 30 days after they are created, so an input file kept there can be deleted before you submit it.

After `202 Accepted`, the batch ends `failed` without creating a Vertex job when the object is missing, your key cannot read it, or it is over the size limit. `error.message` says which. Fix the file or its permissions and submit again.

OpenRouter does not know how many requests the file holds, so `request_counts.total` is `0` when the batch is accepted and stays `0` while it runs. The counts come from Vertex when the job ends. At submit, the batch adds nothing toward the limits on requests per batch and requests in progress. The limit on batches in progress applies. Google also limits how many requests one job can hold. OpenRouter does not count the lines, so Vertex applies that limit, not the submit.

Vertex writes the output under `openrouter-batch/output/` in your batch bucket, the same place as for other Vertex BYOK batches.

The same field takes a BigQuery table: `"input": { "uri": "bq://your-project.your_dataset.your_table" }`. The URI names one table by project, dataset, and table: no wildcard. The same account enablement, the same three endpoints (chat completions, responses, and messages), and the same account-level limits apply as for an input file. The project ID, dataset ID, and table ID can hold only the characters Google documents for unquoted names: a table name with other Unicode characters is refused. These rules replace the file rules above:

- The table is in Vertex's own Gemini batch table format (a `request` column per row), with a `custom_id` column that is unique per row.
- The table's dataset is in the region the batch job runs in, and is not a multi-region dataset. Google requires this for a table job. When the dataset is in a different single region, the batch ends `failed` without creating a Vertex job. Other mismatches fail in Vertex.
- The service account of your provider key can read the table's metadata (`bigquery.tables.get`). OpenRouter uses it to check, before the job is created, that the table exists and that the row count BigQuery reports for it is within Google's limit on requests in one batch job (200,000 rows when this page was written). A table that is missing, unreadable, or reports more rows than the limit ends the batch `failed` without creating a Vertex job. BigQuery does not report a count for every table type and does not count rows still in the streaming buffer, so Vertex's own limit still applies.
- Vertex writes the output to a new table named `openrouter_<batch id>` in the same project and dataset as the input table, with the dashes of the batch ID written as underscores. You do not name the output table, and each batch gets its own. The identity Vertex runs the job as must be able to create that table and write to it, and the service account of your provider key must be able to read it. That service account also needs `bigquery.tables.get` on the dataset, because OpenRouter looks up the output table name before the job is created. If a table with that name already exists, the batch ends `failed` without creating a Vertex job.
- The table stays unchanged and readable until the batch ends. OpenRouter checks only its metadata and passes the URI on, so Vertex reads whatever rows the table holds when the job starts.
- OpenRouter does not delete the output table.

No batch has been run against a real BigQuery table yet, so the permissions above are what the code needs, not a tested setup. The same holds for what this section says about billing and the output location of a table batch.

When the Vertex job ends, OpenRouter reads the job output with your provider key, stores a copy, and bills from that copy. For an input table, the output it reads is the output table, one page of rows at a time. Once the copy is recorded, OpenRouter does not read your bucket or your table again for this batch. If an attempt fails before then, the next attempt reads the output again. OpenRouter keeps the copy for 30 days as the record of what it billed. Vertex writes each request back on its output line, so the copy of an output file holds your requests as well as the answers. The copy of an output table holds each row's `custom_id`, `response`, and `status`, and no other column. Each output line or table row is billed once, from the token counts in its own `usageMetadata`, with the same BYOK fee as any other BYOK batch:

- A line or row with a complete answer is billed whether or not it has a `custom_id`. Lines or rows that repeat a `custom_id` are each billed.
- A line or row is not billed when it carries a Vertex error, is not valid JSON, has no `usageMetadata`, or has no finish reason. A line that Vertex blocked, at the prompt or with a finish reason such as `SAFETY` or `RECITATION`, is not billed either, even when it reports token counts.

OpenRouter does not have the requests, so it cannot estimate tokens for a line or row that reports none, and activity for these batches carries no `user` value.

OpenRouter does not serve the results of these batches. When the batch has ended, `GET /api/v1/batches/{id}` returns `"results": null` and, when Vertex reported an output location, an `output` field that names it. For an input file that is a Cloud Storage folder:

```json title="Completed batch submitted with an input file"
{
  "id": "batch_123",
  "status": "completed",
  "results": null,
  "output": {
    "uri": "gs://your-bucket/openrouter-batch/output/org_abc/batch_123/batch_123/prediction-model-2026-01-01T00:00:00.000000Z"
  }
}
```

Read the `predictions` JSONL files in that folder from your bucket, as you would for a job you ran on Vertex yourself. The lines are in Vertex's own batch output format. OpenRouter does not convert them to the `results` format, does not restore the order of your input lines, and adds no generation IDs. For a batch Vertex completed, `output` appears only once billing has read the output, so the batch can report `finalizing` for a short time first. A `failed`, `cancelled`, or `expired` batch also names the folder when Vertex reported one. That folder may hold rows Vertex exported before the job ended, and it can be empty. OpenRouter does not read or bill those rows. `output` is left out of batch lists.

For an input table, `output.uri` names the output table instead:

```json title="Completed batch submitted with an input table"
{
  "id": "batch_123",
  "status": "completed",
  "results": null,
  "output": {
    "uri": "bq://your-project.your_dataset.openrouter_batch_123"
  }
}
```

Query that table in your own project. Its rows are in Vertex's own batch output format, with the columns of your input table plus `response` and `status`. The same rules apply as for a folder: `output` appears on a completed batch only once billing has read the table, and a `failed`, `cancelled`, or `expired` batch names the table only when Vertex reported output for the job. Rows in the table of a batch that did not complete are not read or billed.

The output folder or table is yours. OpenRouter never deletes it: not when you [delete the batch](#delete-the-batch), and not after 30 days. Deleting the batch removes the Vertex job and OpenRouter's billing copy of the output, and nothing in your bucket or dataset. If the folder is in a bucket that OpenRouter created for your key, that bucket's own rule deletes objects 30 days after they are written, so copy what you need before then. Do not change that rule: OpenRouter checks it and refuses to use the bucket without it. To keep output longer, set your own bucket on the key. The status response keeps naming the folder after the files are gone.

Input files and tables are not enabled for any account yet.

OpenRouter does not run [content screening](#common-errors) or moderation on a batch submitted this way, because it never reads the file's lines. The per-request checks that `requests` gets at submit do not run either: Vertex decides what it accepts.

Account-level limits still apply. The batch ends `failed` without creating a Vertex job when your account cannot run inference, cannot send image inputs, or must have its input moderated or its image inputs screened, and when your account is banned from the model or its author. A model whose author your privacy settings block, or leave out of a nonempty allowed-authors list, is rejected at submit with `404` instead, so no batch is created. A provider block in your settings does not apply to your own provider key, as on every BYOK request. OpenRouter cannot tell whether the file holds images, so it treats every input file as if it does.

Vertex BYOK also accepts `gs://your-bucket/image.jpg` image URLs. Your project's Vertex AI service agent must be able to read the image until the batch finishes. Image names must end in `.png`, `.jpg`, `.jpeg`, `.webp`, `.heic`, or `.heif`.

Vertex BYOK accepts PDFs and text files in Cloud Storage the same way, and image, audio, or video files for accounts OpenRouter has enabled for that media type. Put the `gs://your-bucket/report.pdf` URI where a file URL goes: `file_url` on an `input_file` part, `file_data` on a `file` part, or the `url` of a `document` source. The file must be in a user message and the model must list file input. Images, audio, and video also need the model to list that input type; other accounts get `gs:// image files are not enabled for this account in batch` (or `audio` or `video`). The object name must end in one of these extensions, upper or lower case: `.pdf`, `.txt`, `.csv`, `.md`, `.html`, `.png`, `.jpg`, `.jpeg`, `.webp`, `.heic`, `.heif`, `.wav`, `.mp3`, `.m4a`, `.aac`, `.ogg`, `.flac`, `.weba`, `.mp4`, `.mpeg`, `.mpg`, `.mov`, `.avi`, `.flv`, `.webm`, `.wmv`, or `.3gp`. Inline base64 audio and video stay unsupported. Other file types and extensions, such as JSON, Word, GIF, or `.log`, and `gs://` files on batches billed through OpenRouter, are rejected. OpenRouter's known-CSAM screening does not read the contents of `gs://` references, whether sent as image URLs or file parts. OpenRouter has confirmed that Vertex reads `gs://` PDF, text, audio, and video files on its synchronous API, but not yet inside a batch job, and has not confirmed `gs://` image files inside a batch job either, so try a small batch first.

Google reads the file as your project's Vertex AI service agent (`service-PROJECT_NUMBER@gcp-sa-aiplatform.iam.gserviceaccount.com`), not as the service account in your key. Grant that agent `roles/storage.objectViewer` on the file's bucket and keep the grant until the batch finishes, or the rows that reference the file fail.
An Anthropic key declared for the US [data region](/guides/overview/auth/byok) sends `inference_geo: "us"` on Claude 4.6 and later models, the same as sync requests. The key's `endpoint_url` must be a DashScope OpenAI-compatible base that ends in `/v1`, for example `https://dashscope.aliyuncs.com/compatible-mode/v1`. Other URLs return `400` at submit. A key that contains characters that cannot go in an HTTP header, such as a Unicode dash or an invisible character from copy and paste, returns `400` at submit. Fix the key in your provider key settings and resubmit. Google Vertex service-account JSON keys are exempt.

Common errors

Status Cause Fix
400 Bad input, requests before other fields, a private or unresolvable callback_url, or a model with no batch variant. Read error.message. Pick a model from the batch models list.
400 input was sent, but the batch did not resolve to a Google Vertex endpoint on your own provider key. Add a Vertex provider key that covers the model. If you also have a key for another provider that serves the model, pin the batch with provider: { only: ["google-vertex"] }. Otherwise send requests.
402 The estimated cost is more than your balance. Add credits, or split the batch.
403 input was sent, and input files are not enabled for your account. Send requests.
404 No provider in provider.only, your account's allowlist, or your guardrails serves the model in batch mode, or every batch endpoint for the model requires your own provider key and no eligible key is configured (the error names those providers). Change provider.only, your provider settings, or the guardrail on your key; add a provider key; or pick a model with a platform-billed batch endpoint.
409 You tried to delete a batch that is still running. Wait until it finishes.
410 The batch is older than 30 days, so its results were deleted. Results cannot be recovered.
413 The batch has more requests than the per-batch limit, or is larger than 200 MB. error.message states the limit. Split the batch, or use file URLs instead of inline files.
422 The model targets the :online variant, or the batch uses a feature batch rejects at submit. Remove it, or use the sync API.
429 Too many submits, or too many batches or requests in progress. Wait for running batches to finish, then retry.

Content screening. A batch that names an input file or table is not screened; see BYOK. A batch can fail validation after 202 Accepted when image screening rejects a request. The batch moves to status: "failed", its error identifies the request and reports This request contains prohibited content., and no requests in that batch are submitted to the provider. This refusal is terminal; do not retry the same content. If screening is temporarily unavailable, the batch usually continues to the provider without waiting for screening. For accounts where screening is required, the batch instead stays in validating while OpenRouter retries screening; keep polling the same batch instead of submitting a replacement, which could leave both batches eligible to run. Screening one batch has a time limit: when it runs out, the remaining requests usually continue unscreened, but where screening is required the batch fails with This batch has too many images to screen within the submit window. Split it into smaller batches. and will not succeed on retry, so split it into smaller batches.