# Batch API

> The Launchables Batch API: send many decisions as one JSON Lines file and collect the answers within 24 hours at half the live price per input token, with webhooks, limits, retention and errors.

Section: API reference. Web page: https://www.launchables.ai/docs/api/batches

```
POST https://api.launchables.ai/v1/batches
```

The Launchables Batch API answers many decisions from one submission. You send a file with one decision per line, and collect the answers within 24 hours, at half the live price per input token. Every line is an ordinary [`POST /v1/decide`](https://www.launchables.ai/docs/api/decide) body, with an id of your own and the model that should answer it, and every answer is exactly what `/v1/decide` would have returned. It uses the same [API keys](https://www.launchables.ai/docs/authentication), the same prepaid credit and the same workspace rules as live requests.

## When to use a batch

- **Use a batch** for work that does not need an answer straight away: a back catalogue of tickets or documents, a nightly review of the day's cases, re-checking past decisions after a policy changes, or measuring a new question against thousands of examples before you put it live.
- **Use `/v1/decide`** when someone or something is waiting for the answer. A batch can take up to 24 hours, and there is no way to hurry one line along.

Batches run on spare capacity, so live requests always come first, and a batch may take the whole window. Plan on 24 hours.

## The batch file

A batch is a [JSON Lines](https://jsonlines.org) file: UTF-8 text with one JSON object per line. Each line is a `/v1/decide` request body plus two fields:

| Field | Type | Meaning |
|---|---|---|
| `custom_id` | text | Your id for the line: 1 to 64 letters, digits or `_ . : -`, unique within the batch. Its result comes back under the same id. |
| `model` | text | Required on every line: a batch has no default model. Any model id from [`GET /v1/models`](https://www.launchables.ai/docs/api/models) that your key's workspace may use, today `myles-g-12b` or `myles-g-31b`, or a dated build of either. Lines in one batch may name different models. |
| `state`, `questions` |  | As for [`/v1/decide`](https://www.launchables.ai/docs/api/decide): the situation, and up to 16 questions about it. |
| `assets`, `image_budget` | optional | Images, as for [Images](https://www.launchables.ai/docs/api/images), sent inline with `data_base64`. A batch file cannot carry multipart file parts. |

Blank lines are skipped. A newline inside a value is written `\n` in JSON, so it never breaks a line. One line of each question type:

```json
{"custom_id": "case-0001", "model": "myles-g-12b", "state": {"policy": "Refunds within 30 days with proof of purchase.", "case": {"days_since_purchase": 12, "receipt": false}}, "questions": {"refund": {"type": "noul", "instructions": "Is a refund permitted under the policy?"}}}
{"custom_id": "case-0002", "model": "myles-g-12b", "state": "Charged twice for order 4471, need the extra payment back", "questions": {"route": {"type": "choice", "instructions": "Which team should handle this message?", "criteria": {"billing": "Payments, invoices and refunds", "returns": "Returning or exchanging an item", "technical": "The product does not work", "other": "Anything else"}}}}
{"custom_id": "case-0003", "model": "myles-g-31b", "state": "The heater in flat 4 has failed and the tenant has a newborn. Overnight frost is forecast.", "questions": {"priority": {"type": "score", "instructions": "How urgent is this repair?", "criteria": ["Routine", "Within a week", "Within 24 hours", "Emergency, attend now"]}}}
```

A line may ask several questions at once, of any mix of types, exactly as a `/v1/decide` request may. See [Question types](https://www.launchables.ai/docs/question-types) for what each type returns.

## Submit a batch

Send the file itself as the request body, with `Content-Type: application/jsonl`. It may be compressed with `Content-Encoding: gzip` or `zstd`, as for `/v1/decide`, but not sent as `multipart/form-data`.

```bash
curl https://api.launchables.ai/v1/batches \
  -H "Authorization: Bearer $LAUNCHABLES_API_TOKEN" \
  -H "Content-Type: application/jsonl" \
  --data-binary @cases.jsonl
```

Use `--data-binary`, not `-d`: `-d` strips the newlines that separate the lines. In JavaScript (Node 18 or later):

```
import { readFile } from "node:fs/promises";

const res = await fetch("https://api.launchables.ai/v1/batches", {
  method: "POST",
  headers: {
    Authorization: `Bearer ${process.env.LAUNCHABLES_API_TOKEN}`,
    "Content-Type": "application/jsonl",
  },
  body: await readFile("cases.jsonl"),
});
const batch = await res.json();
if (!res.ok) throw new Error(`${res.status} ${batch.type}: ${batch.error}`);
console.log(batch.id, batch.estimated_cost);
```

With the SDKs, from version 0.5.0, pass the lines as objects and the client writes the file: `await client.batch(lines)` with the [JavaScript SDK](https://www.launchables.ai/docs/sdk-javascript#batches), `client.batch(lines)` with the [Python SDK](https://www.launchables.ai/docs/sdk-python#batches).

Every line is checked before anything is stored: that it is a JSON object with a valid and unique `custom_id`, names a model the key may use, has a state and 1 to 16 valid questions, and sends images only to a model that reads them. If any line is invalid, nothing is queued, and the answer lists the lines at fault (see [Errors](https://www.launchables.ai/docs/api/batches#errors)). A valid file is queued at once:

```json
{
  "id": "batch_5f0c3a9e1b7d4c2a8e6f0b13",
  "object": "batch",
  "status": "queued",
  "lines": 3,
  "estimated_cost": { "amount": 0.00000163, "currency": "GBP", "input_tokens": 1240, "rate": 0.5, "basis": "estimate" },
  "expires_at": "2026-10-21T09:30:00+00:00",
  "data_region": "CANADA-1 (Canada)"
}
```

Here, about 840 estimated tokens on Myles-G 12B at £0.00075 a million and 400 on Myles-G 31B at £0.0025 a million. `estimated_cost` is a guide only, worked out from the size of each line before any model has read it. The bill uses the input tokens the model actually reads, line by line (see [Pricing](https://www.launchables.ai/docs/api/batches#pricing)). `expires_at` is the end of the batch's 24-hour window.

## Check a batch

```bash
curl https://api.launchables.ai/v1/batches/batch_5f0c3a9e1b7d4c2a8e6f0b13 \
  -H "Authorization: Bearer $LAUNCHABLES_API_TOKEN"
```

```json
{
  "id": "batch_5f0c3a9e1b7d4c2a8e6f0b13",
  "object": "batch",
  "status": "running",
  "lines": 3,
  "counts": { "done": 2, "failed": 0, "pending": 1 },
  "models": ["myles-g-12b", "myles-g-31b"],
  "created_at": "2026-10-20T09:30:00+00:00",
  "started_at": "2026-10-20T09:31:12+00:00",
  "finished_at": null,
  "expires_at": "2026-10-21T09:30:00+00:00",
  "reason": null,
  "results_available": false,
  "webhook": false,
  "estimated_cost": { "amount": 0.00000163, "currency": "GBP", "input_tokens": 1240, "rate": 0.5, "basis": "estimate" },
  "data_region": "CANADA-1 (Canada)",
  "delete_after": null
}
```

| Status | Meaning |
|---|---|
| `queued` | Accepted, and no line has started yet. |
| `running` | Lines are being answered. `counts` shows how many are `done`, `failed` and still `pending`. |
| `done` | Finished, with at least one line answered. Some lines may still have failed or expired; each result says. |
| `failed` | Finished with no line answered, or stopped because the key that submitted it was revoked or expired. `reason` says why, for example `expired` or `key_revoked`. |
| `cancelled` | You cancelled it. Lines answered before the cancel keep their answers. |

Once a batch has finished, `counts.failed` includes the lines that were cancelled or expired, and `delete_after` says when it will be deleted (see [Retention and deletion](https://www.launchables.ai/docs/api/batches#retention)). In JavaScript, poll with a growing wait rather than a fixed one:

```
const headers = { Authorization: `Bearer ${process.env.LAUNCHABLES_API_TOKEN}` };
const url = `https://api.launchables.ai/v1/batches/${batch.id}`;

let wait = 5_000, status;
for (;;) {
  const res = await fetch(url, { headers });
  status = await res.json();
  if (!res.ok && res.status < 500 && res.status !== 429) throw new Error(`${res.status} ${status.type}: ${status.error}`);
  if (res.ok && ["done", "failed", "cancelled"].includes(status.status)) break;
  await new Promise((r) => setTimeout(r, wait));
  wait = Math.min(wait * 1.5, 60_000); // 5 s, 7.5 s, 11 s ... then once a minute
}
```

The SDKs do this for you: `await client.batches.waitFor(batch.id)` in JavaScript, `client.batches.wait_for(batch.id)` in Python. For a long batch, a [webhook](https://www.launchables.ai/docs/api/batches#webhooks) saves polling altogether.

## Collect the results

```bash
curl https://api.launchables.ai/v1/batches/batch_5f0c3a9e1b7d4c2a8e6f0b13/results \
  -H "Authorization: Bearer $LAUNCHABLES_API_TOKEN" \
  -o results.jsonl
```

The results are a JSON Lines file (`application/jsonl`) with one line for every line you sent, in the order you sent them, each under its `custom_id`. They are ready once the batch has finished: before then, and for a few seconds after a cancel while lines already in progress come back, the answer is `409` `batch_not_finished`. You can collect them as often as you like until the batch is deleted.

```json
{"custom_id": "case-0001", "model": "myles-g-12b-20261016", "status": 200, "response": {"model": "myles-g-12b-20261016", "answers": {"refund": {"type": "noul", "noul": 0.03}}, "usage": {"input_tokens": 131, "output_tokens": 1}}, "usage": {"input_tokens": 131, "output_tokens": 1, "batch": true}}
{"custom_id": "case-0002", "model": "myles-g-12b-20261016", "status": 200, "response": {"model": "myles-g-12b-20261016", "answers": {"route": {"type": "choice", "choice": "billing", "confidence": 0.89, "probabilities": {"billing": 0.93, "returns": 0.04, "technical": 0.01, "other": 0.02}}}, "usage": {"input_tokens": 162, "output_tokens": 1}}, "usage": {"input_tokens": 162, "output_tokens": 1, "batch": true}}
{"custom_id": "case-0003", "model": "myles-g-31b", "status": null, "error": {"type": "expired", "error": "this line was not run (expired)"}, "usage": {"input_tokens": 0, "output_tokens": 0, "batch": true}}
```

| Field | Meaning |
|---|---|
| `custom_id` | The id you gave the line. |
| `model` | The exact model build that answered, which may be a dated build of the model you named. For a line that was not answered, the model it named. |
| `status` | The HTTP status the line got: `200` when it was answered, the refusal's status (such as `422`) when the model could not answer it, and `null` when it never ran. |
| `response` | When answered: the `/v1/decide` response, unchanged. |
| `error` | When not answered: `{ "error", "type" }` saying why (see [Line errors](https://www.launchables.ai/docs/api/batches#line-errors)). |
| `usage` | The line's tokens, with `"batch": true`. `input_tokens` is what the line is billed on; a line that was not answered shows 0 and costs nothing. |

Illustrative probabilities and token counts. In JavaScript, read the file a line at a time so a large one is never held in memory at once:

```
import { createInterface } from "node:readline";
import { Readable } from "node:stream";

const res = await fetch(`https://api.launchables.ai/v1/batches/${batch.id}/results`, { headers });
if (!res.ok) throw new Error(`${res.status} ${(await res.json()).type}`);

for await (const text of createInterface({ input: Readable.fromWeb(res.body) })) {
  if (!text.trim()) continue;
  const line = JSON.parse(text);
  if (line.response) console.log(line.custom_id, line.response.answers);
  else console.log(line.custom_id, "not answered:", line.error.type);
}
```

With the SDKs: `for await (const r of client.batches.streamResults(batch.id))` in JavaScript, or `for r in client.batches.stream_results(batch.id)` in Python, with each answer typed exactly as `decide()` returns it.

## Cancel a batch

```bash
curl -X POST https://api.launchables.ai/v1/batches/batch_5f0c3a9e1b7d4c2a8e6f0b13/cancel \
  -H "Authorization: Bearer $LAUNCHABLES_API_TOKEN"
```

```
await fetch(`https://api.launchables.ai/v1/batches/${batch.id}/cancel`, { method: "POST", headers });
```

The answer is the batch's status, now `cancelled`. No new line starts. Lines already answered stay in the results and are billed; lines in progress at the moment of the cancel finish, come back and are billed too; every other line reads as `cancelled` in the results and costs nothing. Cancelling a cancelled batch is not an error. A batch that is already `done` or `failed` answers `409` `batch_finished`.

## List your batches

```bash
curl https://api.launchables.ai/v1/batches -H "Authorization: Bearer $LAUNCHABLES_API_TOKEN"
```

Answers `{ "object": "list", "data": [...] }` with the status of every batch the calling key has submitted that has not yet been deleted, oldest first.

## Batches and keys

A batch belongs to the API key that submitted it. Only that key can see it, collect its results or cancel it: any other key, even one in the same organisation, gets `404` `batch_not_found`, exactly as for an id that does not exist.

- **Rolling a key** makes a new key. Collect the results of batches submitted with the old key before the old key stops working, at the end of the next day.
- **Running out of credit** or reaching the workspace's monthly budget pauses the key. A paused key cannot submit a batch (`402`), and its open batches wait, starting no new line, until it is topped up or the window ends. It can still check, collect and cancel.
- **Revoking a key**, or letting it expire, ends its open batches as `failed` with `reason` `key_revoked`. Lines already answered are billed; the rest are not. Only that key could collect the results, so collect them before you revoke it.
- **Removing a model** from the workspace fails that model's remaining lines with `model_not_allowed`, unbilled.

## Webhooks

To be told when a batch finishes, send its address in the `Launchables-Webhook-Url` header when you submit:

```bash
curl https://api.launchables.ai/v1/batches \
  -H "Authorization: Bearer $LAUNCHABLES_API_TOKEN" \
  -H "Content-Type: application/jsonl" \
  -H "Launchables-Webhook-Url: https://hooks.example.com/launchables/batches" \
  --data-binary @cases.jsonl
```

When the batch finishes (`done`, `failed` or `cancelled`), the address receives one `POST` with a JSON body holding the batch's id and status, and nothing else:

```json
{ "id": "batch_5f0c3a9e1b7d4c2a8e6f0b13", "status": "done" }
```

- **It carries no results and is not signed.** Treat it as a prompt only: fetch the status and the results yourself, with your key. Anyone who learns the address could post to it, so never act on the body alone.
- **Delivery:** a `2xx` answer counts as delivered. Anything else, a time-out after 10 seconds or a refused connection, is tried again, up to 3 attempts in all. Redirects are not followed. If every attempt fails, the status call still tells you.
- **The address** must be `https`, up to 2,048 characters, with no user name, password or `#` fragment, and its host must resolve only to public internet addresses. It is checked when you submit (`422` `invalid_webhook_url` otherwise) and again when the call is made. A query string is kept, so you can add your own reference to it.
- The address is stored encrypted, and the status shows only whether a batch has one (`"webhook": true`).

## Limits

| Limit | Value |
|---|---|
| Lines per batch | 50,000 |
| Batch file | 32 MiB as sent, and 32 MiB once decompressed |
| Open batches per key | 10 queued or running at once |
| Each line | The limits of `/v1/decide`: up to 16 questions, 26 options a question, images as on the [Images](https://www.launchables.ai/docs/api/images) page, and the model's context window |
| Time to answer | 24 hours from submission (see [The 24-hour window](https://www.launchables.ai/docs/api/batches#window)) |
| Rate limits | Batch lines do not count against your key's requests per minute or any daily cap. The batch file's bytes do count against the key's 256 MiB a minute of request bytes. |

For more lines or more open batches, split the work into several batches, or [ask us](https://www.launchables.ai/contact?about=api) about higher limits for your organisation.

## The 24-hour window

Every batch has 24 hours from submission, until its `expires_at`. Lines not answered by then are not run: they read as `expired` in the results and cost nothing. The batch then ends `done` if any line was answered, or `failed` if none was. Lines in progress at the deadline are allowed to finish. A line that hits a temporary fault is tried again with a growing wait, up to five tries in all, before it fails.

## Retention and deletion

Unlike a live request, a batch has to be kept while it runs and until you collect it. We keep the least we can, for as short a time as we can:

- **Encrypted at rest.** Your lines, their results and any webhook address are stored encrypted (AES-256-GCM), each record bound to its batch.
- **Your lines** are deleted as soon as the batch finishes; only the results remain.
- **The whole batch**, results included, is deleted 7 days after you first collect the results, or 14 days after the batch finished, whichever comes first. The status shows the time as `delete_after`. Deletion runs every ten minutes, so allow a few minutes after that time. After it, the batch's id answers `404`.
- **What stays:** the counts we bill from (lines and input tokens per key, model and day), as for live requests. They hold numbers only, never anything you sent.
- **Never logged, never used for training.** Our logs record the method, the path, the status and the time taken, never a line, an id you chose, an answer or a webhook address. Batch data is never used to train models.

To keep results longer, save the results file yourself. There is no call to delete a batch early today; if you need one deleted sooner, [ask us](https://www.launchables.ai/contact?about=api) with its id.

## Pricing

A batch line costs half the live price of the model that answers it, per input token, with output tokens free:

| Model | Live, per million input tokens | Batch, per million input tokens |
|---|---|---|
| Myles-G 12B `myles-g-12b` | £0.0015 | £0.00075 |
| Myles-G 31B `myles-g-31b` | £0.005 | £0.0025 |

Each answered line is billed on its own `usage.input_tokens`, at the batch price in force on the day it is answered, from the same prepaid credit as live requests. Lines that fail, expire or are cancelled cost nothing. The `estimated_cost` given when you submit is a guide, not a quote: the bill uses the tokens actually read. Prices exclude VAT. See [Hosted API](https://www.launchables.ai/docs/deploy/hosted) for credit and keys.

## Where batch data is stored

Batch files are stored in **Canada** (region CANADA-1), on the servers that receive your API requests. They are not stored in the UK. Every submission and status answer names the region in `data_region`. Live `/v1/decide` requests are not stored. If your data may not be stored outside the UK, or anywhere at all, send it as live requests instead, or run Myles [on your own hardware](https://www.launchables.ai/docs/deploy/self-hosting).

## Errors

Errors come back as JSON with a message and a type, as for every endpoint (see [Errors and limits](https://www.launchables.ai/docs/api/errors)). An invalid batch lists every line at fault, by its line number in the file (blank lines are counted), up to 100 of them, and `invalid_lines` gives the total:

```json
{
  "error": "2 invalid line(s); nothing was queued",
  "type": "invalid_batch",
  "errors": [
    { "line": 4, "type": "model_required", "error": "every batch line names its \"model\"; batches have no default model" },
    { "line": 9, "type": "duplicate_custom_id", "error": "custom_id is used on an earlier line" }
  ],
  "invalid_lines": 2
}
```

| Status | Meaning |
|---|---|
| `400` | Type `invalid_jsonl`: the file is not UTF-8 text. Type `empty_batch`: it has no lines. Also `bad_encoding` for a compressed body that does not decompress. |
| `401` | Missing, wrong, expired or revoked API token. |
| `402` | Type `out_of_credit` or `budget_exceeded`: the key is paused, so it cannot submit a batch. It can still check, collect and cancel. |
| `404` | Type `batch_not_found`: no batch with that id belongs to this key, or it has been deleted. |
| `405` | Type `method_not_allowed`: cancel with `POST`, not `GET`. |
| `409` | Type `batch_not_finished`: results asked for before the batch has finished. Type `batch_finished`: a cancel for a batch already `done` or `failed`. |
| `411` | No `Content-Length` header. |
| `413` | Type `too_many_lines`: more lines than the key may send in one batch (`max_lines` gives the limit). Type `body_too_large`: over 32 MiB, as sent or once decompressed. |
| `415` | Type `unsupported_media_type`: a multipart body; send the file itself. Type `unsupported_encoding`: a `Content-Encoding` other than `gzip` or `zstd`. |
| `422` | Type `invalid_batch`: one or more lines are invalid, listed in `errors`; nothing was queued. Type `invalid_webhook_url`: the webhook address is refused. |
| `429` | Type `too_many_batches`: the key already has 10 batches queued or running; try again when one finishes (`Retry-After` suggests 300 seconds). Type `rate_limited`: over the key's request bytes a minute. |
| `502` | The endpoint did not answer in time. The request may still have been received: see below before resubmitting a batch. |
| `503` | Type `batch_unavailable`: the Batch API is not available on this endpoint. A self-hosted endpoint has it only when it is set up for it. |

Types you may see for a line in `invalid_batch`: `invalid_json`, `invalid_line`, `invalid_custom_id`, `duplicate_custom_id`, `model_required`, `model_not_found`, `model_not_allowed`, `missing` (no state), `invalid_questions`, `too_many_questions`, `invalid_type`, `invalid_question`, `too_many_options`, `images_not_supported` and the image types on the [Images](https://www.launchables.ai/docs/api/images) page.

Retry `429` `rate_limited` and `502` after a wait. Do not resubmit a batch after a lost connection or a `502` without first listing your batches: the first submission may have been queued, and a second would be answered and billed again. Do not retry other errors without changing the request.

### Line errors

Checks at submission cover everything that can be known before a model reads the line. A line can still fail when it runs; its result then has `error` and costs nothing:

| Type | Meaning |
|---|---|
| a refusal type, such as `invalid_question` | The model could not answer the line, for example because the state and question are longer than the model's context window. `status` is the refusal's status. The line would have been refused as a live request too. |
| `expired` | Not answered within the 24-hour window. |
| `cancelled` | The batch was cancelled before the line ran. |
| `key_revoked` | The key that submitted the batch was revoked or expired. |
| `model_not_allowed` | The key's workspace may no longer use the line's model. |
| others | The model could not answer after five tries, for example `model_unavailable`. Submit the line again in a new batch. |

## Paths

| Request | What it does |
|---|---|
| `POST /v1/batches` | Submit a batch file. |
| `GET /v1/batches` | List the key's batches. |
| `GET /v1/batches/<id>` | A batch's status. |
| `GET /v1/batches/<id>/results` | Its results, once finished. |
| `POST /v1/batches/<id>/cancel` | Cancel it. |

All of them are at `https://api.launchables.ai` only, not under the per-model `/m/<model>/` paths: each line names its own model.