API reference

Batch API

Many decisions in one file, answered within 24 hours at half the price.

POST https://api.launchables.ai/v1/batches

The Launchables Batch API answers many decisions from one submission. You send a file with one decision per line, and collect the answers within 24 hours, at half the live price per input token. Every line is an ordinary POST /v1/decide body, with an id of your own and the model that should answer it, and every answer is exactly what /v1/decide would have returned. It uses the same API keys, the same prepaid credit and the same workspace rules as live requests.

When to use a batch

• Use a batch for work that does not need an answer straight away: a back catalogue of tickets or documents, a nightly review of the day's cases, re-checking past decisions after a policy changes, or measuring a new question against thousands of examples before you put it live.
• Use /v1/decide when someone or something is waiting for the answer. A batch can take up to 24 hours, and there is no way to hurry one line along.

Batches run on spare capacity, so live requests always come first, and a batch may take the whole window. Plan on 24 hours.

The batch file

A batch is a JSON Lines file: UTF-8 text with one JSON object per line. Each line is a /v1/decide request body plus two fields:

Blank lines are skipped. A newline inside a value is written \n in JSON, so it never breaks a line. One line of each question type:

{"custom_id": "case-0001", "model": "myles-g-12b", "state": {"policy": "Refunds within 30 days with proof of purchase.", "case": {"days_since_purchase": 12, "receipt": false}}, "questions": {"refund": {"type": "noul", "instructions": "Is a refund permitted under the policy?"}}}
{"custom_id": "case-0002", "model": "myles-g-12b", "state": "Charged twice for order 4471, need the extra payment back", "questions": {"route": {"type": "choice", "instructions": "Which team should handle this message?", "criteria": {"billing": "Payments, invoices and refunds", "returns": "Returning or exchanging an item", "technical": "The product does not work", "other": "Anything else"}}}}
{"custom_id": "case-0003", "model": "myles-g-31b", "state": "The heater in flat 4 has failed and the tenant has a newborn. Overnight frost is forecast.", "questions": {"priority": {"type": "score", "instructions": "How urgent is this repair?", "criteria": ["Routine", "Within a week", "Within 24 hours", "Emergency, attend now"]}}}

A line may ask several questions at once, of any mix of types, exactly as a /v1/decide request may. See Question types for what each type returns.

Submit a batch

Send the file itself as the request body, with Content-Type: application/jsonl. It may be compressed with Content-Encoding: gzip or zstd, as for /v1/decide, but not sent as multipart/form-data.

curl https://api.launchables.ai/v1/batches \
  -H "Authorization: Bearer $LAUNCHABLES_API_TOKEN" \
  -H "Content-Type: application/jsonl" \
  --data-binary @cases.jsonl

Use --data-binary, not -d: -d strips the newlines that separate the lines. In JavaScript (Node 18 or later):

import { readFile } from "node:fs/promises";

const res = await fetch("https://api.launchables.ai/v1/batches", {
  method: "POST",
  headers: {
    Authorization: `Bearer ${process.env.LAUNCHABLES_API_TOKEN}`,
    "Content-Type": "application/jsonl",
  },
  body: await readFile("cases.jsonl"),
});
const batch = await res.json();
if (!res.ok) throw new Error(`${res.status} ${batch.type}: ${batch.error}`);
console.log(batch.id, batch.estimated_cost);

With the SDKs, from version 0.5.0, pass the lines as objects and the client writes the file: await client.batch(lines) with the JavaScript SDK, client.batch(lines) with the Python SDK.

Every line is checked before anything is stored: that it is a JSON object with a valid and unique custom_id, names a model the key may use, has a state and 1 to 16 valid questions, and sends images only to a model that reads them. If any line is invalid, nothing is queued, and the answer lists the lines at fault (see Errors). A valid file is queued at once:

{
  "id": "batch_5f0c3a9e1b7d4c2a8e6f0b13",
  "object": "batch",
  "status": "queued",
  "lines": 3,
  "estimated_cost": { "amount": 0.00000163, "currency": "GBP", "input_tokens": 1240, "rate": 0.5, "basis": "estimate" },
  "expires_at": "2026-10-21T09:30:00+00:00",
  "data_region": "CANADA-1 (Canada)"
}

Here, about 840 estimated tokens on Myles-G 12B at £0.00075 a million and 400 on Myles-G 31B at £0.0025 a million. estimated_cost is a guide only, worked out from the size of each line before any model has read it. The bill uses the input tokens the model actually reads, line by line (see Pricing). expires_at is the end of the batch's 24-hour window.

Check a batch

curl https://api.launchables.ai/v1/batches/batch_5f0c3a9e1b7d4c2a8e6f0b13 \
  -H "Authorization: Bearer $LAUNCHABLES_API_TOKEN"

{
  "id": "batch_5f0c3a9e1b7d4c2a8e6f0b13",
  "object": "batch",
  "status": "running",
  "lines": 3,
  "counts": { "done": 2, "failed": 0, "pending": 1 },
  "models": ["myles-g-12b", "myles-g-31b"],
  "created_at": "2026-10-20T09:30:00+00:00",
  "started_at": "2026-10-20T09:31:12+00:00",
  "finished_at": null,
  "expires_at": "2026-10-21T09:30:00+00:00",
  "reason": null,
  "results_available": false,
  "webhook": false,
  "estimated_cost": { "amount": 0.00000163, "currency": "GBP", "input_tokens": 1240, "rate": 0.5, "basis": "estimate" },
  "data_region": "CANADA-1 (Canada)",
  "delete_after": null
}

Once a batch has finished, counts.failed includes the lines that were cancelled or expired, and delete_after says when it will be deleted (see Retention and deletion). In JavaScript, poll with a growing wait rather than a fixed one:

const headers = { Authorization: `Bearer ${process.env.LAUNCHABLES_API_TOKEN}` };
const url = `https://api.launchables.ai/v1/batches/${batch.id}`;

let wait = 5_000, status;
for (;;) {
  const res = await fetch(url, { headers });
  status = await res.json();
  if (!res.ok && res.status < 500 && res.status !== 429) throw new Error(`${res.status} ${status.type}: ${status.error}`);
  if (res.ok && ["done", "failed", "cancelled"].includes(status.status)) break;
  await new Promise((r) => setTimeout(r, wait));
  wait = Math.min(wait * 1.5, 60_000); // 5 s, 7.5 s, 11 s ... then once a minute
}

The SDKs do this for you: await client.batches.waitFor(batch.id) in JavaScript, client.batches.wait_for(batch.id) in Python. For a long batch, a webhook saves polling altogether.

Collect the results

curl https://api.launchables.ai/v1/batches/batch_5f0c3a9e1b7d4c2a8e6f0b13/results \
  -H "Authorization: Bearer $LAUNCHABLES_API_TOKEN" \
  -o results.jsonl

The results are a JSON Lines file (application/jsonl) with one line for every line you sent, in the order you sent them, each under its custom_id. They are ready once the batch has finished: before then, and for a few seconds after a cancel while lines already in progress come back, the answer is 409 batch_not_finished. You can collect them as often as you like until the batch is deleted.

{"custom_id": "case-0001", "model": "myles-g-12b-20261016", "status": 200, "response": {"model": "myles-g-12b-20261016", "answers": {"refund": {"type": "noul", "noul": 0.03}}, "usage": {"input_tokens": 131, "output_tokens": 1}}, "usage": {"input_tokens": 131, "output_tokens": 1, "batch": true}}
{"custom_id": "case-0002", "model": "myles-g-12b-20261016", "status": 200, "response": {"model": "myles-g-12b-20261016", "answers": {"route": {"type": "choice", "choice": "billing", "confidence": 0.89, "probabilities": {"billing": 0.93, "returns": 0.04, "technical": 0.01, "other": 0.02}}}, "usage": {"input_tokens": 162, "output_tokens": 1}}, "usage": {"input_tokens": 162, "output_tokens": 1, "batch": true}}
{"custom_id": "case-0003", "model": "myles-g-31b", "status": null, "error": {"type": "expired", "error": "this line was not run (expired)"}, "usage": {"input_tokens": 0, "output_tokens": 0, "batch": true}}

Illustrative probabilities and token counts. In JavaScript, read the file a line at a time so a large one is never held in memory at once:

import { createInterface } from "node:readline";
import { Readable } from "node:stream";

const res = await fetch(`https://api.launchables.ai/v1/batches/${batch.id}/results`, { headers });
if (!res.ok) throw new Error(`${res.status} ${(await res.json()).type}`);

for await (const text of createInterface({ input: Readable.fromWeb(res.body) })) {
  if (!text.trim()) continue;
  const line = JSON.parse(text);
  if (line.response) console.log(line.custom_id, line.response.answers);
  else console.log(line.custom_id, "not answered:", line.error.type);
}

With the SDKs: for await (const r of client.batches.streamResults(batch.id)) in JavaScript, or for r in client.batches.stream_results(batch.id) in Python, with each answer typed exactly as decide() returns it.

Cancel a batch

curl -X POST https://api.launchables.ai/v1/batches/batch_5f0c3a9e1b7d4c2a8e6f0b13/cancel \
  -H "Authorization: Bearer $LAUNCHABLES_API_TOKEN"

await fetch(`https://api.launchables.ai/v1/batches/${batch.id}/cancel`, { method: "POST", headers });

The answer is the batch's status, now cancelled. No new line starts. Lines already answered stay in the results and are billed; lines in progress at the moment of the cancel finish, come back and are billed too; every other line reads as cancelled in the results and costs nothing. Cancelling a cancelled batch is not an error. A batch that is already done or failed answers 409 batch_finished.

List your batches

curl https://api.launchables.ai/v1/batches -H "Authorization: Bearer $LAUNCHABLES_API_TOKEN"

Answers { "object": "list", "data": [...] } with the status of every batch the calling key has submitted that has not yet been deleted, oldest first.

Batches and keys

A batch belongs to the API key that submitted it. Only that key can see it, collect its results or cancel it: any other key, even one in the same organisation, gets 404 batch_not_found, exactly as for an id that does not exist.

• Rolling a key makes a new key. Collect the results of batches submitted with the old key before the old key stops working, at the end of the next day.
• Running out of credit or reaching the workspace's monthly budget pauses the key. A paused key cannot submit a batch (402), and its open batches wait, starting no new line, until it is topped up or the window ends. It can still check, collect and cancel.
• Revoking a key, or letting it expire, ends its open batches as failed with reason key_revoked. Lines already answered are billed; the rest are not. Only that key could collect the results, so collect them before you revoke it.
• Removing a model from the workspace fails that model's remaining lines with model_not_allowed, unbilled.

Webhooks

To be told when a batch finishes, send its address in the Launchables-Webhook-Url header when you submit:

curl https://api.launchables.ai/v1/batches \
  -H "Authorization: Bearer $LAUNCHABLES_API_TOKEN" \
  -H "Content-Type: application/jsonl" \
  -H "Launchables-Webhook-Url: https://hooks.example.com/launchables/batches" \
  --data-binary @cases.jsonl

When the batch finishes (done, failed or cancelled), the address receives one POST with a JSON body holding the batch's id and status, and nothing else:

{ "id": "batch_5f0c3a9e1b7d4c2a8e6f0b13", "status": "done" }

• It carries no results and is not signed. Treat it as a prompt only: fetch the status and the results yourself, with your key. Anyone who learns the address could post to it, so never act on the body alone.
• Delivery: a 2xx answer counts as delivered. Anything else, a time-out after 10 seconds or a refused connection, is tried again, up to 3 attempts in all. Redirects are not followed. If every attempt fails, the status call still tells you.
• The address must be https, up to 2,048 characters, with no user name, password or # fragment, and its host must resolve only to public internet addresses. It is checked when you submit (422 invalid_webhook_url otherwise) and again when the call is made. A query string is kept, so you can add your own reference to it.
• The address is stored encrypted, and the status shows only whether a batch has one ("webhook": true).

Limits

For more lines or more open batches, split the work into several batches, or ask us about higher limits for your organisation.

The 24-hour window

Every batch has 24 hours from submission, until its expires_at. Lines not answered by then are not run: they read as expired in the results and cost nothing. The batch then ends done if any line was answered, or failed if none was. Lines in progress at the deadline are allowed to finish. A line that hits a temporary fault is tried again with a growing wait, up to five tries in all, before it fails.

Retention and deletion

Unlike a live request, a batch has to be kept while it runs and until you collect it. We keep the least we can, for as short a time as we can:

• Encrypted at rest. Your lines, their results and any webhook address are stored encrypted (AES-256-GCM), each record bound to its batch.
• Your lines are deleted as soon as the batch finishes; only the results remain.
• The whole batch, results included, is deleted 7 days after you first collect the results, or 14 days after the batch finished, whichever comes first. The status shows the time as delete_after. Deletion runs every ten minutes, so allow a few minutes after that time. After it, the batch's id answers 404.
• What stays: the counts we bill from (lines and input tokens per key, model and day), as for live requests. They hold numbers only, never anything you sent.
• Never logged, never used for training. Our logs record the method, the path, the status and the time taken, never a line, an id you chose, an answer or a webhook address. Batch data is never used to train models.

To keep results longer, save the results file yourself. There is no call to delete a batch early today; if you need one deleted sooner, ask us with its id.

Pricing

A batch line costs half the live price of the model that answers it, per input token, with output tokens free:

Each answered line is billed on its own usage.input_tokens, at the batch price in force on the day it is answered, from the same prepaid credit as live requests. Lines that fail, expire or are cancelled cost nothing. The estimated_cost given when you submit is a guide, not a quote: the bill uses the tokens actually read. Prices exclude VAT. See Hosted API for credit and keys.

Where batch data is stored

Batch files are stored in Canada (region CANADA-1), on the servers that receive your API requests. They are not stored in the UK. Every submission and status answer names the region in data_region. Live /v1/decide requests are not stored. If your data may not be stored outside the UK, or anywhere at all, send it as live requests instead, or run Myles on your own hardware.

Errors

Errors come back as JSON with a message and a type, as for every endpoint (see Errors and limits). An invalid batch lists every line at fault, by its line number in the file (blank lines are counted), up to 100 of them, and invalid_lines gives the total:

{
  "error": "2 invalid line(s); nothing was queued",
  "type": "invalid_batch",
  "errors": [
    { "line": 4, "type": "model_required", "error": "every batch line names its \"model\"; batches have no default model" },
    { "line": 9, "type": "duplicate_custom_id", "error": "custom_id is used on an earlier line" }
  ],
  "invalid_lines": 2
}

Types you may see for a line in invalid_batch: invalid_json, invalid_line, invalid_custom_id, duplicate_custom_id, model_required, model_not_found, model_not_allowed, missing (no state), invalid_questions, too_many_questions, invalid_type, invalid_question, too_many_options, images_not_supported and the image types on the Images page.

Retry 429 rate_limited and 502 after a wait. Do not resubmit a batch after a lost connection or a 502 without first listing your batches: the first submission may have been queued, and a second would be answered and billed again. Do not retry other errors without changing the request.

Line errors

Checks at submission cover everything that can be known before a model reads the line. A line can still fail when it runs; its result then has error and costs nothing:

Paths

All of them are at https://api.launchables.ai only, not under the per-model /m/<model>/ paths: each line names its own model.