Resources

Python SDK

The launchables package for Python: install, ask, batch, handle errors.

The launchables package on PyPI is the Python client for the Decision API. It uses the standard library only and runs on Python 3.9 or later. It calls the Myles-G models over HTTP; it does not run the CPU models (see Model types).

pip install launchables

This page describes version 0.5.0, the first with batch support. The JavaScript SDK has the same calls.

Authentication

The client reads your key from the LAUNCHABLES_API_TOKEN environment variable, or you can pass it as api_key. Get a key in the console. Keep it on a server.

export LAUNCHABLES_API_TOKEN=launchables_...

Ask a question

from launchables import Launchables, noul

client = Launchables()  # reads LAUNCHABLES_API_TOKEN

resp = client.decide(
    "Customer bought the item 12 days ago and has no receipt. "
    "Policy: refunds within 30 days with proof of purchase.",
    {"refund": noul("Is a refund permitted under the policy?")},
    model="myles-g-12b",
)

answer = resp.answers["refund"]
print(answer.probability)                                  # probability of yes, e.g. 0.04
print(answer.decide(threshold_yes=0.9, threshold_no=0.1))  # "yes", "no" or "person"
print(resp.usage.input_tokens, resp.model)

decide(state, questions, model) returns a DecisionResponse with answers, usage, model and raw, the JSON as it came. The state can be a string or a dict. Always name the model, "myles-g-12b" or "myles-g-31b": every request should say which model answers it. A request without one is still answered by a default model for now, but do not rely on it.

Question types

from launchables import noul, choice, score

resp = client.decide(state, {
    "refund": noul("Is a refund permitted?", true="Policy clearly allows it", false="Policy clearly forbids it"),
    "topic": choice("What is this about?", ["billing", "returns", "other"]),       # or {label: description}
    "urgency": score("How urgent is it?", ["not urgent", "low", "medium", "high"]),  # lowest level first, numbered from 0
}, model="myles-g-12b")

resp.answers["topic"].choice          # "returns"
resp.answers["topic"].probabilities   # {"billing": 0.06, "returns": 0.89, ...}
resp.answers["urgency"].level         # 2
resp.answers["urgency"].score         # 1.92

The helpers check your questions before anything is sent and raise ValidationError if a request would be refused: at most 16 questions a request, at most 26 options a choice, and instructions that are not empty. See Question types for what each type returns.

Images

From version 0.4.0, decide takes assets, a dict of reference to image. Write the reference where the model should see the image:

from launchables import Image, noul

resp = client.decide(
    {"receipt": "receipt_photo"},
    {"total_matches": noul("Does the total printed on receipt_photo equal 23.40?")},
    model="myles-g-12b",
    assets={"receipt_photo": Image.from_path("receipt.jpg")},
    image_budget=140,  # 70, 140 or 280
)

Image.from_path and Image.from_bytes detect PNG or JPEG from the bytes and raise ValidationError for anything else or anything over 16 MiB; decide raises it for an image_budget other than 70, 140 or 280. The server's image errors arrive as InvalidRequestError with a type such as image_dimensions. See Images for references, limits and billing.

Batches

From version 0.5.0, the client sends work to the Batch API: many decisions in one submission, answered within 24 hours at half the live price per input token. Each line is a dict: a decide body with your own custom_id and the model to answer it, which every line must name. Field names are as they are sent: custom_id, state, questions, and optionally assets and image_budget.

from launchables import Launchables, choice, noul

client = Launchables()

batch = client.batch([
    {"custom_id": "ticket-1001", "model": "myles-g-12b", "state": "Charged twice for order 4471",
     "questions": {"refund": noul("Is a refund due?")}},
    {"custom_id": "ticket-1002", "model": "myles-g-31b", "state": {"message": "The app crashes on login"},
     "questions": {"team": choice("Which team?", ["billing", "returns", "technical", "other"])}},
])  # CreatedBatch(id="batch_...", status="queued", lines=2, estimated_cost=..., expires_at=..., data_region=...)

client.batches.wait_for(batch.id)  # checks after 5 s, then 1.5 times longer each time, up to a minute apart

for r in client.batches.stream_results(batch.id):
    if r.ok:
        print(r.custom_id, r.response.answers)  # a DecisionResponse, as decide() returns
    else:
        print(r.custom_id, r.error.get("type"))  # for example "expired" or "cancelled"

A result has ok, custom_id, model, status, response, error, usage and raw. response is set when the line was answered and error (a dict) when it was not. The client checks the lines before sending and raises ValidationError naming the line. If the server finds an invalid line it refuses the whole batch with an InvalidRequestError whose type is "invalid_batch", with each line at fault in body["errors"]. A submission is retried only after a refusal that stored nothing (429 rate_limited or 503 busy), never after a lost connection or a 502, so a retry cannot submit the same batch twice. A batch belongs to the key that submitted it: collect its results with that key.

Other calls

client.models()   # [Model(id="myles-g-12b", ...), Model(id="myles-g-31b", ...)]
client.health()   # {"status": "ok"}

models() and health() need no key.

Options

client = Launchables(api_key="launchables_...", timeout=10, max_retries=5)

Retries wait 0.5, 1 and 2 seconds, or as long as a Retry-After header asks, up to 60 seconds. Other errors are raised straight away.

Errors

Every error derives from LaunchablesError and carries status, message, body and type, the server's error type such as "out_of_credit".

from launchables import InvalidRequestError, LaunchablesError, OutOfCreditError, RateLimitError

try:
    client.decide(state, questions, model="myles-g-12b")
except OutOfCreditError as e:
    print("workspace budget reached" if e.type == "budget_exceeded" else "top up credit")
except RateLimitError as e:
    print("slow down", e.retry_after)
except InvalidRequestError as e:
    print(e.status, e.message)
except LaunchablesError as e:
    print("failed:", e)

To update an existing install, run pip install --upgrade launchables. Batch support needs version 0.5.0 or later; images need 0.4.0 or later. The source is at launchables-ai/launchables-sdk.