launchables°

Examples

Supported claims

Check each claim in a draft against its sources.

A draft answer, summary or letter has been written from source documents. Before it goes out, check that each claim in it is supported by the sources. Split the draft into claims in your own code, then ask one yes/no question per claim in one request (up to 16).

The request

{
  "model": "myles-g-31b",
  "state": {
    "sources": "Order 4471 was placed on 3 March and delivered on 6 March. The customer was charged 49.00 twice on 3 March. One charge was refunded on 10 March.",
    "draft": "Your order was delivered on 6 March. You were charged twice. Both charges have now been refunded."
  },
  "questions": {
    "claim_1": {"type": "noul", "instructions": "Do the sources support this claim: the order was delivered on 6 March?"},
    "claim_2": {"type": "noul", "instructions": "Do the sources support this claim: the customer was charged twice?"},
    "claim_3": {"type": "noul", "instructions": "Do the sources support this claim: both charges have been refunded?"}
  }
}

Reading the answer

"claim_1": {"type": "noul", "noul": 0.98},
"claim_2": {"type": "noul", "noul": 0.97},
"claim_3": {"type": "noul", "noul": 0.03}

Claim 3 is not supported: only one charge was refunded. Hold the draft, correct it or send it to a person, and show them which claim failed.

What to watch

  • Release a draft only when every claim is above your threshold. One weak claim is enough to stop it.
  • Claims that need arithmetic (totals, dates, counts) are best checked in your own code first; see Keep arithmetic and dates in your code.
  • Long sources cost more: send the passages the draft was written from, not whole documents.