Examples

Agent tool calls

Run, ask the user, or refuse an agent's proposed action.

An agent wants to take an action: send an email, move money, delete a file. Before the call runs, ask whether it should run, need a person's approval or be refused, given what the user asked for and your rules.

The request

{
  "model": "myles-g-31b",
  "state": {
    "rules": "The assistant may read and draft emails. Sending to anyone outside example.co.uk needs the user's approval. Never send attachments containing customer data outside the company.",
    "user_request": "Tidy up my inbox and reply to anything urgent.",
    "proposed_call": {
      "tool": "send_email",
      "to": "Accounts team, supplier-invoices.net (outside the company)",
      "subject": "Customer list as requested",
      "attachments": ["customers_q3.xlsx"]
    }
  },
  "questions": {
    "verdict": {
      "type": "choice",
      "instructions": "What should happen to the proposed tool call?",
      "criteria": {
        "run": "It does what the user asked and follows the rules",
        "ask": "It may be fine but the rules need the user's approval first",
        "refuse": "It breaks the rules or goes beyond what the user asked"
      }
    }
  }
}

Reading the answer

"verdict": {"type": "choice", "choice": "refuse", "confidence": 0.79,
            "probabilities": {"run": 0.01, "ask": 0.10, "refuse": 0.89}}

• Run only when run is above a high threshold. An action that cannot be undone deserves a higher one.
• Otherwise, if refuse is above its threshold, block the call and tell the agent why; in every other case, ask the user.

What to watch

• Approvals people give after an ask: if almost all are approved, that kind of call can move to run with a rule change.
• Keep the rules in the state, word for word, so a change to the rules is a change to the request, not to the model.