# Retrieval relevance

> Grade how well a passage answers a question.

Section: Examples. Web page: https://www.launchables.ai/docs/examples/retrieval-relevance

A search or retrieval step returns passages for a question. Before passing them to a writer, a person or a larger model, grade how well each one answers the question, and keep only the useful ones.

## The request

```json
{
  "model": "myles-g-12b",
  "state": {
    "question": "How long do I have to return an unopened item?",
    "passage": "Unopened items can be returned within 30 days of delivery for a full refund. Opened items are accepted only if faulty."
  },
  "questions": {
    "relevance": {
      "type": "score",
      "instructions": "How well does the passage answer the question?",
      "criteria": [
        "Unrelated to the question",
        "On the topic but does not answer it",
        "Answers part of it",
        "Answers it fully"
      ]
    }
  }
}
```

## Reading the answer

```
"relevance": {"type": "score", "score": 2.94, "level": 3, "confidence": 0.96,
              "legend": {"0": "Unrelated to the question", "1": "On the topic but does not answer it",
                         "2": "Answers part of it", "3": "Answers it fully"},
              "probabilities": {"0": 0.0, "1": 0.0, "2": 0.06, "3": 0.94}}
```

- Rank passages by `score`, the probability-weighted level, rather than by `level` alone: it separates a confident 3 from a borderline one.
- Drop passages whose probability of level 0 or 1 together is high.
- Send one request per passage. Requests are independent, so they can run in parallel, or as a [batch](https://www.launchables.ai/docs/api/batches) for a large index.

## What to watch

- Questions where no passage reaches level 2 or 3: they show gaps in what you have indexed.
- Keep the question and the passage as separate fields in the state, so the model never mistakes one for the other.