Evidence
Measured, then claimed.
Every result we publish, with its method and its date. Each model is tested on frozen sets it has never seen. Results that have not been independently checked are marked as our own.
Updated 14 October 2026
Accuracy and cost, against hosted models
Every system answers the same 1,400 decisions, frozen before any comparison: 800 from an open decision dataset (half of them from a shifted distribution), 300 product tasks and 300 everyday tasks. The hosted models answer zero-shot with one fixed prompt.
| System | Correct | Cost per million decisions | Median time |
|---|---|---|---|
| Myles-400M, on one CPU thread | 91.8% | £4 | 500 ms |
| Hosted LLM, small tier | 86.9% | £900 | 770 ms |
| Hosted LLM, medium tier | 89.6% | £1,880 | 1,390 ms |
| Hosted LLM, large tier | 96.2% | £4,110 | 1,870 ms |
Our own measurement, September 2026. Myles-400M leads the medium tier by 2.2 points (95% confidence interval 0.4 to 4.0 points) at about one 450th of its cost. Local cost assumes a cloud CPU at about 3p per hour; hosted prices are list prices converted at £1 = $1.33; hosted times include the network.
Certified error rates
Myles keeps a decision when its confidence clears a threshold set on held-out data, one per kind of input, and hands the rest to a person. These are the results on unseen test data.
| Target error | Decisions kept | Error on kept decisions | Inside target |
|---|---|---|---|
| 5% | 73% | 2.7% | Yes |
| 2% | 40% | 1.4% | Yes |
| 1% | 16% | 0.5% | Yes |
Our own measurement: the Myles-400M build behind the 91.8% result, September 2026. How certification works
Reasoning it was never shown
Our own measurement, on the September 2026 Myles-400M build. The reasoning problems are generated with exact answers, and whole domains are held out of training. The benchmark score covers its 231 public tasks, scored with the benchmark's own code.
Myles-G on one GPU
| Model | Public benchmark items | Median decision | Hardware |
|---|---|---|---|
| Myles-G 31B | 211 of 231 | 48 ms | One data-centre GPU |
| Myles-G 12B | 203 of 231 | 31 ms | One workstation GPU |
| Myles-G Edge | 189 of 231 | 21 ms | Laptop- and phone-class GPUs |
Unverified: our own measurement on the 231 public JevBench items: Myles-G 31B and 12B as our API serves them, 16 October 2026 (the 31B reads 210 to 212 across server restarts), Edge on 11 October 2026. Not an official placement. Official evaluation pending; we will add the official result here, with its date and source, when it is published. Gemma is a trademark of Google LLC.
Myles-G 12B on images
Internal measurement on 934 held-out images across receipts, app screens, product photos and object boxes, 16 Oct 2026; not an official benchmark placement. On the Decision API, one image at the default size answers in about a fifth of a second (median on our current GPU, measured the same day). Images in the docs