| Task | Claude Fable 5.1(High) | GPT 6 Astra(High) | Grok 4.7(High) | Kimi K3.1 Pro(High) | Claude Opus 5.5(High) | GPT 6.1 Sol(High) | Muse Spark(High) | Claude Haiku 5.5(High) | GPT 6 Luna(High) |
|---|---|---|---|---|---|---|---|---|---|
| Title Abstract ScreeningReward | 0.866$10.8743 | 0.775$1.8362 | 0.792$18.8588 | No published HIGH result | 0.852$3.2730 | 0.780$0.2702 | 0.846$2.1343 | 0.787$2.4267 | 0.752$0.0142 |
| Full Text ExtractionF1 | 0.287$3.4585 | 0.325$0.7543 | 0.178$4.6972 | 0.249$0.6376 | 0.376$0.9762 | 0.477$0.2672 | 0.279$0.4968 | 0.248$1.4929 | 0.229$0.0065 |
| Risk of BiasCohen’s κ | 0.264$5.2480 | 0.270$0.7805 | 0.193$8.2117 | No published HIGH result | 0.205$0.7088 | 0.201$0.1949 | 0.284$0.6513 | 0.210$1.3118 | 0.206$0.0083 |
Kimi K3.1 Pro ran on a Kimi Code subscription. Its cost estimates apply standard API rates to the recorded token usage.
Title/abstract screening
Model | |||
|---|---|---|---|
| 0.866 | 0.973 | 0.619 | |
| 0.852 | 0.975 | 0.595 | |
| 0.846 | 0.948 | 0.667 | |
| 0.796 | 0.964 | 0.539 | |
| 0.792 | 0.922 | 0.610 | |
| 0.787 | 0.966 | 0.530 | |
| 0.780 | 0.965 | 0.503 | |
| 0.775 | 0.964 | 0.503 | |
| 0.752 | 0.865 | 0.612 |
Full-text extraction
Model | |
|---|---|
| 0.477 | |
| 0.383 | |
| 0.376 | |
| 0.331 | |
| 0.325 | |
| 0.287 | |
| 0.279 | |
| 0.249 | |
| 0.248 | |
| 0.229 | |
| 0.178 |
Risk of Bias
Model | |
|---|---|
| 0.284 | |
| 0.270 | |
| 0.264 | |
| 0.210 | |
| 0.206 | |
| 0.205 | |
| 0.201 | |
| 0.193 | |
| 0.170 |