FINAL Bench Released: The Real Bottleneck to AGI Is Self-Correction
text
FINAL Bench Released: The Real Bottleneck to AGI Is Self-Correction
new_line
new_line
We release FINAL Bench, the first benchmark for measuring functional metacognition in LLMs — the ability to detect and correct one's own reasoning errors. Every existing benchmark measures final-answer accuracy. None measures whether AI knows it is wrong.
text
We release FINAL Bench, the first benchmark for measuring functional metacognition in LLMs — the ability to detect and correct one's own reasoning errors. Every existing benchmark measures final-answer accuracy. None measures whether AI knows it is wrong.
new_line
new_line
Dataset: [FINAL-Bench/Metacognitive](
text
Dataset: [FINAL-Bench/Metacognitive](
https://huggingface.co/datasets/FINAL-Bench/Metacognitive
FINAL-Bench/Metacognitive
dataset
resource
https://huggingface.co/datasets/FINAL-Bench/Metacognitive
) | 100 Tasks | 15 Domains | 8 TICOS Types | Apache 2.0
text
) | 100 Tasks | 15 Domains | 8 TICOS Types | Apache 2.0
new_line
new_line
Leaderboard: 
text
Leaderboard: 
https://huggingface.co/spaces/FINAL-Bench/Leaderboard
FINAL-Bench/Leaderboard
space
resource
https://huggingface.co/spaces/FINAL-Bench/Leaderboard
new_line
new_line
Article: 
text
Article: 
https://huggingface.co/blog/FINAL-Bench/metacognitive
https://huggingface.co/blog/FINAL-Bench/metacognitive
link
new_line
new_line
Core Innovation
text
Core Innovation
new_line
new_line
Our 5-axis rubric separates what no prior benchmark could: MA (Metacognitive Accuracy) — the ability to say "I might be wrong", and ER (Error Recovery) — the ability to actually fix it. This maps directly to the monitoring-control model of Nelson & Narens (1990) in cognitive psychology.
text
Our 5-axis rubric separates what no prior benchmark could: MA (Metacognitive Accuracy) — the ability to say "I might be wrong", and ER (Error Recovery) — the ability to actually fix it. This maps directly to the monitoring-control model of Nelson & Narens (1990) in cognitive psychology.
new_line
new_line
Three Findings Across 9 SOTA Models
text
Three Findings Across 9 SOTA Models
new_line
new_line
We evaluated GPT-5.2, Claude Opus 4.6, Gemini 3 Pro, DeepSeek-V3.2, Kimi K2.5, and others across 100 expert-level tasks:
text
We evaluated GPT-5.2, Claude Opus 4.6, Gemini 3 Pro, DeepSeek-V3.2, Kimi K2.5, and others across 100 expert-level tasks:
new_line
new_line
1. ER Dominance. 94.8% of MetaCog gain comes from Error Recovery alone. The bottleneck to AGI is not knowledge or reasoning — it is self-correction.
text
1. ER Dominance. 94.8% of MetaCog gain comes from Error Recovery alone. The bottleneck to AGI is not knowledge or reasoning — it is self-correction.
new_line
new_line
2. Declarative-Procedural Gap. All 9 models can verbalize uncertainty (MA = 0.694) but cannot act on it (ER = 0.302). They sound humble but fail to self-correct — the most dangerous AI safety profile.
text
2. Declarative-Procedural Gap. All 9 models can verbalize uncertainty (MA = 0.694) but cannot act on it (ER = 0.302). They sound humble but fail to self-correct — the most dangerous AI safety profile.
new_line
new_line
3. Difficulty Effect. Harder tasks benefit dramatically more from metacognition (Pearson r = -0.777, p < 0.001).
text
3. Difficulty Effect. Harder tasks benefit dramatically more from metacognition (Pearson r = -0.777, p < 0.001).
new_line
new_line
from datasets import load_dataset
dataset = load_dataset("FINAL-Bench/Metacognitive", split="train")
python
```python
from datasets import load_dataset
dataset = load_dataset("FINAL-Bench/Metacognitive", split="train")
```
code_fence
new_line
new_line
Paper:  FINAL Bench: Measuring Functional Metacognitive Reasoning in LLMs
text
Paper:  FINAL Bench: Measuring Functional Metacognitive Reasoning in LLMs
new_line
new_line
FINAL Bench is the first tool to tell apart what AI truly knows from what it merely pretends to know.
text
FINAL Bench is the first tool to tell apart what AI truly knows from what it merely pretends to know.
new_line
FINAL Bench Released: The Real Bottleneck to AGI Is Self-Correction

We release FINAL Bench, the first benchmark for measuring functional metacognition in LLMs — the ability to detect and correct one's own reasoning errors. Every existing benchmark measures final-answer accuracy. None measures whether AI knows it is wrong.

Dataset: [FINAL-Bench/Metacognitive](https://huggingface.co/datasets/FINAL-Bench/Metacognitive) | 100 Tasks | 15 Domains | 8 TICOS Types | Apache 2.0

Leaderboard: https://huggingface.co/spaces/FINAL-Bench/Leaderboard

Article: https://huggingface.co/blog/FINAL-Bench/metacognitive

Core Innovation

Our 5-axis rubric separates what no prior benchmark could: MA (Metacognitive Accuracy) — the ability to say "I might be wrong", and ER (Error Recovery) — the ability to actually fix it. This maps directly to the monitoring-control model of Nelson & Narens (1990) in cognitive psychology.

Three Findings Across 9 SOTA Models

We evaluated GPT-5.2, Claude Opus 4.6, Gemini 3 Pro, DeepSeek-V3.2, Kimi K2.5, and others across 100 expert-level tasks:

1. ER Dominance. 94.8% of MetaCog gain comes from Error Recovery alone. The bottleneck to AGI is not knowledge or reasoning — it is self-correction.

2. Declarative-Procedural Gap. All 9 models can verbalize uncertainty (MA = 0.694) but cannot act on it (ER = 0.302). They sound humble but fail to self-correct — the most dangerous AI safety profile.

3. Difficulty Effect. Harder tasks benefit dramatically more from metacognition (Pearson r = -0.777, p < 0.001).

```python
from datasets import load_dataset
dataset = load_dataset("FINAL-Bench/Metacognitive", split="train")
```

Paper:  FINAL Bench: Measuring Functional Metacognitive Reasoning in LLMs

FINAL Bench is the first tool to tell apart what AI truly knows from what it merely pretends to know.

👍
Ctulu82 ginigini SeaWolf-AI Dawaii Rickie77 aknot47 Anserwise JKsolution ItWillNeverWork Fishtiks mayafree andreavictory solarbeams Highgroundbkk nyxia Ujjwal-Tyagi ausntmarzi Mylove007 openfree ginipick hgseo akiko19191 jiamf ChunDe 916COM
👀
seawolf2357 Dawaii Anserwise mayafree Rickie77 ginigini JKsolution andreavictory Ujjwal-Tyagi SeaWolf-AI solarbeams ausntmarzi jungletsubasa John6666 ChunDe 916COM
😎
mayafree Anserwise cutechicken Rickie77 Dawaii ginigini JKsolution andreavictory Ujjwal-Tyagi 916COM
❤️
mayafree unmodeled-tyler andreavictory nyxia Ujjwal-Tyagi Mylove007 nightmedia 916COM
🔥
mayafree Fishtiks mrs83 andreavictory Ujjwal-Tyagi Dcas89 916COM
🤝
AGI-lab ausntmarzi Mylove007 Ujjwal-Tyagi jungletsubasa 916COM
🤗
aiqcamp andreavictory Ujjwal-Tyagi jungletsubasa 916COM
🚀
Ujjwal-Tyagi 916COM
🧠
Ujjwal-Tyagi 916COM
🤯
Ujjwal-Tyagi 916COM
6999b9356ad71099cedfb115
/avatars/428b92f83009b51d0d21359037e88d7a.svg
Amir Reza
iAmirIran
user
68dff9449ab7f4d313d5d74e
https://cdn-avatars.huggingface.co/v1/production/uploads/68dff9449ab7f4d313d5d74e/_TS11DlSrSH29w1c4pkFh.jpeg
Ujjwal Tyagi
Ujjwal-Tyagi
user
656bbeb09c8778992f8835df
/avatars/9503eb5cc25fb15cd26ac442b43fbc5c.svg
Tom
TomLucidor
user
6905bc786cb49b1f11d32728
https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/504I_H8J9NkiytiHvbx-h.jpeg
VIDRAFT_LAB
SeaWolf-AI
user