seawolf2357/darwin_list
Updated
seawolf PRO
seawolf2357
AI & ML interests
None yet
Recent Activity
published a model about 4 hours ago
seawolf2357/darwin_list liked a model 3 days ago
FINAL-Bench/Darwin-36B-Opus-VKAE liked a model 3 days ago
FINAL-Bench/Qwen3.5-35B-A3B-VKAEOrganizations
published a
model about 4 hours ago
liked 2
models 3 days ago
liked a
model 4 days ago
upvoted an article 5 days ago
Article
Does Your LLM Know *When It's About to Be Wrong*?
ginigen-ai
โข โข 20 reacted to ginigen-ai's post with โค๏ธ 5 days ago
Post
10376
๐ง Does your LLM know when it's about to be wrong?
Most leaderboards measure accuracy. We measure metacognition โ whether a model catches its own errors. Benchmark + leaderboard + adapters, all open. ๐
The surprise: even a K-AI #1 model (JGOS-31B-Citizen) is the strongest on multiple-choice traps (trap_rate 0.005 โ ~2 misses in 400) yet blind to its own free-form mistakes (self-confidence AUROC = 0.5, pure random). A tiny base-frozen adapter recovers that signal.
Two independent axes (never compared across a row): โ trap_rate โ does it fall for tempting trap options? (lower = stronger) โก adapter gain ฮ โ how much a lightweight adapter catches errors the model itself misses. (higher = more adapter value)
What's open: ๐ 300+100 trap problems (each with a hidden trap + TICOS type) ๐ 24-model leaderboard ๐งฉ 11 per-model adapters โ adapters, NOT fine-tunes (base stays frozen; the adapter just reads the hidden state โ P(wrong))
Submit any HF model โ auto-scored daily at 09:00 KST and added to the board.
๐ Leaderboard โ ginigen-ai/Metacognition-Leaderboard-Space
๐ Benchmark โ ginigen-ai/Metacognition-Bench
๐งฉ Adapters โ FINAL-Bench/metacognition-adapters-6a42c032e6beb803dd032961
๐ Article โ https://huggingface.co/blog/ginigen-ai/metacognition
Benchmark by ginigen-ai ยท Adapters by FINAL-Bench (Darwin/Chimera platform + AETHER metacognition tech).
Most leaderboards measure accuracy. We measure metacognition โ whether a model catches its own errors. Benchmark + leaderboard + adapters, all open. ๐
The surprise: even a K-AI #1 model (JGOS-31B-Citizen) is the strongest on multiple-choice traps (trap_rate 0.005 โ ~2 misses in 400) yet blind to its own free-form mistakes (self-confidence AUROC = 0.5, pure random). A tiny base-frozen adapter recovers that signal.
Two independent axes (never compared across a row): โ trap_rate โ does it fall for tempting trap options? (lower = stronger) โก adapter gain ฮ โ how much a lightweight adapter catches errors the model itself misses. (higher = more adapter value)
What's open: ๐ 300+100 trap problems (each with a hidden trap + TICOS type) ๐ 24-model leaderboard ๐งฉ 11 per-model adapters โ adapters, NOT fine-tunes (base stays frozen; the adapter just reads the hidden state โ P(wrong))
Submit any HF model โ auto-scored daily at 09:00 KST and added to the board.
๐ Leaderboard โ ginigen-ai/Metacognition-Leaderboard-Space
๐ Benchmark โ ginigen-ai/Metacognition-Bench
๐งฉ Adapters โ FINAL-Bench/metacognition-adapters-6a42c032e6beb803dd032961
๐ Article โ https://huggingface.co/blog/ginigen-ai/metacognition
Benchmark by ginigen-ai ยท Adapters by FINAL-Bench (Darwin/Chimera platform + AETHER metacognition tech).
- 11 replies
upvoted a collection 5 days ago
liked 10
models 6 days ago
FINAL-Bench/metacog-adapter-FastContext-1.0-4B-SFT
Updated โข 43 โข 21
FINAL-Bench/metacog-adapter-gemma-4-26B-A4B-it
Updated โข 51 โข 20
FINAL-Bench/metacog-adapter-gemma-4-12B-it
Updated โข 45 โข 20
FINAL-Bench/metacog-adapter-Qwen3.5-9B
Updated โข 44 โข 20
FINAL-Bench/metacog-adapter-Qwen3.5-35B-A3B
Updated โข 50 โข 20
FINAL-Bench/metacog-adapter-Qwen3.5-27B
Updated โข 41 โข 20
FINAL-Bench/metacog-adapter-Qwen-AgentWorld-35B-A3B
Updated โข 44 โข 21
FINAL-Bench/metacog-adapter-Darwin-9B-NEG
Updated โข 35 โข 21
FINAL-Bench/metacog-adapter-Darwin-28B-Opus
Updated โข 34 โข 23
FINAL-Bench/metacog-adapter-Ornith-1.0-9B
Updated โข 39 โข 22
liked a
Space 7 days ago
liked a
dataset 7 days ago
liked a
Space 7 days ago