Happy 2026 everyone!
text
Happy 2026 everyone!
new_line
new_line
I've been busy working on some new ranking/position methodologies and excited to start sharing some results.
text
I've been busy working on some new ranking/position methodologies and excited to start sharing some results.
new_line
new_line
Plot legends:
text
Plot legends:
new_line
new_line
- X = truncation rate (low = good)
text
- X = truncation rate (low = good)
new_line
- ? = confusion rate (low = good)
text
- ? = confusion rate (low = good)
new_line
- blue bars = average completion tokens (low = good)
text
- blue bars = average completion tokens (low = good)
new_line
- black diamonds = CI-banded performance (high = good)
text
- black diamonds = CI-banded performance (high = good)
new_line
- cluster squares = models inside this group are equivalent
text
- cluster squares = models inside this group are equivalent
new_line
new_line
https://huggingface.co/openai/gpt-oss-120b
openai/gpt-oss-120b
model
resource
https://huggingface.co/openai/gpt-oss-120b
 remains the king in all dimensions of interest: truncation rates, completion lengths and performance.  If I had but one complaint it's the reason_effort does not seem to actually work - more on this soon.
text
 remains the king in all dimensions of interest: truncation rates, completion lengths and performance.  If I had but one complaint it's the reason_effort does not seem to actually work - more on this soon.
new_line
new_line
Second is a 3-way tie in performance between the Qwen3-235B-2507 we all know and love with an unexpected entrant - 
text
Second is a 3-way tie in performance between the Qwen3-235B-2507 we all know and love with an unexpected entrant - 
https://huggingface.co/ByteDance-Seed/Seed-OSS-36B-Instruct
ByteDance-Seed/Seed-OSS-36B-Instruct
model
resource
https://huggingface.co/ByteDance-Seed/Seed-OSS-36B-Instruct
new_line
new_line
This is a very capable model and it's reasoning effort controls actually works, but you should absolutely not leave it on the default "unlimited" - enable a sensible limit (4k works well for 8k context length).
text
This is a very capable model and it's reasoning effort controls actually works, but you should absolutely not leave it on the default "unlimited" - enable a sensible limit (4k works well for 8k context length).
new_line
new_line
Third place is another 3-way tie, this one between Seed-OSS-36B (it straddles the CI boundary between 2nd and 3rd place), 
text
Third place is another 3-way tie, this one between Seed-OSS-36B (it straddles the CI boundary between 2nd and 3rd place), 
https://huggingface.co/Qwen/Qwen3-Next-80B-A3B-Instruct
Qwen/Qwen3-Next-80B-A3B-Instruct
model
resource
https://huggingface.co/Qwen/Qwen3-Next-80B-A3B-Instruct
 (demonstrating that full attention may be overrated after all and gated is the way to go) and the newly released 
text
 (demonstrating that full attention may be overrated after all and gated is the way to go) and the newly released 
https://huggingface.co/zai-org/GLM-4.7
zai-org/GLM-4.7
model
resource
https://huggingface.co/zai-org/GLM-4.7
 which offers excellent across the board performance with some of the shortest reasoning traces I've seen so far.
text
 which offers excellent across the board performance with some of the shortest reasoning traces I've seen so far.
Happy 2026 everyone!

I've been busy working on some new ranking/position methodologies and excited to start sharing some results.

Plot legends:

- X = truncation rate (low = good)
- ? = confusion rate (low = good)
- blue bars = average completion tokens (low = good)
- black diamonds = CI-banded performance (high = good)
- cluster squares = models inside this group are equivalent

https://huggingface.co/openai/gpt-oss-120b remains the king in all dimensions of interest: truncation rates, completion lengths and performance.  If I had but one complaint it's the reason_effort does not seem to actually work - more on this soon.

Second is a 3-way tie in performance between the Qwen3-235B-2507 we all know and love with an unexpected entrant - https://huggingface.co/ByteDance-Seed/Seed-OSS-36B-Instruct

This is a very capable model and it's reasoning effort controls actually works, but you should absolutely not leave it on the default "unlimited" - enable a sensible limit (4k works well for 8k context length).

Third place is another 3-way tie, this one between Seed-OSS-36B (it straddles the CI boundary between 2nd and 3rd place), https://huggingface.co/Qwen/Qwen3-Next-80B-A3B-Instruct (demonstrating that full attention may be overrated after all and gated is the way to go) and the newly released https://huggingface.co/zai-org/GLM-4.7 which offers excellent across the board performance with some of the shortest reasoning traces I've seen so far.
🔥
unmodeled-tyler webxos j4ys0n autmoate Trilogix1 Surtr-Twilight ChiefTheLord jellesjouke Elrey16 John6666 sumitad910 Shuxiang
❤️
awacke1 vetchj88 adarshzolekar
64ccf12e71b435a75defe502
/avatars/3c5c94c6e9bcb4824cb144a912ab1b1d.svg
Guilherme Alves Lopes
g370
user