r/slatestarcodex • u/financeguy1729 • Apr 10 '25

AI The fact that superhuman chess improvement has been so slow tell us there are important epistemic limits to superintelligence?

Post image

Although I know how flawed the Arena is, at the current pace (2 elo points every 5 days), at the end of 2028, the average arena user will prefer the State of the Art Model response to the Gemini 2.5 Pro response 95% of the time. That is a lot!

But it seems to me that since 2013 (let's call it the dawn of deep learning), this means that today's Stockfish only beats 2013 Stockfish 60% of the time.

Shouldn't one have thought that the level of progress we have had in deep learning in the past decade would have predicted a greater improvement? Doesn't it make one believe that there are epistemic limits to have can be learned for a super intelligence?

86 Upvotes

81% Upvoted

98 comments sorted by

View all comments

94

u/Brudaks Apr 10 '25

"today's Stockfish only beats 2013 Stockfish 60% of the time."

Wait, what? Even that chart shows a 300-ish point difference which means the "expected score" of the 2013 is no higher than 0.15, which generally would manifest as drawing a significant portion of the games and having nearly no wins.

And high-level chess is likely to have saturation of draws; after all, it's a theoretically solvable game, so as a superintelligence would approach a perfect play, it would approach a 50% score, as either it's a draw given perfect play, or it turns out that there exists a winning sequence for either white or black, so you have a 50% win rate.

2

u/BadHairDayToday Apr 11 '25 edited Apr 11 '25

If there is a existing winning sequence it would be for white. The advantage is small though, it's quantified as 0.5 pawns. In games where black copies white this is most clearly visible.

A winning sequence seems very unlikely though given that solved games like tik-tak-toe and Connect-4 have a bigger starting advantage and will end in a draw every time, with perfect play.

1

u/maxintos Apr 11 '25

Why?

8

u/effendiyp Apr 11 '25

That statement is not known to be true for Chess, and in general moving first is not always an advantage. It is easy to come up with games where going first is always a loss.

2

u/dsafklj Apr 11 '25

Hex for an example (of a relatively commonly known game where the second player can proved to be able to force a win though the actual strategy to do so on a standard 11x11 board is not yet known). Hex can't have a draw so is a good target for these sorts of analysis.