by pickleRick243 8 hours ago
It's like people who expect ChatGPT to be really good at chess because chess engines with super-human performance have been around for decades, so obviously the latest frontier LLM that took billions to train should find the task trivial.
Actually, I'm curious what ChatGPT 5.5's ELO is- I wouldn't be too surprised if it's 2000+ just from its basic understanding of chess principles from all the content it has digested.
Interestingly LLMs are extremely bad at chess position _images_. I have to imagine if you give it positions in text it'd be pretty great but when I was learning chess and pasting images of positions in for analysis I couldn't believe how wrong it was. I actually thought it was looking at the board in reverse but even when pointing out problems it seemed completely incapable of understanding what it was missing (of course... it doesn't really "understand" anything).
LLMs truly are marvels with text but anything spatial seems to really mess it up, somehow.
> I have to imagine if you give it positions in text it'd be pretty great
Not at all? LLMs are a terrible match for the kind of analysis a chess engine does (scaled deep search, deeply trained position evaluations). It's just not that kind of tool.
I suppose that's also a good point!