Rendered at 18:47:43 GMT+0000 (Coordinated Universal Time) with Cloudflare Workers.
WoodenChair 2 hours ago [-]
Just a note that this is not novel. Automatic analysis of chess games built on traditional engines has existed for a long time. Without using stockfish as a sidekick, this wouldn’t be interesting because of the poor play of LLMs, however having an LLM enhance the commentary is interesting and I suspect has already been done by chess.com for years.
kuboble 1 hours ago [-]
I'm an ok chess player (roughly OP level) and slightly more experienced claude code user.
I tried to use claude to analyze my games using stockfish - this is not a problem claude is good at.
I spent few hours (however admittedly around opus 4.8) with claude on this and it was unable to use stockfish in any useful way.
I think if the skill created by OP works then it's a real contribution. I'll give it a try.
brumar 55 minutes ago [-]
Thanks for trying it out! I'd be happy to see how it goes for you and compare our implementation notes if the analysis you got is not too bad.
brumar 2 hours ago [-]
This what I think too. I also think there are much more refined approaches than the one I tried here. From a cursory look, I saw there are both scientific litterature on the subject of mixing llms and tools like stockfish and some dedicated closed platforms that put that into action.
Let's make clear that I did not spend much time on this project. Ideally I would have tried to put other models into the mix, like maybe Maia to better see the game from a "real player" eyes and pinpoint where expected move and stockfish moves differ.
Anyway, to me, it's good enough to be usable and shared.
dverlaeckt80 1 hours ago [-]
As a chess player I feel this is fascinating although there are plenty of analysis options already on lichess on chess.com. Still, sometimes it is more about learning and curiosity. Well done.
brausepulver 1 hours ago [-]
Does this give sensible analysis if you don't record your thinking out loud?
I've been using Elogram (https://elogram.gg/, made by a friend), which gives interpretability by showing how difficulty of a position changes with elo.
skyfantom 35 minutes ago [-]
You say chess and I’m here to shamelessly promote my “coin chess” version:)
This needs some balancing. The fact you can buy a bishop on turn 1 and check their king to disrupt their economy is overpowered
kokoe 2 hours ago [-]
You could also download my free chess app that hooks up to Claude for real time game analysis or chat about the game.
https://pwalessi.itch.io/chess-buddy
Simpliplant 2 hours ago [-]
Looks really cool, would love to try it but I'm on Mac so would appreciate Mac version.
kokoe 7 minutes ago [-]
I don’t have a Mac handy but I’ll see if I can build for Mac in Godot without a Mac
brumar 2 hours ago [-]
Thanks for sharing! So you gave stockfish to Claude too? Did you try other techniques?
kokoe 7 minutes ago [-]
Yeah I give the board position and stockfish’s analysis to Claude. The opening name too.
michaelastreiko 1 hours ago [-]
Nice use of a skill for something concrete. I've been happier with AI helpers that spit out a short, checkable report than ones that try to be a whole coach.
qwend 2 hours ago [-]
How does it work? Does it just use stockfish under the hood?
I just remember the times when I tried to use an AI model to play chess and half of the time it either hung a piece or made an illegal move.
qwend 2 hours ago [-]
Was a bit to overzealous... Just read the readme. Looks pretty cool. Maybe I'll give it a try and see how it compares to the chess.com analysis (although I have to say that I never really use it).
pelican0 2 hours ago [-]
Very cool work!
Did you write the skills (text, and code) all by hand, or are those prompt outputs?
brumar 2 hours ago [-]
Not by hand, no, the code was generated with claude code. The readme too, but with some extra efforts to avoid the awful ai generated readme.
It took multiple sessions to get to this result. At first I only generated annotated pgns and standalone html page inspired by lichess. The video generation was the cherry on top, it took few iterations too to fix issues and add markers and arrows. I only use consume the generated video these days, for the moment.
pelican0 1 hours ago [-]
Makes sense, good use of the tools at hand. And the SKILL.md files, also generated similarly?
brumar 54 minutes ago [-]
Yes.
pelican0 49 minutes ago [-]
Nice. The best part about this kind of development using models is when you have them write instructions for themselves / other models :).
NietTim 2 hours ago [-]
Neat! I've been slopping together various games to have AI play them, balatro, poker, blackjack, it's just so fun. Was just thinking about chess, wondering if, if I gave them an unrestricted sandbox, they'd start using existing solvers or not. They're surprisingly good at doing cardcounting, by the way.
yesitcan 58 minutes ago [-]
I find it funny that a video game (Balatro) is lumped with poker and blackjack.
IMTDb 3 hours ago [-]
I am very interested in this, but it looks like the video link gives 404
brumar 3 hours ago [-]
Weird, the video is supposed to be embedded. On my chrome desktop it displays properly. EDIT: fixed
TZubiri 1 hours ago [-]
Writing skills files like this one don't seem to be the future of computing or AI, not by the quality, nor by the cost (15$ per usage?).
Also, you seem to have written the actual SKILL.md prompts themselves with AI? I don't know what to say that's insane, at least write the prompts? This idea of asking chatgpt to write the prompts for you is beyond lazy. Then presenting it as a project or tool of value to share to others is delusional.
Sorry for being harsh
Victoria59 27 minutes ago [-]
[flagged]
techwizrd 3 hours ago [-]
[flagged]
physicalecon 2 hours ago [-]
[dead]
notloganhogg 3 hours ago [-]
[flagged]
Madmallard 2 hours ago [-]
"The result is not perfect" as LLMs are
"The fact that it reflects on my own thinking during the game makes it interesting from a teaching point of view, so I thought it was worth sharing." As AI the premium sycophantic scammer does
"but for me it is a much more pleasant and memorable experience than clicking around Stockfish branches"
And SO much less effective than doing the harder, more tedious feeling work
brumar 2 hours ago [-]
The jury is out for the effectiveness, it's hard to debate this subject. From my cognitive background I know very well how important the generative effect is for learning. Maybe I'll add features that leverage generative/testing effect one day. Anyway, clicking on stockfish branches can be quite a passive activity too if done badly. I don't know how to do that well to be honest. My goal is often to just to understand what I missed, full stop.
For the sycophancy, I can say I did not feel that at all. When stockfish says your move suck, claude would have a hard time saying the opposite (no "you are absolutely right" when I am not).
Madmallard 2 hours ago [-]
Generative AI is really good at making you feel like you understand a lot without actually making you understand anything in-depth
Because lengthy difficult cognitively intensive labor is required for the brain to actually change its structure and connections
brumar 2 hours ago [-]
Very true. I like to think AI somehow optimizes for "efficient vagueness", which is bad news for our brain.
Nonetheless, as many do, I often ask AI to explain me stuff. I know it's not perfect, but it's convenient, it's a trade-off to make.
2 hours ago [-]
porridgeraisin 2 hours ago [-]
The elo of opus 5 models is 1300 or so. I wouldn't take chess lessons from a 1300.
brumar 2 hours ago [-]
That was exactly my estimation when I played "raw" against Opus. But Opus with stockfish as a tool and much time available can, from what I experienced, generate good comments.
porridgeraisin 1 hours ago [-]
Well yeah, of course. But also, analysing with stockfish is _also_ useless, unless you're a super GM.
I tried to use claude to analyze my games using stockfish - this is not a problem claude is good at.
I spent few hours (however admittedly around opus 4.8) with claude on this and it was unable to use stockfish in any useful way.
I think if the skill created by OP works then it's a real contribution. I'll give it a try.
Let's make clear that I did not spend much time on this project. Ideally I would have tried to put other models into the mix, like maybe Maia to better see the game from a "real player" eyes and pinpoint where expected move and stockfish moves differ.
Anyway, to me, it's good enough to be usable and shared.
I've been using Elogram (https://elogram.gg/, made by a friend), which gives interpretability by showing how difficulty of a position changes with elo.
https://chesspoly.com/
Did you write the skills (text, and code) all by hand, or are those prompt outputs?
It took multiple sessions to get to this result. At first I only generated annotated pgns and standalone html page inspired by lichess. The video generation was the cherry on top, it took few iterations too to fix issues and add markers and arrows. I only use consume the generated video these days, for the moment.
Also, you seem to have written the actual SKILL.md prompts themselves with AI? I don't know what to say that's insane, at least write the prompts? This idea of asking chatgpt to write the prompts for you is beyond lazy. Then presenting it as a project or tool of value to share to others is delusional.
Sorry for being harsh
"The fact that it reflects on my own thinking during the game makes it interesting from a teaching point of view, so I thought it was worth sharing." As AI the premium sycophantic scammer does
"but for me it is a much more pleasant and memorable experience than clicking around Stockfish branches"
And SO much less effective than doing the harder, more tedious feeling work
For the sycophancy, I can say I did not feel that at all. When stockfish says your move suck, claude would have a hard time saying the opposite (no "you are absolutely right" when I am not).
Because lengthy difficult cognitively intensive labor is required for the brain to actually change its structure and connections
Nonetheless, as many do, I often ask AI to explain me stuff. I know it's not perfect, but it's convenient, it's a trade-off to make.