Help me test my chess bot
The last couple of weeks I've been fooling around with different ideas for how to build a chess bot that's fun for beginners to play against. I don't have much real experience with chess, so I've mostly just been testing it against myself.
After looking into the different techniques that are used to force engines to play at a certain level, I put together my own (somewhat silly) approach and have had some fun playing against it. The problem is, as I don't really know what I'm doing when it comes to the actual game, I can't be a particularly good judge of how others will feel playing with it.
Regardless of your own skill level, I'd be super appreciative if anyone would give it a try and let me know what they think.
I'm working on a full write up of how it works, but here's the short version:
Click to view the hidden text
The inspiration came from this paper which describes a "Tutoring Search" wherein the engine attempts to find the worst move available that it predicts the opponent will not recognize as an error.
My implementation doesn't follow this exactly, but it does have the same aim. Two engines are used: one (Stockfish) as an oracle treated as a true measure of any state's quality, another (Maia) as a substitute for the opponent model. On each move the bot consults both of these to identify a move which:
- Would plausibly be played by a skilled human, judged by its probability of being played by Maia.
- Provides an advantage to the opponent, judged by Stockfish.
The idea is that, if a human would be likely to play the move, they also would be unlikely to identify it as an error. The ultimate goal is a bot which gives the player plenty of opportunities to win, but only if the mistakes are likely to go unnoticed.
There are a few other supplements to the implementation like adapting to opponent choices and some tweaks to early and end-game play, but the above is the core idea.
Your link doesn't quite work, it pre-selected an Unlimited game, which the bot then rejected. A 5+3 however was accepted afterwards. I actually won against it. It started out as a rather even game, with it eking out a bit of an advantage, but then around 20 moves in it made a series of blunders and then even walked into a forced mate sequence. ConfidenceBuilder is definitely an apt name, I did feel rather clever against it. https://lichess.org/h8T15ns5/black
I almost never play rapid so I was on my back foot from the very beginning. But I abandoned a game and now it seems stuck in "waiting for opponent" when trying to start a new game.
Oh no! Thanks for letting me know, I'll have to see how that's supposed to work. I'm guessing it times out after a certain point? I just manually ended it for now.
It should accept pretty much any game timing, so feel free to play whichever.
I tried selecting a different timing, but when I did so the "challenge confidencebuilder" button disabled itself.
Ah, I thought the link just set the default, but it was actually setting a requirement. I changed it and it should work now.
Thanks! I've played a few games now and my general impression is it feels strong enough in the early game but totally falls apart and starts blundering mid game. Total gut feeling but maybe around an 800 rating?
Thanks! I think you're right. Looking through some of the human games, it does seem to collapse a bit too obviously after 10-20 moves.
There's an artificial improvement in the early game (as early mistakes felt too obvious as a player), but the shift can feel abrupt when it happens. It's programmed to be a slow transition, but needs some fine tuning.
I tried, and got challenge declined. No matter, I'd have lost anyway.
I have played a little of these 5 min games on lichess, but I usually loose against human opponents. I used to play a bit on the tram to work, but haven't done so in a while. Just tried your bot and to my surprise I won (I must be some kind of wunderkind)! I have no idea of how difficult the bot is or what blunders I (or the bot) made, but it felt challenging to me. Nice work!
https://lichess.org/HzV5SQ7O - Not my best game ever, but I did win, largely because it missed a lot of obvious moves (in particular
33. Ne4???, turning what I thought was a queen trade into a free queen capture for me).https://lichess.org/wNliXQCf - Feels like it played better in this one, but also I destroyed it more thoroughly.
It is also no longer accepting my challenges (just spins forever waiting for opponent). I choose to believe I am just so amazing that it finds me intimidating.