When the reporter entered the confessional, AI Jesus warned, āDo not disclose personal information under any circumstances. Use this service at your own risk.
Do not worry my child, for everything you say in this hallowed chamber is between you, AI Jesus, and the army of contractors OpenAI hires to evaluate the quality of their LLM output.
Iām not a Chess person or familiar with Stockfish so take this with a grain of salt, but I found a few interesting things perusing the code / docs which I think makes useful context.
Skill Level
I assume ālevelā refers to Stockfishās Skill Level option.
If I mathed right, Stockfish roughly estimates Skill Level 1 to be around 1445 ELO (source). However it says āThis Elo rating has been calibrated at a time control of 60s+0.6sā so it may be significantly lower here.
Skill Level affects the search depth (appears to use depth of 1 at Skill Level 1). It also enables MultiPV 4 to compute the four best principle variations and randomly pick from them (more randomly at lower skill levels).
Move Time & Hardware
This is all independent of move time. This author used a move time of 10 milliseconds (for stockfish, no mention on how much time the LLMs got). ā¦ or at least they did if they accounted for the āMove Overheadā option defaulting to 10 milliseconds. If they left that at itās default then 10ms - 10ms = 0ms so š¤·āāļø.
There is also no information about the hardware or number of threads they ran this one, which I feel is important information.
Evaluation Function
Stockfishās FAQ mentions that they have gone beyond centipawns for evaluating positions, because itās strong enough that material advantage is much less relevant than it used to be. I assume it doesnāt really matter at level 1 with ~0 seconds to produce moves though.
Still since the author has Stockfish handy anyway, itād be interesting to use it in itās not handicapped form to evaluate who won.