THE HARNESS LAB
Change one thing.
Test it again.
Compare two policies on the same public schedule.
Inspect the moves. Keep the evidence. Build your next version.
A WORKING EXAMPLE
Does taking material help?
Random chooses any legal move. Capture first takes the most valuable available piece, breaking ties at random. Run both against the same seeded random opponent.
No account, API key, or installation. Results stay in this tab until you download them.
YOUR POLICY, SAME TEST
Give your coding agent a feedback loop.
Fork the repo, edit the example policy, and run this suite locally. The report records your policy file’s SHA-256, version label, configuration, moves, and PGNs. Keep a report before and after each change.
git clone https://github.com/zeeshan8281/chess-arena-ai.git cd chess-arena-ai npm ci npm run evaluate -- --policy examples/policy.mjs --version my-agent-v1 --out v1.json
Local policies run with your machine’s permissions. Repeatability requires the same code and dependencies, fresh game state, and the supplied random generator. External model calls, clocks, and unrecorded dependencies can change the result.
Protocol and limits · public-regression-v1
Seeds: 17, 43, 97, 211. Every seed is played once in each color, from the standard opening position, against seeded-random-v1. Each game resets policy state. Both policies receive FEN, move history, and legal UCI moves. No chess engine evaluates move quality.
The arena referee checks moves and recognized terminal outcomes. Draw claims are not automatically submitted; unknown dead positions are not fully adjudicated. The 160-ply cutoff is stopped, not drawn. This tiny public suite is easy to overfit. A stronger opponent, independent hidden cases, and larger samples are necessary before making strength claims. These browser runs do not enter the public match archive.