Finding the value of pieces

The problem

Evaluating chess positions is hard. That is why over the centuries a simple point system was established that assigns a value to each piece. Every chess player knows it. But are these values accurate? And how would we find them if we wouldn't have knowledge from centuries of experience? E.g., what if we want to find them for a chess variant like crazyhouse or atomic chess?

The approach

Given we have a game for which we want to establish a piece value system, but we don't have sufficient knowledge and data to derive it from human experience. We can still try to guess e.g. based on a piece's mobility, and this can give decent results, but won't be very accurate.

However, nowadays we can get strong engines in almost any given game, either with AlphaZero-like approaches or with a more heuristic solution like the chess variant engine Fairy-Stockfish. Using a strong engine we can then

1. Generate high-level engine games.
2. Determine the material difference of each position and associate with the game's result.
3. Train a predictor that estimates the winning chance from the material difference. The predictor's parameters can then be interpreted as piece values.

The math

If you aren't interested in the mathematical details, you can skip this section.

If not, you are likely interested in the predictor. Logistic regression seems like a natural choice here, because we want a linear combination of piece values and we want to map a linear scale to a probability in the range [0, 1]. This has also been tried for chess and atomic on human games before already.

For the logistic regression we define the evaluation of a position as

eval = (white_pawns - black_pawns) * pawn_value + ... + side_to_move_offset

and the evaluation is then mapped to a probability/score using

p(eval) = 1 / (1 + exp(-eval))

The results

The process described above I implemented in python, see the code on github. The games were generated using Fairy-Stockfish and the currently strongest available NNUE evaluation for it, which is far above human level for most games. In order to eliminate problems with unstable material situations, e.g., in case of recaptures, I filtered out positions where the material difference changed with the last move, i.e., captures and promotions.

Chess

Using 100,000 positions the results of the logistic regression for chess are:

q 9.82
r 4.93
b 3.28
n 3.16
p 1.00
k 0.00
move 0.01

The results match pretty well with independent piece value estimates. With that confirmation, let's try it on variants where piece values aren't that well established. Note that the king generally isn't assigned a piece value here because its count never changes, so it has no correlation with the game result.

Crazyhouse

For crazyhouse again with 100k positions, the results are:

q 3.94
r 3.05
n 1.73
b 1.69
p 1.00
k 0.00
move 0.07

In crazyhouse the range of piece values generally is compressed compared to chess, since the ability to reintroduce pieces via drops makes weaker pieces more useful and valuable pieces quite exposed to attacks.

For comparison, the most common human piece value estimates for crazyhouse I am aware of is the simple 1 2 4, i.e., 1 for pawns, 4 for the queen, and 2 for knights, bishops, and rooks.

Atomic

For atomic once again using 100k positions, the results are:

q 5.57
r 2.72
b 1.92
n 1.53
p 1.00
k 0.00
move 0.05

It is well-known that in atomic pawns are comparatively stronger, since they are not removed as part of explosions unless they directly participate in the capture, which also clearly shows in these results.

---

Just like lichess itself, the code producing these results is open source, you can find it on github. So feel free to try it out yourself if you are interested.

Also see my follow-up post that addresses some questions.