This is my first post, so I need to start with a bang!
text
This is my first post, so I need to start with a bang!
new_line
new_line
The people over at 
text
The people over at 
https://cdn-avatars.huggingface.co/v1/production/uploads/5e70f6048ce3c604d78fe133/JRhjPDJZpjaWKmxUL5L6M.png
https://huggingface.co/Lichess
Lichess
org
resource
https://huggingface.co/Lichess
 published some amazing data sets over the past weeks, including a collection of >1M standard chess games (
text
 published some amazing data sets over the past weeks, including a collection of >1M standard chess games (
https://huggingface.co/datasets/Lichess/standard-chess-games
Lichess/standard-chess-games
dataset
resource
https://huggingface.co/datasets/Lichess/standard-chess-games
).
text
).
new_line
new_line
Finally it's time to revive my chess buddy project from back in 2021 🎉
text
Finally it's time to revive my chess buddy project from back in 2021 🎉
new_line
new_line
So without any further ado... I'm currently training my first character level LLM, and to be quite frank, I'm pretty astonished with the quality of my testing samples.
text
So without any further ado... I'm currently training my first character level LLM, and to be quite frank, I'm pretty astonished with the quality of my testing samples.
new_line
new_line
I'm using 
text
I'm using 
e4 g6
`e4 g6`
inline_code
, the Modern Defense (
text
, the Modern Defense (
https://en.wikipedia.org/wiki/Modern_Defense
https://en.wikipedia.org/wiki/Modern_Defense
link
) as a validation sample.
text
) as a validation sample.
new_line
My model currently predicts mostly 
text
My model currently predicts mostly 
d4 Bg7
`d4 Bg7`
inline_code
 which are the strongest next moves for white and black.
text
 which are the strongest next moves for white and black.
new_line
new_line
Now in between I see some results that take lower ranked moves, which makes me very excited.
text
Now in between I see some results that take lower ranked moves, which makes me very excited.
new_line
new_line
Once the pre-training is done for the base model, I want to run some fine tuning on more specific data sets, which are 
text
Once the pre-training is done for the base model, I want to run some fine tuning on more specific data sets, which are 
new_line
https://huggingface.co/datasets/Lichess/chess-openings
Lichess/chess-openings
dataset
resource
https://huggingface.co/datasets/Lichess/chess-openings
new_line
https://huggingface.co/datasets/Lichess/chess-puzzles
Lichess/chess-puzzles
dataset
resource
https://huggingface.co/datasets/Lichess/chess-puzzles
new_line
new_line
Here are some intermediate examples
text
Here are some intermediate examples
new_line
new_line
Step 6000: 
1. e4 g6 13. Rb1 d5 14. Bd3 Nxf3 15. Nxf3 Nxe3+ 16. Rxd3 Rxd3 17. Rxd6 Rhe8 18. Nd6 Rxd4 19. Rxd7+ Kxd7 20. Re7 Rxe7 21. Qxe7 1-0

Step 12000:
1. e4 g6 22. Be2 Re8 23. Kg2 1-0
1. d4 d5 2. c4 c6 3. Nf3 e6 4. dxe6 Qe7 5. Bb5+ Be8 6. Bxb7# 1-0
1. d4 d5 2. dxe5 Bd6 3. Nc3 h6 4. e4 Bf5 5. exf5 Nd7 6. exd5 Nxd5 7. Bxc4 Bxe2 8. f4 d4 9. Ng3 Bb4+ 10. Bxd4 Qxd4 11. Nfxe2 O-O-O 12. Ne6 Qf5 13. fxg4 Nxe5

Step 30000:
1. e4 g6 2. d4 Bg7 3. Nf3 d6 4. b3 e6 5. Bb2 f5 6. e5 c5 7. dxc5 dxc5 8. Nbd2 Nf6 9. Nce2 O-O 10. Qe2 c4 11. Na4 Bd6 12. f3 Ng4 13. fxg4 1-0
1. c4 c5 2. a3 Nc6 3. cxd5 Nxd5 4. Bf4 g6 5. Be2 Bg7 6. Nf3 Bg4 7. b4 Nf6 8. h3 Bxf3 9. Bxf3 a6 10. Nc3 O-O 11. Qc2 e
```
Step 6000: 
1. e4 g6 13. Rb1 d5 14. Bd3 Nxf3 15. Nxf3 Nxe3+ 16. Rxd3 Rxd3 17. Rxd6 Rhe8 18. Nd6 Rxd4 19. Rxd7+ Kxd7 20. Re7 Rxe7 21. Qxe7 1-0

Step 12000:
1. e4 g6 22. Be2 Re8 23. Kg2 1-0
1. d4 d5 2. c4 c6 3. Nf3 e6 4. dxe6 Qe7 5. Bb5+ Be8 6. Bxb7# 1-0
1. d4 d5 2. dxe5 Bd6 3. Nc3 h6 4. e4 Bf5 5. exf5 Nd7 6. exd5 Nxd5 7. Bxc4 Bxe2 8. f4 d4 9. Ng3 Bb4+ 10. Bxd4 Qxd4 11. Nfxe2 O-O-O 12. Ne6 Qf5 13. fxg4 Nxe5

Step 30000:
1. e4 g6 2. d4 Bg7 3. Nf3 d6 4. b3 e6 5. Bb2 f5 6. e5 c5 7. dxc5 dxc5 8. Nbd2 Nf6 9. Nce2 O-O 10. Qe2 c4 11. Na4 Bd6 12. f3 Ng4 13. fxg4 1-0
1. c4 c5 2. a3 Nc6 3. cxd5 Nxd5 4. Bf4 g6 5. Be2 Bg7 6. Nf3 Bg4 7. b4 Nf6 8. h3 Bxf3 9. Bxf3 a6 10. Nc3 O-O 11. Qc2 e
```
code_fence
new_line
(each line starting with 
text
(each line starting with 
1.
`1.`
inline_code
 is a set of moves)
text
 is a set of moves)
new_line
new_line
You can find a first pre trained version here:
text
You can find a first pre trained version here:
new_line
https://huggingface.co/philipp-zettl/chessPT
philipp-zettl/chessPT
model
resource
https://huggingface.co/philipp-zettl/chessPT
This is my first post, so I need to start with a bang!

The people over at https://huggingface.co/Lichess published some amazing data sets over the past weeks, including a collection of >1M standard chess games (https://huggingface.co/datasets/Lichess/standard-chess-games).

Finally it's time to revive my chess buddy project from back in 2021 🎉

So without any further ado... I'm currently training my first character level LLM, and to be quite frank, I'm pretty astonished with the quality of my testing samples.

I'm using `e4 g6`, the Modern Defense (https://en.wikipedia.org/wiki/Modern_Defense) as a validation sample.
My model currently predicts mostly `d4 Bg7` which are the strongest next moves for white and black.

Now in between I see some results that take lower ranked moves, which makes me very excited.

Once the pre-training is done for the base model, I want to run some fine tuning on more specific data sets, which are 
https://huggingface.co/datasets/Lichess/chess-openings
https://huggingface.co/datasets/Lichess/chess-puzzles

Here are some intermediate examples

```
Step 6000: 
1. e4 g6 13. Rb1 d5 14. Bd3 Nxf3 15. Nxf3 Nxe3+ 16. Rxd3 Rxd3 17. Rxd6 Rhe8 18. Nd6 Rxd4 19. Rxd7+ Kxd7 20. Re7 Rxe7 21. Qxe7 1-0

Step 12000:
1. e4 g6 22. Be2 Re8 23. Kg2 1-0
1. d4 d5 2. c4 c6 3. Nf3 e6 4. dxe6 Qe7 5. Bb5+ Be8 6. Bxb7# 1-0
1. d4 d5 2. dxe5 Bd6 3. Nc3 h6 4. e4 Bf5 5. exf5 Nd7 6. exd5 Nxd5 7. Bxc4 Bxe2 8. f4 d4 9. Ng3 Bb4+ 10. Bxd4 Qxd4 11. Nfxe2 O-O-O 12. Ne6 Qf5 13. fxg4 Nxe5

Step 30000:
1. e4 g6 2. d4 Bg7 3. Nf3 d6 4. b3 e6 5. Bb2 f5 6. e5 c5 7. dxc5 dxc5 8. Nbd2 Nf6 9. Nce2 O-O 10. Qe2 c4 11. Na4 Bd6 12. f3 Ng4 13. fxg4 1-0
1. c4 c5 2. a3 Nc6 3. cxd5 Nxd5 4. Bf4 g6 5. Be2 Bg7 6. Nf3 Bg4 7. b4 Nf6 8. h3 Bxf3 9. Bxf3 a6 10. Nc3 O-O 11. Qc2 e
```
(each line starting with `1.` is a set of moves)

You can find a first pre trained version here:
https://huggingface.co/philipp-zettl/chessPT
🔥
YaTharThShaRma999 John6666 osanseviero nicolay-r Nelathan louisbrulenaudet