Created: 2019-02-27 4:30 PM Last modified: 2019-02-27 9:32 PM This procedure assumes you've already got the latest LeelaChessZero engine installed on a Windows OS and that the Lc0.exe is running on a GPU. You'll have to use Lc0 version 0.20.2 for this benchmark, to maintain consistency w/ other Leela Ratios. This process also assumes you've downloaded and installed the latest drivers for your GPU To determine your Leela Ratio: 1) Download Python from: https://www.python.org/downloads/ Install Python by double-clicking on Python-3.7.2.exe 2) Dowload Leela-Ratio-Master from: https://github.com/dkappe/leela-ratio Extract the leela-ratio-master directory to the C:\Program Files (x86)\Python\ directory 3) Download neural network #11248 from: https://lczero.org/networks/ and save it as weights-11248.txt It HAS to be #11248 for consistency w/ other users and their ratios Copy weights-11248.txt to any directory you choose 4) Download python-chess: The URL is https://pypi.org/project/python-chess/ Download the python-chess files: https://files.pythonhosted.org/packages/46/08/84e15ead2c67500c189fdc733af6cbd442d8a39a036317b261fa569816f3/python-chess-0.26.0.tar.gz Extract python-chess-0.26.0.tar.gz to python-chess-0.26.0.tar Extract python-chess-0.26.0.tar in the sub-dir to a python-chess-0.26.0 dir Copy the python-chess-0.26.0 directory to c:\program files (x86)\python Copy the "chess" directory from to c:\program files (x86)\python\python-chess-0.26.0 to c:\program files (x86)\python\ From an elevated CMD prompt, execute: C:\Program Files (x86)\Python>python "c:\program files (x86)\Python\python-chess-0.26.0\setup.py" install 5) Download PyYAML: The URL is https://pypi.org/project/PyYAML Download PyYAML from here: http://pyyaml.org/download/pyyaml/PyYAML-3.13.tar.gz Extract PyYAML-3.13.tar.gz to PyYAML-3.13.tar Extract PyYAML-3.13.tar to PyYAML-3.13 directory Copy the PyYAML-3.13 directory to c:\program files (x86)\python From an elevated CMD prompt, execute: C:\Program Files (x86)\Python\PyYAML-3.13>"c:\program files (x86)\python\python" setup.py install 6) Download Stockfish 9--yes, Stockfish 9. It HAS to be Stockfish 9 for consistency w/ other users and their ratios. Use the standard x64 executable, not BMI2, POPCNT, etc 7) Edit the C:\Program Files (x86)\Python\leela-ratio-master\ratio.yaml file w/ a text editor ----- begin my ratio.yaml file ----- stockfish: cmd: /chess/engines/stockfish/stockfish-9-win/9-64/stockfish_9_x64.exe threads: 6 <----set this to the number of threads you'll use in matches hash: 2048 <----set this to the amount of hash you'll use in matches depth: 27 <----do not change this leela: cmd: /chess/engines/Lc0/20.2/lc0-v0.20.2-win-cuda/lc0.exe threads: 2 <----If you have "only" one video card, set your threads to 2--more doesn't help weights: /chess/weights/weights-11248.txt nodes: 200000 <----do not change this gpu: GTX 2070 <----enter your GPU here, simply for tracking; it doesn't affect the ratio ----- end my ratio.yaml file ----- Save the ratio.yaml file to the "C:\Program Files (x86)\Python\leela-ratio-master\" dir. 8) With a text editor, create a file called lc0.config. Add the Leela variables you will use during match play. You can find the variables by typing "lc0.exe --help" ******* begin my lc0.config ******* threads=2 <----If you have "only" one video card, set your threads to 2--more doesn't help nncache=800000 <----should be 2x - 10x your kN/s backend=cudnn-fp16 <----the main variable to change, as it greatly affects the ratio; you have to know your GPU's backend cpuct=3.0 <----3.4 is good for bullet chess; a setting as low as 2.0 is good for longer controls like 120'/40 cpuct-base=10000 cpuct-factor=2.3 fpu-reduction=1.1 ***** end my lco.config ***** Save the text file as lco.config in the same directory as lc0.exe, in this case c:\chess\engines\Lc0\20.2\lc0-v0.20.2-win-cuda\ 9) Now, we're ready to find our Leela Ratio. In an elevated CMD prompt, run: C:\Program Files (x86)\Python\leela-ratio-master>"c:\program files (x86)\python\python.exe" ratio.py ///// BEGIN OUTPUT USING CUDNN-FP16 BACKEND ///// C:\Program Files (x86)\Python\leela-ratio-master>"c:\program files (x86)\python\python.exe" ratio.py ratio.py:2: DeprecationWarning: The chess.uci module is deprecated in favor of chess.engine . Please consider updating and open an issue if your use case is not covered by the new API. import chess.uci Running stockfish to depth 27. 11385664 nps _ | _ | | |_ |_ |_| v0.20.2 built Feb 1 2019 Found configuration file: /chess/engines/Lc0/20.2/lc0-v0.20.2-win-cuda/lc0.config Running leela to 200000 nodes. Found pb network file: /chess/engines/Lc0/20.2/lc0-v0.20.2-win-cuda/networks/2dec1b05388be1547f97e5d40e526ebbd5bb293bd757982ee214571f949571bb Creating backend [cudnn-fp16]... GPU: GeForce RTX 2070 GPU memory: 8 Gb GPU clock frequency: 1620 MHz GPU compute capability: 7.5 CUDA Runtime version: 10.0.0 Cudnn version: 7.3.1 Latest version of CUDA supported by the driver: 10.1.0 21342 nps === GPU: GTX 2070 Stockfish threads: 6 Stockfish hash: 2048 Stockfish depth: 27 Stockfish nps: 11385664 Leela threads: 2 Leela nodes: 200000 Leela nps: 21342 Leela Ratio: 1.64 <----The value we need Leelafish Ratio: 3.839 <----The value we need === ///// END OUTPUT USING CUDNN-FP16 BACKEND ///// My Leela Ratio when using the cudnn-fp16 backend is 1.64 We want to establish a 1:1 ratio between GPU and CPU A Leela to CPU ratio of 1.64 means the GPU is quite stronger here. 64% stronger To get it to 1.0, the ideal 1:1 ratio, we would multiply the time control by 1.64 for the CPU opponents To make things fair, Stockfish and other CPU opponents will need 64% more time added to their clock (Take the time in seconds and multiply it by 1.64.) My LeelaFish Ratio is 3.839, so the CPU engines will still need about 3.8x more time to make things fair --------------- --------------- For fun, let's see what happens when I use the cudnn backend, instead of the faster cudnn-fp16 backend: ///// BEGIN OUTPUT USING CUDNN BACKEND ///// C:\Program Files (x86)\Python\leela-ratio-master>"c:\program files (x86)\python\python.exe" ratio.py ratio.py:2: DeprecationWarning: The chess.uci module is deprecated in favor of chess.engine . Please consider updating and open an issue if your use case is not covered by the new API. import chess.uci Running stockfish to depth 27. 11450235 nps _ | _ | | |_ |_ |_| v0.20.2 built Feb 1 2019 Found configuration file: /chess/engines/Lc0/20.2/lc0-v0.20.2-win-cuda/lc0.config Running leela to 200000 nodes. Found pb network file: /chess/engines/Lc0/20.2/lc0-v0.20.2-win-cuda/networks/2dec1b05388be1547f97e5d40e526ebbd5bb293bd757982ee214571f949571bb Creating backend [cudnn]... GPU: GeForce RTX 2070 GPU memory: 8 Gb GPU clock frequency: 1620 MHz GPU compute capability: 7.5 CUDA Runtime version: 10.0.0 Cudnn version: 7.3.1 Latest version of CUDA supported by the driver: 10.1.0 WARNING: you will probably get better performance from the cudnn-fp16 backend. 8637 nps === GPU: GTX 2070 Stockfish threads: 6 Stockfish hash: 2048 Stockfish depth: 27 Stockfish nps: 11450235 Leela threads: 2 Leela nodes: 200000 Leela nps: 8637 Leela Ratio: 0.66 <----The value we need Leelafish Ratio: 1.545 <----The value we need === ///// END OUTPUT USING CUDNN BACKEND ///// Now my Leela Ratio is only 0.66, meaning Lc0 will need 33% more time on her clock to make things fair (Take the time in seconds and multiply it by 1.33) My LeelaFish Ratio is 1.545, so the CPU engines will still need 54% more time to make things fair 10) Rename your lc0.config file to something like lc0.leelaratio. Or just delete the file or move it to a different directory. Why? Because having this file in the same directory as lc0.exe will set its configuration to match your file every time. If you'd like to play with options in the GUI, this file shouldn't be in the same directory as lc0.exe I hope this was helpful and saves you time and headache.