In addition to Chainer, ChainerMN depends on the following software libraries: CUDA-Aware MPI, NVIDIA NCCL, and a few Python packages including MPI4py.
ChainerMN adds distributed training features to Chainer; thus it naturally requires Chainer. Please refer to the official instructions to install.
ChainerMN relies on MPI. In particular, for efficient communication between GPUs, it uses CUDA-aware MPI. For details about CUDA-aware MPI, see this introduction article. (If you use only the CPU mode, MPI does not need to be CUDA-Aware. See Non-GPU environments for more details.)
The CUDA-aware features depend on several MPI packages, which need to be configured and built properly. The following are examples of Open MPI and MVAPICH.
Open MPI (for details, see the official instructions):
$ ./configure --with-cuda $ make -j4 $ sudo make install
MVAPICH (for details, see the official instructions):
$ ./configure --enable-cuda $ make -j4 $ sudo make install $ export MV2_USE_CUDA=1 # Should be set all the time when using ChainerMN
Please properly configure environment variables to expose NCCL both when you install and use ChainerMN. Typical configurations should look like the following:
export NCCL_ROOT=<path to NCCL directory> export CPATH=$NCCL_ROOT/include:$CPATH export LD_LIBRARY_PATH=$NCCL_ROOT/lib/:$LD_LIBRARY_PATH export LIBRARY_PATH=$NCCL_ROOT/lib/:$LIBRARY_PATH
ChainerMN requires NCCL even if you have only one GPU per node. The only exception is when you run ChainerMN on CPU-only environments. See Non-GPU environments for more details.
ChainerMN depends on a few Python packages, which are automatically installed when you install ChainerMN.
However, among them, we need to be a little careful about MPI4py. It links to MPI at installation time, so please be sure to properly configure environment variables so that MPI is available at installation time. In particular, if you have multiple MPI implementations in your environment, please expose the implementation that you want to use both when you install and use ChainerMN.
In addition, Cython may not be installed automatically. It can be installed manually via pip:
$ pip install cython
Install via pip¶
We recommend to install ChainerMN via pip:
$ pip install chainermn
Install from Source¶
You can use
setup.py to install ChainerMN from source:
$ tar zxf chainermn-x.y.z.tar.gz $ cd chainermn-x.y.z $ python setup.py install
For users who want to try ChainerMN in a CPU-only environment,
typically for testing for debugging purpose, ChainerMN can be built
$ python setup.py install --no-nccl
In this case, the MPI does not have to be CUDA-aware.
naive communicator works with the CPU mode.