Installation Guide¶
This page gives instructions of how to build and install the mxnet package from
scratch on various systems. It consists of two steps, first we build the shared
library from the C++ codes (libmxnet.so
for linux/osx and libmxnet.dll
for
windows). Then we install the language, e.g. Python, packages. If the
instructions on this page do not work for you, please feel free to ask questions
at mxnet/issues, or even better to send
pull request if you can fix the problem.
Contents¶
Python Package Installation¶
The python package is located at mxnet/python. It requires
python>=2.7
and numpy
. To install the latter, if pip
is available, then
sudo pip install numpy
otherwise use your package manager, e.g.
sudo apt-get install python-numpy # for debian
sudo yum install python-numpy # for redhat
To have a quick test of the python package, we can train a MLP on the mnist dataset:
python example/image-classification/train_mnist.py
or train a convolution neural network using GPU 0 if we set USE_CUDA=1
during
compiling:
python example/image-classification/train_mnist.py --network lenet --gpus 0
There are several ways to install the package:
Install system-widely, which requires root permission
cd python; sudo python setup.py install
You will however need Python
distutils
module for this to work. It is often part of the core python package or it can be installed using your package manager, e.g. in Debian usesudo apt-get install python-setuptools
Only set the environment variable
PYTHONPATH
to tell python where to find the library. For example, assume we clonedmxnet
on the home directory~
. then we can added the following line in~/.bashrc
It is recommended for developers who may change the codes. The changes will be immediately reflected once you pulled the code and rebuild the project (no need to callsetup
again)export PYTHONPATH=~/mxnet/python
Install only for the current user.
cd python; python setup.py develop --user
Copy the package into the working directory which contains the mxnet application programs. In this approach we don’t need to change the system, and therefore is recommended for distributed training.
Assume we are on the working directory, and
mxnet
is cloned on the home directory~
.cp -r ~/mxnet/python/mxnet . cp ~/mxnet/lib/libmxnet.so mxnet/
R Package Installation¶
For Windows/Mac users, we provide pre-built binary package using CPU. You can install weekly updated package directly in R console:
install.packages("drat", repos="https://cran.rstudio.com")
drat:::addRepo("dmlc")
install.packages("mxnet")
To install the R package. First finish the Build MXNet Library step. Then use the following command to install dependencies and build the package at root folder
Rscript -e "install.packages('devtools', repo = 'https://cran.rstudio.com')"
cd R-package
Rscript -e "library(devtools); library(methods); options(repos=c(CRAN='https://cran.rstudio.com')); install_deps(dependencies = TRUE)"
cd ..
make rpkg
Now you should have the R package as a tar.gz file and you can install it as a normal package by (the version number might be different)
R CMD INSTALL mxnet_0.5.tar.gz
If you can’t load mxnet
after enabling CUDA during the installation. Please add following lines into $RHOME/etc/ldpaths
. You can find your $RHOME
by using R.home()
inside R.
export CUDA_HOME=/usr/local/cuda
export LD_LIBRARY_PATH=${CUDA_HOME}/lib64:${LD_LIBRARY_PATH}
To install the package using GPU on Windows without building the package from scratch. Note that you need a couple of programs installed already:
- You’ll need the CUDA Toolkit. This depends on Visual Studio, and a free compatible version would be Visual Studio Community 2013. For instructions and compatibility checks, read http://docs.nvidia.com/cuda/cuda-getting-started-guide-for-microsoft-windows/ .
- You will also need to register as a developer at nvidia and download CUDNN V3, https://developer.nvidia.com/cudnn .
- Download the mxnet package as a ZIP from the Github repository https://github.com/dmlc/mxnet and unpack it. You will be editing the
/mxnet/R-package
folder. - Download the most recent GPU-enabled package from the Releases tab. Unzip this file so you have a folder
/nocudnn
. Note that this file and the folder you’ll save it in will be used for future reference and not directly for installing the package. Only some files will be copied from it into theR-package
folder.
(Note: you now have 2 folders we’re working with, possibly in different locations, that we’ll reference with R-package/
and nocudnn/
.)
- Download CUDNN V3 from https://developer.nvidia.com/cudnn. Unpack the .zip file and you’ll see 3 folders,
/bin
,/include
,/lib
. Copy and replace these 3 folders intonocudnn/3rdparty/cudnn/
, or unpack the .zip file there directly. - Create the folder
R-package/inst/libs/x64
. We only support 64-bit operating system now, so you need the x64 folder; - Put dll files in
R-package/inst/libs/x64
.
The first dll file you need is nocudnn/lib/libmxnet.dll
. The other dll files you need are the ones in all 4 subfolders of nocudnn/3rdparty/
, for the cudnn
and openblas
you’ll need to look in the /bin
folders. There should be 11 dll files now in R-package/inst/libs/x64
.
- Copy the folder
nocudnn/include/
toR-package/inst/
. So now you should have a folderR-package/inst/include/
with 3 subfolders. - Run
R CMD INSTALL --no-multiarch R-package
. Make sure that R is added to your PATH in Environment Variables. Running the commandWhere R
in Command Prompt should return the location.
Note on Library Build:
We isolate the library build with Rcpp end to maximize the portability
- MSVC is needed on windows to build the mxnet library, because of CUDA compatiblity issue of toolchains.
Julia Package Installation¶
The Julia package is hosted in a separate repository MXNet.jl. To use the Julia binding with an existing libmxnet installation, set the following environment variable
export MXNET_HOME=/path/to/libmxnet
The path should be the root directory of libmxnet, in other words, libmxnet.so
should be found at $MXNET_HOME/lib
. You might want to add it to your .bashrc
. Then the Julia package could be installed via
Pkg.add("MXNet")
in a Julia console. For more details, please refer to the full documentation of MXNet.jl.
Scala Package Installation¶
For Linux/Mac users, we provide pre-built binary packages, with GPU or CPU-only supported.
You can use the following dependency in maven, change the artifactId according to your own architecture, e.g., mxnet-full_2.10-osx-x86_64-cpu
for OSX (and cpu-only).
<dependency>
<groupId>ml.dmlc.mxnet</groupId>
<artifactId>mxnet-full_2.10-linux-x86_64-gpu</artifactId>
<version>0.1.1</version>
</dependency>
In case your native environment is slightly different from which the assembly package provides, e.g., you use openblas
instead of atlas
, a more recommended way is to use mxnet-core
and put the compiled Java native library somewhere in your load path.
<dependency>
<groupId>ml.dmlc.mxnet</groupId>
<artifactId>mxnet-core_2.10</artifactId>
<version>0.1.1</version>
</dependency>
To build with your own environment. First finish the Build MXNet Library step. Then run following command from the root directory.
make scalapkg
Now you will find jars for assembly
, core
and example
modules.
Also it produces the native library in native/{your-architecture}/target
, which you can use to cooperate with the core
module.
To install the scala package into your local maven repository, run
make scalainstall
Docker Images¶
Builds of MXNet are available as Docker images: MXNet Docker (CPU) or MXNet Docker (CUDA). These are updated on a weekly basis with the latest builds of MXNet. Examples of running bash in a Docker container are as follows:
sudo docker run -it kaixhin/mxnet
sudo nvidia-docker run -it kaixhin/cuda-mxnet:7.0
For a guide to Docker, see the official docs. CUDA support requires NVIDIA Docker. For more details on how to use the MXNet Docker images, consult the source project.
Build Dependent Libraries from Source¶
This section we provide instructions to build MXNet’ dependent libraries from source. It is often useful in two situations:
- You use a low version or server linux, there is no according packages or the package versions are low by using
yum
orapt-get
- You do not have the root permission to install packages. In this case, you need to change the install directory from
/usr/local
into another one such as${HOME}
in the following examples.
Build GCC from Source¶
Building gcc needs 32-bit libc, you can install it by
- Ubuntu:
sudo apt-get install libc6-dev-i386
- Red Hat
sudo yum install glibc-devel.i686
- CentOS 5.8,
sudo yum install glibc-devel.i386
- CentOS 6 / 7,
sudo yum install glibc-devel.i686
First download
wget http://mirrors.concertpass.com/gcc/releases/gcc-4.8.5/gcc-4.8.5.tar.gz
tar -zxf gcc-4.8.5.tar.gz
cd gcc-4.8.5
./contrib/download_prerequisites
Then build
mkdir release && cd release
../configure --prefix=/usr/local --enable-languages=c,c++
make -j10
sudo make install
Finally you may want to add lib path in your ~/.bashrc
export LD_LIBRARY_PATH=${LD_LIBRARY_PATH}:/usr/local/lib64
Build Opencv from Source¶
First download opencv
wget https://codeload.github.com/opencv/opencv/zip/2.4.13
unzip 2.4.13
cd opencv-2.4.13
mkdir release
cd release/
Building opencv needs cmake, if you do not have cmake or your cmake verion is too low (e.g the one installed by default on RHEL), then
wget https://cmake.org/files/v3.6/cmake-3.6.1-Linux-x86_64.tar.gz
tar -zxvf cmake-3.6.1-Linux-x86_64.tar.gz
alias cmake="cmake-3.6.1-Linux-x86_64/bin/cmake"
Now build opencv. We disable GPU support, which may significantly slow down to run a MXNet program on GPU. We also disable 1394 which may generate warning.
cmake -D BUILD_opencv_gpu=OFF -D WITH_CUDA=OFF -D WITH_1394=OFF -D CMAKE_BUILD_TYPE=RELEASE -D CMAKE_INSTALL_PREFIX=/usr/local ..
make -j8
sudo make install
Finally, you may want to add the following into the end of your ~/.bashrc
:
export PKG_CONFIG_PATH=$PKG_CONFIG_PATH:/usr/local/lib/pkgconfig/