Build OpenCV 2.4.9 & Caffe with CUDA 9.0 on Ubuntu 16.04

There are quite a lot of things you could feel annoying about the research. One of the most annoying I found recently, is trying to spending a couple of days only to build an environment for experiment while have not started any experiment yet. Here I record my procedure of building Caffe and OpenCV, with CUDA on my Ubuntu system.

# An Important Thing

The most important thing that I learned in my path of building Caffe is — Never think of restarting from scratch is a waste of time. I finally succeeded by re-installing the OS (Ubuntu) and then install all of the things one by one.

# Prerequisite

• Ubuntu 16.04
• Python 2.7
• Cuda 9.0 (9.1 should also work), ensure your CUDA is compatible with your GPU
• cudnn 7.1 (7.0 should also work)

# Installing OpenCV

Firstly, install some depedencies as following: (you can put them into a shell file and then execute it.)

sudo apt-get install build-essential
sudo apt-get install cmake git libgtk2.0-dev pkg-config libavcodec-dev libavformat-dev libswscale-dev
sudo apt-get install python-dev python-numpy libtbb2 libtbb-dev libjpeg-dev libpng-dev libtiff-dev libjasper-dev libdc1394-22-dev

sudo apt-get install libtiff4-dev libopenexr-dev libeigen2-dev yasm libopencore-amrnb-dev libtheora-dev libvorbis-dev libxvidcore-dev

sudo apt-get install python-tk libeigen3-dev libx264-dev libqt4-dev libqt4-opengl-dev sphinx-common texlive-latex-extra libv4l-dev default-jdk ant libvtk5-qt4-dev

Get opencv-2.4.9 using wget:

wget http://sourceforge.net/projects/opencvlibrary/files/opencv-unix/2.4.9/opencv-2.4.9.zip
unzip opencv-2.4.9.zip
cd opencv-2.4.9
mkdir build
cd build

[Important] Configure the file for cuda-9.0. There are some issues for cuda≥9 to use OpenCV 2.X version so we need to configure it. According to [4, 6] and other resources, there are four files we need to modify:

1. opencv-2.4.9/cmake/FindCUDA.cmake

vim ../cmake/FindCUDA.cmake

(a) find this line and replace it

find_cuda_helper_libs(nppi)

with

find_cuda_helper_libs(nppial)
find_cuda_helper_libs(nppicc)
find_cuda_helper_libs(nppicom)
find_cuda_helper_libs(nppidei)
find_cuda_helper_libs(nppif)
find_cuda_helper_libs(nppig)
find_cuda_helper_libs(nppim)
find_cuda_helper_libs(nppist)
find_cuda_helper_libs(nppisu)
find_cuda_helper_libs(nppitc)

(b) find the line

set(CUDA_npp_LIBRARY "${CUDA_nppc_LIBRARY};${CUDA_nppi_LIBRARY};${CUDA_npps_LIBRARY}")

and replace with

set(CUDA_npp_LIBRARY "${CUDA_nppc_LIBRARY};${CUDA_nppial_LIBRARY};${CUDA_nppicc_LIBRARY};${CUDA_nppicom_LIBRARY};${CUDA_nppidei_LIBRARY};${CUDA_nppif_LIBRARY};${CUDA_nppig_LIBRARY};${CUDA_nppim_LIBRARY};${CUDA_nppist_LIBRARY};${CUDA_nppisu_LIBRARY};${CUDA_nppitc_LIBRARY};${CUDA_npps_LIBRARY}")

(c) find the line

unset(CUDA_nppi_LIBRARY CACHE)

and replace with

unset(CUDA_nppial_LIBRARY CACHE)
unset(CUDA_nppicc_LIBRARY CACHE)
unset(CUDA_nppicom_LIBRARY CACHE)
unset(CUDA_nppidei_LIBRARY CACHE)
unset(CUDA_nppif_LIBRARY CACHE)
unset(CUDA_nppig_LIBRARY CACHE)
unset(CUDA_nppim_LIBRARY CACHE)
unset(CUDA_nppist_LIBRARY CACHE)
unset(CUDA_nppisu_LIBRARY CACHE)
unset(CUDA_nppitc_LIBRARY CACHE)

(d) find the line

set(nvcc_flags "")

and replace with

set(nvcc_flags "--expt-relaxed-constexpr")

2. opencv-2.4.9/cmake/OpenCVDetectCUDA.cmake

vim ../cmake/OpenCVDetectCUDA.cmake

(a) find and replace the line with this:

set(_generations "Fermi" "Kepler" "Maxwell" "Pascal" "Volta")

(b) find the code segment and replace with:

set(__cuda_arch_ptx "")
  if(CUDA_GENERATION STREQUAL "Fermi")
    set(__cuda_arch_bin "2.0")
  elseif(CUDA_GENERATION STREQUAL "Kepler")
    set(__cuda_arch_bin "3.0 3.5 3.7")
  elseif(CUDA_GENERATION STREQUAL "Maxwell")
    set(__cuda_arch_bin "5.0 5.2")
  elseif(CUDA_GENERATION STREQUAL "Pascal")
    set(__cuda_arch_bin "6.0 6.1")
  elseif(CUDA_GENERATION STREQUAL "Volta")
    set(__cuda_arch_bin "7.0")
  elseif(CUDA_GENERATION STREQUAL "Auto")
    ... # no modification

3. configure modules/gpu/src/graphcuts.cpp:

find the line:

#if !defined (HAVE_CUDA) || defined (CUDA_DISABLER)

and replace it with

#if !defined (HAVE_CUDA) || defined (CUDA_DISABLER) || (CUDART_VERSION >= 8000)

4. Download the file NCVPixelOperations.hpp from here:

NCVPixelOperations.hpp

and use it to replace the original NCVPixelOperations.hpp:

mv ~/Downloads/NCVPixelOperations.hpp opencv-2.4.9/modules/gpu/src/nvidia/core/NCVPixelOperations.hpp

Finally, cmake the files at opencv-2.4.9/build , build and install:

cmake -D WITH_TBB=ON -D BUILD_NEW_PYTHON_SUPPORT=ON -D WITH_V4L=ON -D INSTALL_C_EXAMPLES=ON -D INSTALL_PYTHON_EXAMPLES=ON -D BUILD_EXAMPLES=ON -D WITH_QT=ON -D WITH_OPENGL=ON -D WITH_VTK=ON -D CUDA_GENERATION="Pascal" ..
sudo make -j4
sudo make install

Once the installation is complete, do these steps to get OpenCV configured.

sudo sh -c 'echo "/usr/local/lib" > /etc/ld.so.conf.d/opencv.conf'

sudo ldconfig
# Ensure under /usr/local/lib there are libopencv* files. If not, 
# find out the directory and append it to the path.
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/lib

Then you can now test your OpenCV installation using:

$ python
Python 2.7.12 (default, Dec  4 2017, 14:50:18) 
[GCC 5.4.0 20160609] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import cv2
>>> cv2.__version__
'2.4.9'

Congrats! You finished installing the OpenCV 2.4.9!

# Install Caffe

In general, follow this official guide and take note of some points:

• If you hope to reproduce people’s work in GitHub, be sure to build the same version of caffe as they did. For example, in this GitHub repo I needed to set the following flags:

# In your Makefile.config, make sure to have these lines uncommented
WITH_PYTHON_LAYER := 1
USE_NCCL := 1
# Unrelatedly, it's also recommended that you use CUDNN
USE_CUDNN := 1

• In the official guide it’s saying adding this line:

LIBRARIES += glog gflags protobuf leveldb snappy \
  lmdb boost_system boost_filesystem hdf5_hl hdf5 m \
  opencv_core opencv_highgui opencv_imgproc opencv_imgcodecs opencv_videoio

However, the last two are not needed in OpenCV 2, so remove them:

LIBRARIES += glog gflags protobuf leveldb snappy \
  lmdb boost_system boost_filesystem hdf5_hl hdf5 m \
  opencv_core opencv_highgui opencv_imgproc

Then, follow the commands in the official guide. Hopefully you successfully build and install the Caffe, you may then check the installation:

$ python
Python 2.7.12 (default, Dec  4 2017, 14:50:18) 
[GCC 5.4.0 20160609] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import caffe
>>> # Works normally.

Congrats! You can now enjoy your freshly brewed Caffe :)

References

1. Official Guide, https://github.com/BVLC/caffe/wiki/Ubuntu-16.04-or-15.10-Installation-Guide
2. Installation note of OpenCV 2.4.9 and Caffe, https://gist.github.com/arundasan91/b432cb011d1c45b65222d0fac5f9232c#2-caffe-without-installing-anaconda
3. Installation note of OpenCV 2.4.9 (in Chinese), https://blog.csdn.net/Solomon1558/article/details/51967280
4. A patch for OpenCV 2.4.X for CUDA 9, https://davidstutz.de/compiling-opencv-2-4-x-with-cuda-9/
5. Some error message solutions when building OpenCV (in Chinese), https://blog.csdn.net/hehehetanchaow/article/details/77530510
6. Another article stating OpenCV 2.4.X for CUDA 9, https://stackoverflow.com/questions/46584000/cmake-error-variables-are-set-to-notfound