Short Notes: Get CUDA and gputools Running on Ubuntu 14.10

Here's a basic guide for getting CUDA 7.0 and the R package gputools running perfectly under Ubuntu 14.10. It's not difficult, but there are a few issues and this will be helpful to have in a single place.

If you're running Ubuntu 14.10, I'd recommend installing CUDA 7.0. NVIDIA has a 7.0 Debian package specifically for 14.10; this wasn't the case for CUDA 6.5, which only had a Debian package for 14.04.

To get access to CUDA 7.0, you'll first need to register as a CUDA developer.

Join The CUDA Registered Developer Program

Once you have access, navigate to the CUDA 7.0 download page and get the Debian package.

CUDA 7.0 Release Candidate Downloads

You'll either need to be running the NVIDIA 340 or 346 drivers. If you're having trouble upgrading, I'd suggest adding the xorg-edgers PPA.

	sudo add-apt-repository ppa:xorg-edgers/ppa
	sudo apt-get update
	sudo apt-get install nvidia-346

view raw xorg-edgers_PPA.sh hosted with ❤ by GitHub

Once your NVIDIA driver is set, install the CUDA 7.0 Debian package you've downloaded. Don't forget to remove any previously installed CUDA packages or repositories.

	sudo dpkg -i cuda-repo-ubuntu1410-7-0-rc_7.0-18_amd64.deb
	sudo apt-get install cuda

view raw install_CUDA.sh hosted with ❤ by GitHub

You'll need to add paths so everything knows where CUDA is installed. Append the following to the .bashrc in your home directory:

	# CUDA
	# Works with CUDA 7.0, as the NVIDIA CUDA 7.0 Debian package has a symlink set at /usr/local/cuda.
	export CUDA_HOME=/usr/local/cuda
	export PATH=${CUDA_HOME}/bin:${PATH}
	export LD_LIBRARY_PATH=${CUDA_HOME}/lib64:$LD_LIBRARY_PATH
	# R
	export R_INC_PATH=/usr/lib/R/include:${LD_LIBRARY_PATH}

view raw bash_variables.txt hosted with ❤ by GitHub

Execute "source ~/.bashrc" for these changes to be applied. If you want to test your new CUDA install, make the samples provided by NVIDIA.

	mkdir ~/cuda_test
	cp -R /usr/local/cuda/samples/* ~/cuda_test
	cd ~/cuda_test
	make

view raw make_CUDA_samples.sh hosted with ❤ by GitHub

I get the following output when running BlackScholes:

	[./bin/x86_64/linux/release/BlackScholes] - Starting...
	GPU Device 0: "GeForce GTX TITAN" with compute capability 3.5
	Initializing data...
	...allocating CPU memory for options.
	...allocating GPU memory for options.
	...generating input data in CPU mem.
	...copying input data to GPU mem.
	Data init done.
	Executing Black-Scholes GPU kernel (512 iterations)...
	Options count : 8000000
	BlackScholesGPU() time : 0.343826 msec
	Effective memory bandwidth: 232.675712 GB/s
	Gigaoptions per second : 23.267571
	BlackScholes, Throughput = 23.2676 GOptions/s, Time = 0.00034 s, Size = 8000000 options, NumDevsUsed = 1, Workgroup = 128
	Reading back GPU results...
	Checking the results...
	...running CPU calculations.
	Comparing the results...
	L1 norm: 1.771699E-07
	Max absolute error: 1.192093E-05
	Shutting down...
	...releasing GPU memory.
	...releasing CPU memory.
	Shutdown done.
	[BlackScholes] - Test Summary
	NOTE: The CUDA Samples are not meant for performance measurements. Results may vary when GPU Boost is enabled.
	Test passed

view raw BlackScholes.txt hosted with ❤ by GitHub

The next task is to install gputools for R. You can't unfortunately install the current package through R, as the source code contains references to CUDA architectures that are obsolete under CUDA 7.0. But that's easy to fix.

git clone https://github.com/nullsatz/gputools

view raw clone_gputools.sh hosted with ❤ by GitHub

Now do some editing in gputools/src/Makefile:

	# Architectures 1.0 and 1.3 are obsolete
	# Add newer architectures 3.5, 3.7, 5.0 and 5.2
	# Replace:
	#NVCC := $(CUDA_HOME)/bin/nvcc -gencode arch=compute_10,code=sm_10 -gencode arch=compute_13,code=sm_13 -gencode arch=compute_20,code=sm_20 -gencode arch=compute_30,code=sm_30
	# With:
	NVCC := $(CUDA_HOME)/bin/nvcc -gencode arch=compute_20,code=sm_20 -gencode arch=compute_30,code=sm_30 -gencode arch=compute_35,code=sm_35 -gencode arch=compute_37,code=sm_37 -gencode arch=compute_50,code=sm_50 -gencode arch=compute_52,code=sm_52 -gencode arch=compute_52,code=compute_52

view raw gputools_patch.txt hosted with ❤ by GitHub

Now build and install the patched gputools package while you're in the directory immediately above gputools:

	R CMD build gputools
	R CMD INSTALL gputools_0.28.tar.gz

view raw gputools_local.sh hosted with ❤ by GitHub

If you want to make the gputools packages available for all R users

	sudo -s
	export CUDA_HOME=/usr/local/cuda
	export PATH=${CUDA_HOME}/bin:${PATH}
	export LD_LIBRARY_PATH=${CUDA_HOME}/lib64:$LD_LIBRARY_PATH
	export R_INC_PATH=/usr/lib/R/include:${LD_LIBRARY_PATH}
	R CMD INSTALL gputools_0.28.tar.gz

view raw gputools_global.sh hosted with ❤ by GitHub

Keep in mind that they'll have to make the same environmental variable changes as above. Let's test it!

	library(gputools)
	set.seed(5446)
	p <- 20
	X <- matrix(rnorm(2^p),ncol = 2^(p/2))
	dtime <- system.time(d <- dist(X))
	gputime <- system.time(gpud <- gpuDist(X))
	dtime
	gputime
	dtime/gputime
	max(abs(c(d) - c(gpud)))

view raw example.R hosted with ❤ by GitHub

Running gives us:

	> library(gputools)
	>
	> set.seed(5446)
	> p <- 20
	> X <- matrix(rnorm(2^p),ncol = 2^(p/2))
	>
	> dtime <- system.time(d <- dist(X))
	> gputime <- system.time(gpud <- gpuDist(X))
	>
	> dtime
	user system elapsed
	4.995 0.002 5.002
	> gputime
	user system elapsed
	0.193 0.079 0.278
	> dtime/gputime
	user system elapsed
	25.88082902 0.02531646 17.99280576
	> max(abs(c(d) - c(gpud)))
	[1] 6.05222e-06

view raw example.txt hosted with ❤ by GitHub

A nice 26-fold speedup. We're all set!