Octave and separate toolchain

From Octave
Revision as of 13:37, 1 November 2016 by Ederag (talk | contribs) (octave itself)
Jump to navigation Jump to search
The printable version is no longer supported and may have rendering errors. Please update your browser bookmarks and please use the default browser print function instead.

Building of octave together with a separate toolchain (custom built gcc/binutils/...).

This is required mainly when the base linux distribution ships too old a version of gcc. Unexperienced users should upgrade their distribution, though.

The main build directory will be called /usr/local/build. It was first created by root, and then changed to be owned by the user who will perform compilations. You might want change this to another directory in your $HOME for instance.

export buildroot=/usr/local/build

We will need a directory to hold downloaded files

export downloads=$buildroot/downloads
mkdir -p $buildroot/downloads


gcc toolchain

This part is largely inspired by this answer

cd $buildroot
# work in a subshell, to isolate mistakes
bash
# check that PATH and LD_RUN_PATH are normal
echo $PATH
echo $LD_RUN_PATH
cd $downloads
wget ftp://ftp.gnu.org/gnu/binutils/binutils-2.27.tar.bz2
wget ftp://ftp.gnu.org/gnu/gcc/gcc-6.2.0/gcc-6.2.0.tar.bz2
wget ftp://ftp.gnu.org/gnu/mpfr/mpfr-3.1.5.tar.xz
wget ftp://ftp.gnu.org/gnu/mpc/mpc-1.0.3.tar.gz
wget ftp://ftp.gnu.org/gnu/gmp/gmp-6.1.1.tar.xz
wget http://isl.gforge.inria.fr/isl-0.17.tar.xz
cd $buildroot
tar --no-same-owner -xvf $downloads/binutils-2.27.tar.bz2
tar --no-same-owner -xvf $downloads/gcc-6.2.0.tar.bz2
tar --no-same-owner -xvf $downloads/mpfr-3.1.5.tar.xz
tar --no-same-owner -xvf $downloads/mpc-1.0.3.tar.gz
tar --no-same-owner -xvf $downloads/gmp-6.1.1.tar.xz
tar --no-same-owner -xvf $downloads/isl-0.17.tar.xz
cd $buildroot/gcc-6.2.0
ln -s ../mpfr-3.1.5 mpfr
ln -s ../mpc-1.0.3 mpc
ln -s ../gmp-6.1.1 gmp
ln -s ../isl-0.17/ isl
# Now symlink everything from the binutils dir which doesn't exist in the gcc dir,
# so anything which already exists will take priority but the
# binutils tools will look be visible to the build:
for file in ../binutils-2.27/* ; do ln -s "${file}" ; done
# errors ending with "File exists" are normal

Now we are ready to build the toolchain

cd $buildroot
mkdir gcc-6.2.0_isl_build; cd $_
../gcc-6.2.0/configure --prefix=/usr/local/gcc-6.2.0_binutils-2.27_isl --disable-nls --enable-languages=c,c++,fortran,java --disable-multilib
make -j 6 bootstrap
sudo make install
# see what has been installed
ls /usr/local/gcc-6.2.0_binutils-2.27_isl/bin


OpenBLAS

We need an optimized BLAS library, such as OpenBLAS

cd $downloads
wget -O OpenBLAS-0.2.19.tar.gz http://github.com/xianyi/OpenBLAS/archive/v0.2.19.tar.gz
cd $buildroot
tar --no-same-owner -xvf $downloads/OpenBLAS-0.2.19.tar.gz
cd OpenBLAS-0.2.19/

bash
export PATH=/usr/local/gcc-6.2.0_binutils-2.27_isl/bin:$PATH
# check
which gcc
# should be /usr/local/gcc-6.2.0_binutils-2.27_isl/bin/gcc
which gfortran
# should be /usr/local/gcc-6.2.0_binutils-2.27_isl/bin/gfortran

# do not use threads 
# (because I use pararrayfun from the parallel package)
make USE_THREAD=0 CPPFLAGS="-I/usr/local/gcc-6.2.0_binutils-2.27_isl/include" CFLAGS="-O2 -march=native -L/usr/local/gcc-6.2.0_binutils-2.27_isl/lib64" CXXFLAGS="-O2 -march=native -L/usr/local/gcc-6.2.0_binutils-2.27_isl/lib64" FFLAGS="-O2 -march=native -L/usr/local/gcc-6.2.0_binutils-2.27_isl/lib64" LDFLAGS=-Wl,-rpath,/usr/local/gcc-6.2.0_binutils-2.27_isl/lib64 

One should get

OpenBLAS build complete. (BLAS CBLAS LAPACK LAPACKE)

 OS               ... Linux             
 Architecture     ... x86_64               
 BINARY           ... 64bit                 
 C compiler       ... GCC  (command line : gcc)
 Fortran compiler ... GFORTRAN  (command line : gfortran)
 Library Name     ... libopenblas_haswell-r0.2.19.a (Single threaded)  

Checking the runpath:

ldd libopenblas.so
       linux-vdso.so.1 (0x00007fff961fc000)
       libm.so.6 => /lib64/libm.so.6 (0x00007f3ee88cb000)
       libgfortran.so.3 => /usr/local/gcc-6.2.0_binutils-2.27_isl/lib64/libgfortran.so.3 (0x00007f3ee85a3000)
       libc.so.6 => /lib64/libc.so.6 (0x00007f3ee81f9000)
       libquadmath.so.0 => /usr/local/gcc-6.2.0_binutils-2.27_isl/lib64/../lib64/libquadmath.so.0 (0x00007f3ee7fb9000)
       libgcc_s.so.1 => /usr/local/gcc-6.2.0_binutils-2.27_isl/lib64/../lib64/libgcc_s.so.1 (0x00007f3ee7da2000)
       /lib64/ld-linux-x86-64.so.2 (0x00007f3ee9ad6000)

Manual installation

sudo cp libopenblas.a libopenblas_haswell-r0.2.19.a libopenblas_haswell-r0.2.19.so /usr/local/gcc-6.2.0_binutils-2.27_isl/lib64
cd /usr/local/gcc-6.2.0_binutils-2.27_isl/lib64/
sudo ln -s libopenblas_haswell-r0.2.19.so libopenblas.so.0
sudo ln -s libopenblas.so.0 libopenblas.so

arpack-ng

https://github.com/opencollab/arpack-ng/releases

cd $downloads
wget -O arpack-ng-3.4.0.tar.gz https://github.com/opencollab/arpack-ng/archive/3.4.0.tar.gz

Start from a clean state

exit
bash
export PATH=/usr/local/gcc-6.2.0_binutils-2.27_isl/bin:$PATH
which gfortran
# should be /usr/local/gcc-6.2.0_binutils-2.27_isl/bin/gfortran
cd $buildroot
tar --no-same-owner -xvf $downloads/arpack-ng-3.4.0.tar.gz
cd arpack-ng-3.4.0
./bootstrap
./configure CPPFLAGS="-I/usr/local/gcc-6.2.0_binutils-2.27_isl/include" CFLAGS="-O2 -march=native -L/usr/local/gcc-6.2.0_binutils-2.27_isl/lib64" CXXFLAGS="-O2 -march=native -L/usr/local/gcc-6.2.0_binutils-2.27_isl/lib64" FFLAGS="-O2 -march=native -L/usr/local/gcc-6.2.0_binutils-2.27_isl/lib64" LDFLAGS=-Wl,-rpath,/usr/local/gcc-6.2.0_binutils-2.27_isl/lib64 --prefix=/usr/local/gcc-6.2.0_binutils-2.27_isl

make -j 6
make check
 # TOTAL: 12
 # PASS:  12
# check libraries
ldd ./SRC/.libs/libarpack.so
       linux-vdso.so.1 (0x00007fff1f38c000)
       libopenblas.so.0 => /usr/local/gcc-6.2.0_binutils-2.27_isl/lib64/libopenblas.so.0 (0x00007f80c79d8000)
       libgfortran.so.3 => /usr/local/gcc-6.2.0_binutils-2.27_isl/lib64/libgfortran.so.3 (0x00007f80c76b0000)
       libm.so.6 => /lib64/libm.so.6 (0x00007f80c7368000)
       libgcc_s.so.1 => /usr/local/gcc-6.2.0_binutils-2.27_isl/lib64/libgcc_s.so.1 (0x00007f80c7151000)
       libquadmath.so.0 => /usr/local/gcc-6.2.0_binutils-2.27_isl/lib64/libquadmath.so.0 (0x00007f80c6f11000)
       libc.so.6 => /lib64/libc.so.6 (0x00007f80c6b67000)
       /lib64/ld-linux-x86-64.so.2 (0x00007f80c892e000)

as root (su, not sudo, because we want PATH to contain /usr/local/gcc-6.2.0_binutils-2.27_isl/bin)

make install

exit root session, then exit current bash, to return to a clean state

SuiteSparse

http://faculty.cse.tamu.edu/davis/suitesparse.html

cd $downloads
wget http://faculty.cse.tamu.edu/davis/SuiteSparse/SuiteSparse-4.5.3.tar.gz

Save the code below to $downloads/UMFPACK_Demo_Makefile-4.5.3.patch

Code: UMFPACK_Demo_Makefile-4.5.3.patch
--- UMFPACK/Demo/Makefile.orig	2016-09-19 16:20:46.317044240 +0200
+++ UMFPACK/Demo/Makefile	2016-09-19 16:30:15.079302070 +0200
@@ -27,9 +27,9 @@
 C = $(CC) $(CF) $(UMFPACK_CONFIG) $(CONFIG_PARTITION) \
     -I../../include
 
-LIBS = $(LDLIBS) -L../../lib -lumfpack -lamd -lsuitesparseconfig \
+LIBS = $(LDFLAGS) $(LDLIBS) -L../../lib -lumfpack -lamd -lsuitesparseconfig \
 	$(LIB_WITH_CHOLMOD) $(LAPACK) $(BLAS)
 
 libs: metis
 	( cd ../../SuiteSparse_config ; $(MAKE) )
 	( cd ../../AMD ; $(MAKE) library )


cd $buildroot
tar --no-same-owner -xvf $downloads/SuiteSparse-4.5.3.tar.gz
mv SuiteSparse SuiteSparse-4.5.3
cd SuiteSparse-4.5.3/

patch -p0 < $downloads/UMFPACK_Demo_Makefile-4.5.3.patch
# should get: "patching file UMFPACK/Demo/Makefile"

If the following command

find . -name Makefile -exec grep "LDLIBS +=" {} \; -print

yields LDLIBS lines not starting with override, then the patch sent upstream has not been applied, and we should do it here:

find . -name Makefile -exec sed -i 's/LDLIBS +=/override LDLIBS +=/g' {} \;

Now we are ready to build SuiteSparse

make -j6 LAPACK= CF="-march=native -O3 -fexceptions -fPIC -fopenmp" F77FLAGS="-march=native -O3" LDFLAGS="-L/usr/local/build/SuiteSparse-4.5.3/lib -L/usr/local/gcc-6.2.0_binutils-2.27_isl/lib64" LDLIBS="-lm -lrt -Wl,-rpath=/usr/local/build/SuiteSparse-4.5.3/lib:/usr/local/gcc-6.2.0_binutils-2.27_isl/lib64"

fix RUN_PATH and install

cp -Rdp lib lib_chrpath
patchelf --set-rpath /usr/local/gcc-6.2.0_binutils-2.27_isl/lib64 lib_chrpath/*
sudo cp lib_chrpath/* /usr/local/gcc-6.2.0_binutils-2.27_isl/lib64/
sudo mkdir /usr/local/gcc-6.2.0_binutils-2.27_isl/include/suitesparse/
sudo cp include/* /usr/local/gcc-6.2.0_binutils-2.27_isl/include/suitesparse/


GraphicsMagick

download GraphicsMagick (here, GraphicsMagick-1.3.25.tar.xz) to $downloads

https://sourceforge.net/projects/graphicsmagick/files/graphicsmagick/

Start from a clean state

exit
bash
export PATH=/usr/local/gcc-6.2.0_binutils-2.27_isl/bin:$PATH
which gfortran
# should be /usr/local/gcc-6.2.0_binutils-2.27_isl/bin/gfortran
cd $buildroot
tar --no-same-owner -xvf $downloads/GraphicsMagick-1.3.25.tar.xz
cd GraphicsMagick-1.3.25

Disable webp and jbig that fail tests with gcc-6.2.0

./configure --without-webp --without-jbig --enable-static=no --enable-shared=yes --with-quantum-depth=16 CPPFLAGS="-I/usr/local/gcc-6.2.0_binutils-2.27_isl/include" CFLAGS="-O2 -march=native -L/usr/local/gcc-6.2.0_binutils-2.27_isl/lib64" CXXFLAGS="-O2 -march=native -L/usr/local/gcc-6.2.0_binutils-2.27_isl/lib64" FFLAGS="-O2 -march=native -L/usr/local/gcc-6.2.0_binutils-2.27_isl/lib64" LDFLAGS=-Wl,-rpath,/usr/local/gcc-6.2.0_binutils-2.27_isl/lib64 PKG_CONFIG_PATH=/usr/local/gcc-6.2.0_binutils-2.27_isl/lib64/pkgconfig --prefix=/usr/local/gcc-6.2.0_binutils-2.27_isl

The summary should be fine. As comparison and source of relevant informations: GraphicsMagick

make -j 6
make -j6 check
============================================================================
Testsuite summary for GraphicsMagick 1.3.25
============================================================================
# TOTAL: 1215
# PASS:  1169
# SKIP:  46
# XFAIL: 0
# FAIL:  0
# XPASS: 0
# ERROR: 0


as root (su, not sudo, because we want PATH to contain /usr/local/gcc-6.2.0_binutils-2.27_isl/bin)

make install

exit root session, then exit current bash, to return to a clean state


Octave itself

cd $downloads
wget ftp://alpha.gnu.org/gnu/octave/octave-4.2.0-rc3.tar.lz

Start from a clean state

exit
bash
export PATH=/usr/local/gcc-6.2.0_binutils-2.27_isl/bin:$PATH
which gfortran
# should be /usr/local/gcc-6.2.0_binutils-2.27_isl/bin/gfortran
cd $buildroot
tar --no-same-owner -xvf $downloads/octave-4.2.0-rc3.tar.lz
cd octave-4.2.0-rc3
mkdir build_; cd $_
../configure --with-blas="/usr/local/gcc-6.2.0_binutils-2.27_isl/lib64/libopenblas.so" CPPFLAGS="-I/usr/local/gcc-6.2.0_binutils-2.27_isl/include" CFLAGS="-O2 -march=native -L/usr/local/gcc-6.2.0_binutils-2.27_isl/lib64" CXXFLAGS="-O2 -march=native -L/usr/local/gcc-6.2.0_binutils-2.27_isl/lib64" FFLAGS="-O2 -march=native -L/usr/local/gcc-6.2.0_binutils-2.27_isl/lib64" LDFLAGS=-Wl,-rpath,/usr/local/gcc-6.2.0_binutils-2.27_isl/lib64 PKG_CONFIG_PATH=/usr/local/gcc-6.2.0_binutils-2.27_isl/lib64/pkgconfig JAVA_HOME=/usr/lib64/jvm/java-1.8.0-openjdk-1.8.0/ --prefix=/usr/local/gcc-6.2.0_binutils-2.27_isl

make -j 6
make check
# or, for nvidia users,
xvfb-run -s "-screen 0 640x480x24" make check LD_PRELOAD=/usr/lib64/libGL.so

To be continued...