Enable large arrays: Build octave such that it can use arrays larger than 2Gb.: Difference between revisions

From Octave
Jump to navigation Jump to search
No edit summary
No edit summary
Line 10: Line 10:
You will obtain this error even if your system has enough RAM to create this array (4 GB in the above case).
You will obtain this error even if your system has enough RAM to create this array (4 GB in the above case).


To use arrays larger than 2 GB, Octave has to be configured with the option --enable-64. This option is experimental and you are (as always) encouraged to submit bug reports if you find a problem.  
To use arrays with more than (approximately) 2 billion elements, Octave has to be configured with the option --enable-64. This option is experimental and you are (as always) encouraged to submit bug reports if you find a problem.  
With this option, Octave will use internally 64-bit integers for array dimensions and indexing. However, '''all numerical libraries''' used by Octave will need to use also 64-bit integers for array dimensions and indexing, and in most cases they need to be compiled from source.
With this option, Octave will use internally 64-bit integers for array dimensions and indexing. However, '''all numerical libraries''' used by Octave will need to use also 64-bit integers for array dimensions and indexing, and in most cases they need to be compiled from source.


For details about how to compile these libraries please read the [http://www.gnu.org/software/octave/doc/interpreter/Compiling-Octave-with-64_002dbit-Indexing.html#Compiling-Octave-with-64_002dbit-Indexing GNU Octave manual], or alternatively you can use [http://wiki.octave.org/MXE MXE] (M Cross Environment) which takes care of the different packages automatically.
For details about how to compile these libraries please read the [http://www.gnu.org/software/octave/doc/interpreter/Compiling-Octave-with-64_002dbit-Indexing.html#Compiling-Octave-with-64_002dbit-Indexing GNU Octave manual], or alternatively you can use [http://wiki.octave.org/MXE MXE] (M Cross Environment) which takes care of the different packages automatically.
<!--
== BLAS ==
Various implementations of the BLAS library exist. Once you compiled this library make sure that you use this library for compiling LAPACK, <nowiki>SuiteSparse</nowiki>, ARPACK and Octave. In the following we assume that you installed the BLAS library as $BLAS/lib/libblas.a.
=== Reference BLAS ===
To compile this library use the gfortran compiler options -fdefault-integer-8 and -fPIC (for 64-bit CPUs). Either add those options to the variable OPTS in make.inc, or use the following make command:
<pre>
make FORTRAN=gfortran OPTS="-fPIC -O3 -fdefault-integer-8" BLASLIB=libblas.a
</pre>
=== ATLAS ===
<i>Not yet tested.</i>
=== GotoBLAS ===
In Makefile.rule, set your Fortran compiler (F_COMPILER) to GFORTRAN, uncomment the line with INTERFACE64 = 1 and BINARY64 = 1 (if you have a 64-bit CPU) or use the command:
<pre>
make F_COMPILER=GFORTRAN BINARY64=1 INTERFACE64=1
</pre>
Note that <nowiki>GotoBLAS</nowiki> is available free of charge for research but it is not free software.
== LAPACK ==
Copy the file make.inc.example and name it make.inc. The options -fdefault-integer-8 and -fPIC (on 64-bit CPU) have to be added to the variable OPTS and NOOPT. Alternatively, the following command can be used (with some additional optimizations):
<pre>
cp make.inc.example  make.inc
make FORTRAN=gfortran LOADER=gfortran OPTS="-funroll-all-loops -O3 -fPIC -fdefault-integer-8" NOOPT="-fPIC -fdefault-integer-8" PLAT=  \
  LAPACKLIB=liblapack.a  BLASLIB=$BLAS/lib/libblas.a TIMER=INT_ETIME  lapacklib
</pre>
Once you compiled this library make sure that you use this library for compiling Suite Sparse and Octave.
In the following we assume that you installed the LAPACK library as $LAPACK/lib/liblapack.a.
== ARPACK ==
With gfortran 4.2.4 I had to apply the patch from: https://cvs.fedoraproject.org/viewvc/rpms/arpack/F-9/arpack-etime.patch?revision=1.1&view=markup&pathrev=arpack-2_1-10_fc9
In ARmake.inc set the following variables:
<pre>
home=path to directory ARPACK
FC=gfortran
FFLAGS=-fPIC -fdefault-integer-8
MAKE=/usr/bin/make
ARPACKLIB=$(home)/libarpack.a
DIRS=$(UTILdir) $(SRCdir)
</pre>
Alternatively, they can be defined as arguments of make:
<pre>
make FC=gfortran FFLAGS="-fPIC -fdefault-integer-8" MAKE=/usr/bin/make ARPACKLIB=$PWD/libarpack.a home=$PWD DIRS="$PWD/UTIL $PWD/SRC" lib
</pre>
Copy the library libarpack.a to $ARPACK/lib/libarpack.a.
== SuiteSparse ==
In UFconfig/UFconfig.mk use the following options for CFLAGS and F77FLAGS:
<pre>
CFLAGS=-fPIC -O -DLP64 -D'LONGBLAS=long int' -D'LONG=long int'
F77FLAGS=-fPIC -O -fdefault-integer-8
BLAS=-L$BLAS/lib -lblas -lgfortran -lpthread"
LAPACK=-L$LAPACK/lib -llapack"
</pre>
The library -lpthread for BLAS is only needed is you used the multithreaded <nowiki>GotoBLAS</nowiki> library.
Optionally, Suite Sparse needs metis. In metis-4.0/Makefile.in the following options have to be defined:
<pre>
OPTFLAGS = -O2 -fPIC
</pre>
In the following we assume that you installed all <nowiki>SuiteSparse</nowiki> libraries (and metis) in  $SUITESPARSE/lib and all header files in $SUITESPARSE/include/suitesparse. Since <nowiki>SuiteSparse</nowiki> makefile does not have an install target, you might use:
<pre>
cp {AMD,BTF,CAMD,CCOLAMD,CHOLMOD,COLAMD,CXSparse,UMFPACK}/Lib/lib*a metis-4.0/libmetis.a $SUITESPARSE/lib/
cp {AMD,BTF,CAMD,CCOLAMD,CHOLMOD,COLAMD,CXSparse,UMFPACK}/Include/*h UFconfig/UFconfig.h  $SUITESPARSE/include/suitesparse
</pre>
== QHull ==
<i>Suggestions on how to compile qhull will be most welcome.</i>
== Octave ==
Octave's 64-bit index support is activated with the configure option --enable-64.
<pre>
./configure \
  CPPFLAGS="-I$SUITESPARSE/include -I$QHULL/include " \
  LIBS="-L$SUITESPARSE/lib -lmetis -L$QHULL/lib  -L$ARPACK -larpack -L$LAPACK/lib -L$BLAS/lib -lblas -lgfortran -lm -lpthread" \
  FFLAGS=-fdefault-integer-8 \
  F77=gfortran --enable-64
</pre>
-->


[[Category:Building]]
[[Category:Building]]

Revision as of 16:40, 11 June 2015

Note: the following only applies to systems that have 64-bit pointers (64bit architecture)

By default, the size of a single Octave array cannot exceed 2 GB of memory. Trying to create one will produce the following error: 

octave:1> a = zeros(1024,1024,512);
error: memory exhausted or requested size too large for range of Octave's index type -- trying to return to prompt

You will obtain this error even if your system has enough RAM to create this array (4 GB in the above case).

To use arrays with more than (approximately) 2 billion elements, Octave has to be configured with the option --enable-64. This option is experimental and you are (as always) encouraged to submit bug reports if you find a problem. With this option, Octave will use internally 64-bit integers for array dimensions and indexing. However, all numerical libraries used by Octave will need to use also 64-bit integers for array dimensions and indexing, and in most cases they need to be compiled from source.

For details about how to compile these libraries please read the GNU Octave manual, or alternatively you can use MXE (M Cross Environment) which takes care of the different packages automatically.

Retrieved from "https://wiki.octave.org/wiki/index.php?title=Enable_large_arrays:_Build_octave_such_that_it_can_use_arrays_larger_than_2Gb.&oldid=6415"