Editing Enable large arrays: Build octave such that it can use arrays larger than 2Gb.

Jump to navigation Jump to search
Warning: You are not logged in. Your IP address will be publicly visible if you make any edits. If you log in or create an account, your edits will be attributed to your username, along with other benefits.

The edit can be undone. Please check the comparison below to verify that this is what you want to do, and then publish the changes below to finish undoing the edit.

Latest revision Your text
Line 13: Line 13:
* glpk
* glpk
* Qhull
* Qhull
Useful information and projects are listed below in the [[#See also|See also]] section.
To determine the integer size of the BLAS library used by Octave, the following code can be executed:
<syntaxhighlight lang="Octave">
clear all;
N = 2^31;
## The following line requires about 8 GB of RAM!
a = b = ones (N, 1, "single");
c = a' * b
</syntaxhighlight>
If the BLAS library uses '''32-bit integers''', an error will be thrown:
error: integer dimension or index out of range for Fortran INTEGER type
Otherwise, if the BLAS library uses '''64-bit integers''', the result is:
c = 2^31 = 2147483648
Note that the test case above usually requires twice the memory, if <code>a</code> and <code>b</code> are not assigned by <code>a = b = ...</code>.
Note further, that the data type "single" has a precision of about 23 binary bits.
In this particular example no rounding errors occur.


===Versions prior to Octave 4.4===
===Versions prior to Octave 4.4===
On previous versions of Octave, the default is that the size of a single Octave array cannot have more than approximately 2^31 elements, even on systems that use 64-bit pointers. This is because array indices were limited to 32-bit signed integers by default.  Trying to create one will produce the following error:
On previous versions of Octave, the default is that the size of a single Octave array cannot exceed 2 GB of memory. Trying to create one will produce the following error:


<pre>
<pre>
Line 46: Line 22:
</pre>
</pre>


You will obtain this error even if your system has enough RAM to create this array (3 GB in the above case).
You will obtain this error even if your system has enough RAM to create this array (4 GB in the above case).


To use arrays with more than (approximately) <math>2^{31}</math> elements, Octave has to be configured with the option <code>--enable-64</code>. This option is experimental and you are (as always) encouraged to submit bug reports if you find a problem.  
To use arrays with more than (approximately) <math>2^{31}</math> elements, Octave has to be configured with the option <code>--enable-64</code>. This option is experimental and you are (as always) encouraged to submit bug reports if you find a problem.  
With this option, Octave will use internally 64-bit integers for array dimensions and indexing. However, '''all numerical libraries''' used by Octave will need to use also 64-bit integers for array dimensions and indexing, and in most cases they need to be compiled from source.
With this option, Octave will use internally 64-bit integers for array dimensions and indexing. However, '''all numerical libraries''' used by Octave will need to use also 64-bit integers for array dimensions and indexing, and in most cases they need to be compiled from source.


===See also===
For details about how to compile these libraries please read the [http://www.gnu.org/software/octave/doc/interpreter/Compiling-Octave-with-64_002dbit-Indexing.html#Compiling-Octave-with-64_002dbit-Indexing GNU Octave manual], or alternatively you can use [http://wiki.octave.org/MXE MXE] (M Cross Environment) which takes care of the different packages automatically (especially the configure flags <code>--enable-64</code> and <code>--enable-fortran-int64</code>).
 
* [https://octave.org/doc/interpreter/Compiling-Octave-with-64_002dbit-Indexing.html GNU Octave manual] -- Details on how to compile some of Octave's library dependencies for 64-bit indices.
* [[MXE]] (M Cross Environment) which takes care to compile Octave's library dependencies for 64-bit indices.
 
Two more lightweight solutions compared to [[MXE]] to compile Octave's library dependencies for 64-bit indices.
* https://gitlab.com/mtmiller/octave-blas64-builder
* https://github.com/octave-de/GNU-Octave-enable-64


[[Category:Building]]
[[Category:Building]]
Please note that all contributions to Octave may be edited, altered, or removed by other contributors. If you do not want your writing to be edited mercilessly, then do not submit it here.
You are also promising us that you wrote this yourself, or copied it from a public domain or similar free resource (see Octave:Copyrights for details). Do not submit copyrighted work without permission!

To edit this page, please answer the question that appears below (more info):

Cancel Editing help (opens in new window)

Template used on this page:

Retrieved from "https://wiki.octave.org/Enable_large_arrays:_Build_octave_such_that_it_can_use_arrays_larger_than_2Gb."