1,847
edits
NDpar package: Difference between revisions
Jump to navigation
Jump to search
Add Category:External Packages.
(Created page with "ndpar_[cell,array]fun are the par[cell,array]fun from package parallel 2.2.0, extended in order to handle N-dimensionnal arrays as input and output. Hope it will eventually ma...") |
(Add Category:External Packages.) |
||
| (8 intermediate revisions by 2 users not shown) | |||
| Line 1: | Line 1: | ||
ndpar_[cell,array]fun are the par[cell,array]fun from package parallel 2.2.0, extended in order to handle N-dimensionnal arrays as input and output. Hope it will eventually make it to the parallel package, once its stability is proved. | == Introduction == | ||
See also package | ndpar_[cell,array]fun are the par[cell,array]fun from package parallel 2.2.0, extended in order to handle N-dimensionnal arrays as input and output. | ||
Hope it will eventually make it to the parallel package, once its stability is proved. | |||
See also the [[Parallel package]] maintained by Olaf Till | |||
There is an overhead for parallelization. Hence this package is not suited for very fast functions. | |||
== Installation == | == Installation == | ||
| Line 10: | Line 14: | ||
== Usage == | == Usage == | ||
To load the package, {{codeline|pkg load ndpar}} | To load the package, {{codeline|pkg load ndpar}} | ||
=== setting CatDimensions and IdxDimensions - vectorized === | |||
Suppose we have a function that works along different dimensions of its input arguments, and returns arrays of different shapes like | |||
<code> | |||
<pre> | |||
function [x, y] = f(u, v) | |||
x = u + v.'; | |||
y = x.'; | |||
endfunction | |||
</pre> | |||
</code> | |||
This function is used only to illustrate the syntax, not the usefulness of the extension. | |||
Admittedly such a simple case could be handled easily either by changing the function or making a small wrapper to it. But it becomes really interesting when we are dealing with arrays of many and different dimensions (not shown here yet). | |||
Applying this function to arrays u and v can be straightforwardly parallelized : | |||
{{code|setting CatDimensions and IdxDimensions - vectorized| | |||
<pre> | |||
u = [1:10; 2:11]; | |||
v = u.'; | |||
[x, y] = ndpar_arrayfun(2, @f, u, v, "Vectorized", true, "ChunksPerProc", 2, "VerboseLevel", 1, "CatDimensions", [2 1],"IdxDimensions", [2 1] ); | |||
</pre> | |||
}} | |||
Here is the meaning of the options | |||
* {{codeline|"IdxDimensions", [2 1]}} The parallelization (or slicing, or indexing) should be done along the 2nd dimension of {{codeline|u}} and 1st dimension of {{codeline|v}}. A value of 0 means no indexing (no slicing), so the argument would be passed "as is". | |||
* {{codeline|"CatDimensions", [2 1]}} The outputs from each slice should be concatenated along the 2nd dimension of the first output and 1st dimension of the second output | |||
* {{codeline|"Vectorized", true}} Use only if the function is vectorized along the "indexing" dimensions. | |||
* {{codeline|"ChunksPerProc", 2}} It means that each process should make 2 chunks (2 calls to {{codeline|f}} with {{codeline|"Vectorized", true}}). Increase this number to minimize memory usage for instance. Increasing this number is also useful if function executions can have very different durations. If a process is finished, it can take over jobs from another process that is still busy. | |||
=== Output in cell arrays === | |||
The following sample code was an answer to [http://stackoverflow.com/questions/27422219/for-every-row-reshape-and-calculate-eigenvectors-in-a-vectorized-way this question]. The goal was to diagonalize matrices. For this example, 2x2 matrices contained as rows of a 2d array (each row of the array being a flattened 2x2 matrix). Note that in this case the function is too fast, so the performance penalty is too high to gain anything. | |||
{{code|diagonalize NxN matrices contained in an array| | |||
<pre> | |||
A = [0.6060168 0.8340029 0.0064574 0.7133187; | |||
0.6325375 0.0919912 0.5692567 0.7432627; | |||
0.8292699 0.5136958 0.4171895 0.2530783; | |||
0.7966113 0.1975865 0.6687064 0.3226548; | |||
0.0163615 0.2123476 0.9868179 0.1478827]; | |||
N = 2; | |||
[eigenvectors, eigenvalues] = ndpar_arrayfun(nproc, | |||
@(row) eig(reshape(row, N, N)), | |||
A, "IdxDimensions", 1, "UniformOutput", false) | |||
</pre> | |||
}} | |||
With {{codeline|"UniformOutput", false}}, the outputs are contained in cell arrays (one cell per slice). In the sample above, both {{codeline|eigenvectors}} and {{codeline|eigenvalues}} are {{codeline|1x5}} cell arrays. | |||
== Issues == | |||
Please report any issue using [https://sourceforge.net/p/ndpar/tickets/ tickets]. | |||
[[Category:External Packages]] | |||