Dataframe package

Dataframe, Data manipulation toolbox similar to R data.frame

At an mature development stage. hg

Maintainer: Pascal Dupuis
Contributors:

Package: dataframe

This package permits to handle complex (both in the sense of complex numbers and high complexity) data as if they were ordinary arrays, except that each column MAY possess a different type. It also complete a fairly complete interface to CSV files, permitting to cope with a number of oddities, like f.i. CSV files starting with a header spread over a few lines. The resulting array tries as far as it can to mimick an array, in such a way that binary operators and usual functions will work as expected.

Meta-information is also handled. Rows and columns may have a name, and this name is searchable. If for whatever reason the ordering of a CSV file changes, searching by column names will return the expected information.

To get a first taste, let's load the test csv file coming with the package:

 >> experiment = dataframe('data_test.csv')
 warning: load: '/home/padupuis/matlab/dataframe/inst/data_test.csv' found by searching load path
 warning: fopen: '/home/padupuis/matlab/dataframe/inst/data_test.csv' found by searching load path
 ans = dataframe with 10 rows and 7 columns
 Src: data_test.csv
 Comment: #notice there is a extra separator
 Comment: # a comment line and an empty one
 Comment: # the next lines use \r\n \r and \f as linefeed
 Comment: # one empty input field
 _1  DataName   VBIAS   Freq   x_IBIAS_          C       GOUT  OK_
 Nr      char  double double     double     double     double char
  1 DataValue -6.0000 300000 1.6272e-11 7.0215e-13 1.6044e-07    A
  2 DataValue -5.8000 300000 1.5990e-11 6.9607e-13 1.5728e-07    E
  3 DataValue -5.6000 300000 1.3790e-11 6.9048e-13 1.5489e-07    !
  4 DataValue -5.4000 300000 1.4420e-11 6.8517e-13 1.5478e-07    ?
  5 DataValue -5.2000 300000 1.2930e-11 6.7965e-13 1.5189e-07    C
  6 DataValue -5.0000 300000 1.2610e-11 6.7444e-13 1.4931e-07    B
  7 DataValue -4.8000 300000 1.4390e-11 6.7011e-13 1.4876e-07    A
  8 DataValue -4.6000 300000 1.0890e-11 6.6416e-13 1.4890e-07    3
  9 DataValue -4.4000 300000         NA 6.5859e-13 1.4558e-07    C
 10 DataValue -4.2000 300000 1.0610e-11 6.5355e-13 1.4431e-07    B

Those data were produced while performing a voltage sweep on a sensor, measuring with a impedance bridge the parallel capacitor and conductance at a given frequency.

The first lines contains a few meta-information: name of the source file and a few comments found in the csv file. The purpose is to annotate the results.

Then we have the content. Each column starts with a name, then a type. Next we find the content lines, each of them with an index. Then we find the content; control values (polarization voltage, applied frequency), then measured values: DC current, capacitor, conductance. The last column is categorical: the user introduced some code telling if the result makes senses or not.

Let us now select the control values:

 cv = experiment(1:3, ["Vbias"; "Freq"])
 cv = dataframe with 3 rows and 1 columns
 Src: data_test.csv
 Comment: #notice there is a extra separator
 Comment: # a comment line and an empty one
 Comment: # the next lines use \r\n \r and \f as linefeed
 Comment: # one empty input field
 _1   Freq
 Nr double
  1 300000
  2 300000
  3 300000

The selection occurred on a range for the lines, by names on the column. The search criteria is here a string array. All columns whose name match are returned.

The result is returned as a dataframe. This can be changed:

>> experiment.array(6, "OK_")
ans = B
>> class(ans)
ans = char

When selecting vectors, this transformation in array is automatic. The DC current is contained in elements 31 to 40 (fourth column):

 >> experiment31:40)
ans =
Columns 1 through 9:
  1.6272e-11   1.5990e-11   1.3790e-11   1.4420e-11   1.2930e-11   1.2610e-11   1.4390e-11   1.0890e-11           NA
Column 10:
  1.0610e-11

Note that the access 'experiment("x_IBIAS")' is illegal: does it refer to row or column names ?

Accessing in this pseudo-structure way is valid in the following cases
choosing the output format: array, cell, dataframe (may be abbreviated as 'df')
attribute selection: rownames, colnames, rowcnt, colcnt, rowidx, types, source, header, comment
constructor call: new (no other deferencing may occur
column selection: just provide one valid column name

To be similar to R implementation, constructs such as x.as.array are also allowed.

A simple example:

truc={"Id", "Name", "Type";1, "onestring", "bla"; 2, "somestring", "foobar";}
truc =
{
  [1,1] = Id
  [2,1] =  1
  [3,1] =  2
  [1,2] = Name
  [2,2] = onestring
  [3,2] = somestring
  [1,3] = Type
  [2,3] = bla
  [3,3] = foobar
}
>> tt=dataframe(truc)
tt = dataframe with 2 rows and 3 columns
_1     Id       Name   Type
Nr double       char   char
 1      1  onestring    bla
 2      2 somestring foobar

The first cell line is intended to contain column names; the rest is column content. The type is automatically inferred from the cell content. Now let us select one column by its name:

>> tt(:, 'Name')
ans = dataframe with 2 rows and 1 columns
_1       Name
Nr       char
1  onestring
2 somestring

In this case, a sub-dataframe is returned. Struct-like indexing is also implemented:

>> tt.Id
ans =
  1
  2

When the output is a vector and can be simplified to something simple ... it is.

Dataframe package

Navigation menu

Search