TISEAN package:Procedure: Difference between revisions

From Octave
Jump to navigation Jump to search
(→‎Table of functions: updated polynom status)
(→‎External links: replaced dead link with official repo)
 
(4 intermediate revisions by 3 users not shown)
Line 1: Line 1:
== Porting TISEAN ==
== Porting TISEAN ==
=== Goal of the project ===
The goal of this project is not to port the entire [[TISEAN package]] to octave. That would be a desired outcome though it might not be feasible within the time constraints. The goal of this project is to give the TISEAN package a solid start and to port as many functions as possible to create a solid foundation for the future.
=== General division ===
As the TISEAN package consists of 74 programs I have divided the first part into three sub-parts:
# FORTRAN ones that can be re-implemented easily in m-files (a good example of such a program is 'henon') -- there are 5 programs in this class
# C programs which also need to be linked to oct files (an example is 'ghkss') -- there are 41 programs in this class
# FORTRAN ones that need to be linked to oct files (an example of such a program is 'project') -- there are 28 programs in this class
They are ordered so that according to my estimates the difficulty rises with the number. This is because typecasting and implicit typing (which is included in most of the FORTRAN files in the TISEAN library) can be problematic sometimes.
This number can be brought down significantly. This is because some programs are deprecated, others are just C/FORTRAN copies of each other, others are not important in GNU Octave (such as 'compare' and choose'). After taking the factors above into consideration the number of functions that need to be ported drops to 49. I have prepared a detailed discussion of all of those functions [[User:Josiah425:TISEAN_Package:Table_of_functions| here]]. This number will further drop once certain programs are confirmed to have similar programs in GNU Octave or some packages in Octave Forge.
Apart from the qualitative division I propose a work oriented division. In it each subpart can be tackled separately and create an entity in-and-of-itself. I chose to work along the lines of the articles about implementations of nonlinear timeseries included in the documentation. This article discusses various algorithms and what certain programs mean. It can be found [http://www.mpipks-dresden.mpg.de/~tisean/Tisean_3.0.1/docs/chaospaper/TiseanHTML.html here]. I will discuss in which order I would like to port various topics from this article and where my work currently stands.
==== Nonlinear noise reduction ====
This is the first topic I chose. It is because it contains programs from all three categories. It is also relatively small -- it contains 3 programs: project, lazy, ghkss. I have chosen to further implement addnoise and henon, to demonstrate how project and ghkss work. Thus this topic contains programs from each category:
* Re-implementable in mfile (henon)
* Linkable to FORTRAN (project, addnoise, lazy)
* Linkable to c (ghkss)
I have already started working on this stage. My progress can be viewed at [https://bitbucket.org/josiah425/tisean https://bitbucket.org/josiah425/tisean]. So far I have implemented addnoise, project and re-implemented henon as an mfile. As most work on this topic has been completed I estimate that finishing it up will take around 2 days -- throughout my outline I estimate about 1 day per program (that includes documentation and testing).
===== Phase space representation =====
This is the next topic that needs to be implemented. This is because it contains programs (especially 'delay') that are used to visualize data. Whenever an example is given in the package the resulting data is routed through 'delay' before it is plotted. Apart from delay it also contains other functions that can divided into the following categories:
* Linkable to c (delay, corr, mutual, false_nearest, pca)
There are two more programs in this section of the article they are: 'autocorr' and 'pc', both implemented in FORTRAN. There is no need to port them as according to the documentation ([http://www.mpipks-dresden.mpg.de/~tisean/Tisean_3.0.1/docs/contents.html here]) they are redundant with other functions. Further more, it is likely 'corr' does not need to be implemented, because 'xcorr' in signal package seems to have similar functionality. This has not been confirmed yet, once that occurs, a definite answer can be given.
Assuming around a day for each function (with testing and documenting the usage) I assume this stage will take a little under a week.
===== Nonlinear prediction =====
This seems like a reasonable next step. It consists of the following programs:
* Linkable to FORTRAN (predict, upo)
* Linkable to C (lzo-test, lzo-gm, lzo-run, lfo-ar, lfo-gm, lfo-run, rbf, polynom, xzero)
Again assuming around a day for each program (with testing, documenting usage and writing examples) I assume this stage will take about two weeks.
===== Lyapunov exponents =====
This stage will include:
* Linkable to C (lyap_r, lyap_k, lyap_spec)
It will take about 2-3 days to complete.
===== Dimensions and entropies =====
This topic is next on the list. Programs it include are as follows:
* Linkable to FORTRAN (c2, c2t, c2d, c2g, c1)
* Linkable to C (d2, boxcount)
This part of the article also mentions 'c2naive' which is implemented in FORTRAN, but it is also described as redundant by the documentation ([http://www.mpipks-dresden.mpg.de/~tisean/Tisean_3.0.1/docs/contents.html here])
This stage should take little over a week. I expect this stage and the previous one to take about two weeks.
===== Testing for nonlinearity =====
This is the last topic I intend to tackle. The following programs are included here:
* Linkable to FORTAN (surrogates, randomize , timerev)
This stage should take me about 3 days to complete.
===== Tutorial =====
I also plan to port all of the functions needed for the four exercises described in the 'Tutorial' section of the documentation. The programs that need to be ported additionally are as follows:
* Linkable to FORTRAN (stp)
* Linkable to C (ar-model, d2, poincare, recurr, nstat_z)
The programs: 'spectrum', 'historgram', 'extrema', 'corr' need to have a confirmed equivalent function in GNU Octave.
This stage should take me about a week.
==== Notes on time estimates ====
Totaling up the above estimates it should take me 6-7 weeks to complete my task as outlined above.
My estimates might be high, but I believe it is more important to complete the task thoroughly than to port more programs haphazardly.
=== Details of work on each program ===
* FORTRAN linking
For each FORTRAN program that I intend to link to a oct-file I intend to:
# Strip the program of its input validation and transform it into a subroutine
# Create a .cc program (compiled into an oct-file) that will launch the stripped FORTRAN subroutine; this .cc program will also not contain input validation, it will be for internal use only
# Create a m-file that will perform input validation and launch the .cc and contain usage documentation
* C linking
I intend to do here something similar to the FORTRAN programs, although, it might be better to not create any extra m-files and incorporate the program's existing input validation into the .cc file. This might be a desired course of action. I will make a decision once I complete one such linking program.
* Reimplementing in mfile
This is quite straightforward, although it is important not to make a mistake while taking this approach.
== Procedure ==


This section describes the process employed during porting TISEAN package into an Octave Package. This project was started as part of the Google Summer of Code 2015.
This section describes the process employed during porting TISEAN package into an Octave Package. This project was started as part of the Google Summer of Code 2015.
Line 140: Line 210:
|-
|-
|recurr || Creates a recurrence plot || None in GNU Octave || C || wrapped in C++/mfile/octfile code ||
|recurr || Creates a recurrence plot || None in GNU Octave || C || wrapped in C++/mfile/octfile code ||
|-
|-style="background: Yellow"
|resample || Resamples data || There is 'resample' in Octave, but I believe it does something else || C || wrapped in C++/mfile/octfile code ||
|resample || Resamples data || interp1 is similar but cannot specify the polynomial order except for some given || C || wrapped in C++/mfile/octfile code ||
|-
|-style="background: LightGreen"
|rescale || Rescale data set || This should be in Octave, cannot find... || C || wrapped in C++/mfile/octfile code ||
|rescale || Rescale data set || This is just multiplication by scalars || C || wrapped in C++/mfile/octfile code ||Not Needed
|-style="background: LightGreen"
|-style="background: LightGreen"
|rms || Rescale data set and get mean, variance and data interval || This should be in Octave, cannot find...  || FORTRNAN || wrapped in C++/mfile/octfile code || Not Needed
|rms || Rescale data set and get mean, variance and data interval || This should be in Octave, cannot find...  || FORTRNAN || wrapped in C++/mfile/octfile code || Not Needed
Line 177: Line 247:




[[Category:Octave-Forge]]
[[Category:Octave Forge]]


== External links ==
== External links ==
* [https://bitbucket.org/josiah425/tisean Bitbucket repository ] where the porting is taking place.
* [https://sourceforge.net/p/octave/tisean/ci/default/tree/ Repository] where the porting is taking place.

Latest revision as of 14:39, 23 January 2021

Porting TISEAN[edit]

Goal of the project[edit]

The goal of this project is not to port the entire TISEAN package to octave. That would be a desired outcome though it might not be feasible within the time constraints. The goal of this project is to give the TISEAN package a solid start and to port as many functions as possible to create a solid foundation for the future.

General division[edit]

As the TISEAN package consists of 74 programs I have divided the first part into three sub-parts:

  1. FORTRAN ones that can be re-implemented easily in m-files (a good example of such a program is 'henon') -- there are 5 programs in this class
  2. C programs which also need to be linked to oct files (an example is 'ghkss') -- there are 41 programs in this class
  3. FORTRAN ones that need to be linked to oct files (an example of such a program is 'project') -- there are 28 programs in this class

They are ordered so that according to my estimates the difficulty rises with the number. This is because typecasting and implicit typing (which is included in most of the FORTRAN files in the TISEAN library) can be problematic sometimes.

This number can be brought down significantly. This is because some programs are deprecated, others are just C/FORTRAN copies of each other, others are not important in GNU Octave (such as 'compare' and choose'). After taking the factors above into consideration the number of functions that need to be ported drops to 49. I have prepared a detailed discussion of all of those functions here. This number will further drop once certain programs are confirmed to have similar programs in GNU Octave or some packages in Octave Forge.

Apart from the qualitative division I propose a work oriented division. In it each subpart can be tackled separately and create an entity in-and-of-itself. I chose to work along the lines of the articles about implementations of nonlinear timeseries included in the documentation. This article discusses various algorithms and what certain programs mean. It can be found here. I will discuss in which order I would like to port various topics from this article and where my work currently stands.

Nonlinear noise reduction[edit]

This is the first topic I chose. It is because it contains programs from all three categories. It is also relatively small -- it contains 3 programs: project, lazy, ghkss. I have chosen to further implement addnoise and henon, to demonstrate how project and ghkss work. Thus this topic contains programs from each category:

  • Re-implementable in mfile (henon)
  • Linkable to FORTRAN (project, addnoise, lazy)
  • Linkable to c (ghkss)

I have already started working on this stage. My progress can be viewed at https://bitbucket.org/josiah425/tisean. So far I have implemented addnoise, project and re-implemented henon as an mfile. As most work on this topic has been completed I estimate that finishing it up will take around 2 days -- throughout my outline I estimate about 1 day per program (that includes documentation and testing).

Phase space representation[edit]

This is the next topic that needs to be implemented. This is because it contains programs (especially 'delay') that are used to visualize data. Whenever an example is given in the package the resulting data is routed through 'delay' before it is plotted. Apart from delay it also contains other functions that can divided into the following categories:

  • Linkable to c (delay, corr, mutual, false_nearest, pca)

There are two more programs in this section of the article they are: 'autocorr' and 'pc', both implemented in FORTRAN. There is no need to port them as according to the documentation (here) they are redundant with other functions. Further more, it is likely 'corr' does not need to be implemented, because 'xcorr' in signal package seems to have similar functionality. This has not been confirmed yet, once that occurs, a definite answer can be given. Assuming around a day for each function (with testing and documenting the usage) I assume this stage will take a little under a week.

Nonlinear prediction[edit]

This seems like a reasonable next step. It consists of the following programs:

  • Linkable to FORTRAN (predict, upo)
  • Linkable to C (lzo-test, lzo-gm, lzo-run, lfo-ar, lfo-gm, lfo-run, rbf, polynom, xzero)

Again assuming around a day for each program (with testing, documenting usage and writing examples) I assume this stage will take about two weeks.

Lyapunov exponents[edit]

This stage will include:

  • Linkable to C (lyap_r, lyap_k, lyap_spec)

It will take about 2-3 days to complete.

Dimensions and entropies[edit]

This topic is next on the list. Programs it include are as follows:

  • Linkable to FORTRAN (c2, c2t, c2d, c2g, c1)
  • Linkable to C (d2, boxcount)

This part of the article also mentions 'c2naive' which is implemented in FORTRAN, but it is also described as redundant by the documentation (here) This stage should take little over a week. I expect this stage and the previous one to take about two weeks.

Testing for nonlinearity[edit]

This is the last topic I intend to tackle. The following programs are included here:

  • Linkable to FORTAN (surrogates, randomize , timerev)

This stage should take me about 3 days to complete.

Tutorial[edit]

I also plan to port all of the functions needed for the four exercises described in the 'Tutorial' section of the documentation. The programs that need to be ported additionally are as follows:

  • Linkable to FORTRAN (stp)
  • Linkable to C (ar-model, d2, poincare, recurr, nstat_z)

The programs: 'spectrum', 'historgram', 'extrema', 'corr' need to have a confirmed equivalent function in GNU Octave. This stage should take me about a week.

Notes on time estimates[edit]

Totaling up the above estimates it should take me 6-7 weeks to complete my task as outlined above.

My estimates might be high, but I believe it is more important to complete the task thoroughly than to port more programs haphazardly.

Details of work on each program[edit]

  • FORTRAN linking

For each FORTRAN program that I intend to link to a oct-file I intend to:

  1. Strip the program of its input validation and transform it into a subroutine
  2. Create a .cc program (compiled into an oct-file) that will launch the stripped FORTRAN subroutine; this .cc program will also not contain input validation, it will be for internal use only
  3. Create a m-file that will perform input validation and launch the .cc and contain usage documentation
  • C linking

I intend to do here something similar to the FORTRAN programs, although, it might be better to not create any extra m-files and incorporate the program's existing input validation into the .cc file. This might be a desired course of action. I will make a decision once I complete one such linking program.

  • Reimplementing in mfile

This is quite straightforward, although it is important not to make a mistake while taking this approach.


Procedure[edit]

This section describes the process employed during porting TISEAN package into an Octave Package. This project was started as part of the Google Summer of Code 2015.

To aid in understanding the task there are some charts.

The first chart depicts what to do with each function in the function table. I mainly focuses on those functions that might already implemented in Octave.

Work flow TISEAN.png

The chart below depicts how to decide which type of port should be utilized.

Porting Programs TISEAN.png

Both of those charts can be combined into a large one that shows all of the work together.

Flow Together TISEAN.png

Table of functions[edit]

In reference to the TISEAN library alphabetical order of programs which is located here.

The choice whether a program exist in Octave is based only on comparing package/octave documentation with the TISEAN documentation. As of now I have not compared any code, nor checked if any sample data gives the same results from both functions (the octave ones and the TISEAN ones).

The legend to understand the table is as follows:

Green means the function is ported, tested or is not needed
Red means the function was thought to have been similar but isn't
Yellow means that the correlation between the TISEAN and Octave function needs to be verified
No color means that nothing about this function has been determined
Program Name Program Description Corresponding Octave Function Lanuguage of Origin How to port Status
arima-model Fit and possibly iterate an ARIMA model generalizes TSA arma functions C wrapped in C++/mfile/octfile code
ar-model Fit and possibly iterate an Autoregessive model 'aar' in TSA; see also: aarmam, adim, amarma, mvaar from TSA C wrapped in C++/mfile/octfile code
ar-run Iterate an Autoregessive model Same as above FORTRAN wrapped in C++/mfile/octfile code
av-d2 Simply smooth output of d2 Can be implemented with filter in core C
boxcount Renyi Entopies of Qth order None in GNU Octave (maybe in info-theory but it is not worth the pain) C wrapped in C++/mfile/octfile code
c1 Fixed mass estimation of D1 None in GNU Octave FORTRAN wrapped in C++/mfile/octfile code
c2d Get local slopes from correlation integral None in GNU Octave FORTRAN wrapped in C++/mfile/octfile code
c2g Gaussian kernel of C2 None in GNU Octave FORTRAN wrapped in C++/mfile/octfile code
c2t Takens estimator of D2 None in GNU Octave FORTRAN wrapped in C++/mfile/octfile code
choose Choose rows and/or columns from a data file Does not need to be ported FORTRAN ----- Not Needed
compare Compares two data sets Does not need to be ported FORTRAN ------ Not Needed
corr, autocorr Autocorrelation function xcorr in signal corr -C, autocorr (faster according to documentation) - FORTRAN ------- Different usage, same result
d2 Correlation dimension d2 None in GNU Octave C wrapped in C++/mfile/octfile code
delay Creates delay embedding None in GNU Octave (easy to implement in Octave but not worth the effort) C wrapped in C++/mfile/octfile code Ported&Tested
endtoend Determine end-to-end mismatch None in GNU Octave FORTRAN wrapped in C++/mfile/octfile code Ported&Tested
events Interval/event conversion None in GNU Octave FORTRAN To be reimplemented as m-file
extrema Determine the extrema of a time series findpeaks in signal is *not* the same C
false_nearest The false nearest neighbor algorithm None in GNU Octave C wrapped in C++/mfile/octfile code Ported&Tested
ghkss Nonlinear noise reduction None in GNU Octave C wrapped in C++/mfile/octfile code Ported&Tested
henon Create a Hénon time series None in GNU Octave FORTRAN To be reimplemented as m-file Ported&Checked
histogram Creates histograms hist in core C Different usage, same result
ikeda Create an Ikeda time series None in GNU Octave FORTRAN To be reimplemented as mfile Ported&Tested
intervals Event/intervcal conversion Might exist under different name FORTRAN To be reimplemented as mfile
lazy Simple nonlinear noise reduction None in GNU Octave FORTRAN wrapped in C++/mfile/octfile code Ported&Tested
lfo-ar Locally first order model vs. global AR model (old ll-ar) None in GNU Octave C wrapped in C++/mfile/octfile code
lfo-run Iterate a locally first order model (old nstep) None in GNU Octave C wrapped in C++/mfile/octfile code
lfo-test Test a locally first order model (old onestep) None in GNU Octave C wrapped in C++/mfile/octfile code
lorenz Create a Lorenz time series Can be implemented in GNU Octave using lsode or odepkg FORTRAN mfile
low121 Time domain low pass filter There are lowpass filters in Octave: buttap, cheb1ap, cheb2ap, ellipap, sftrans, but I don't think they perform this task C Reimplement as mfile
lyap_k Maximal Lyapunov exponent with the Kantz algorithm None in GNU Octave C wrapped in C++/mfile/octfile code
lyap_r Maximal Lyapunov exponent with the Rosenstein algorithm None in GNU Octave C wrapped in C++/mfile/octfile code
lyap_spec Full spectrum of Lyapunov exponents None in GNU Octave C wrapped in C++/mfile/octfile code
lzo-gm Locally zeroth order model vs. global mean None in GNU Octave C wrapped in C++/mfile/octfile code Ported&Tested
lzo-run Iterate a locally zeroth order model None in GNU Octave C wrapped in C++/mfile/octfile code Ported&Tested
lzo-test Test a locally zeroth order model (old zeroth) None in GNU Octave C wrapped in C++/mfile/octfile code Ported&Tested
makenoise Produce noise Rand exists Should be implemented as mfile using Octave rand functions wrapped in C++/mfile/octfile code
mem_spec Power spectrum using the maximum entropy principle None in GNU Octave C wrapped in C++/mfile/octfile code
mutual Estimate the mutual information None in GNU Octave C wrapped in C++/mfile/octfile code Ported&Tested
notch Notch filter pei_tseng_notch, needs to be verified FORTRAN wrapped in C++/mfile/octfile code
nstat_z Nonstationarity testing via cross-prediction None in GNU Octave C wrapped in C++/mfile/octfile code
pca, pc Principle component analysis 'pcacov' if likely the equivalent; pricomp in statistics pca - C, pc - FORTRAN wrapped in C++/mfile/octfile code Ported&Tested
poincare Create Poincaré sections None in GNU Octave C wrapped in C++/mfile/octfile code Ported&Tested
polyback Fit a polynomial model (backward elimination) polyfit, detrend, wpolyfit are not similar C wrapped in C++ Porting needed
polynom Fit a polynomial model same as above C Ported&Tested
polynomp Fit a polynomial model (reads terms to fit from file) same as above C
polypar Creates parameter file for polynomp same as above C
predict Forecast discriminating statistics for surrogates None in GNU Octave FORTRAN wrapped in C++/mfile/octfile code Same as lzo_test; Not Needed
randomize General constraint randomization (surrogates) There are random function, but I don't think this one exists FORTRAN wrapped in C++/mfile/octfile code
randomize_spikeauto_exp_random Surrogate data preserving event time autocorrelations Same as above FORTRAN wrapped in C++/mfile/octfile code
randomize_spikespec_exp_event Surrogate data preserving event time power spectrum Same as above FORTRAN wrapped in C++/mfile/octfile code
rbf Radial basis functions fit None in GNU Octave C wrapped in C++/mfile/octfile code
recurr Creates a recurrence plot None in GNU Octave C wrapped in C++/mfile/octfile code
resample Resamples data interp1 is similar but cannot specify the polynomial order except for some given C wrapped in C++/mfile/octfile code
rescale Rescale data set This is just multiplication by scalars C wrapped in C++/mfile/octfile code Not Needed
rms Rescale data set and get mean, variance and data interval This should be in Octave, cannot find... FORTRNAN wrapped in C++/mfile/octfile code Not Needed
sav_gol Savitzky-Golay filter sgolayfilt in signal C
spectrum Power spectrum using FFT abs (fft (;)) FORTRAN Rewrite as m-file Ported&Tested
spikeauto Autocorrelation function of event times similar to above FORTRAN
spikespec Power spectrum of event times ar_psd, cpsd is the closest but I do not think they are the same FORTRAN
stp Creates a space-time separation plot None in GNU Octave FORTRAN wrapped in C++/mfile/octfile code
surrogates Creates surrogate data None in GNU Octave FORTRAN wrapped in C++/mfile/octfile code Ported&Tested
timerev Time reversal discrimating statistics for surrogates None in GNU Octave FORTRAN To be reimplemented as mfile Ported&Tested
upo Finds unstable periodic orbits and estimates their stability None in GNU Octave FORTRAN wrapped in C++/mfile/octfile code Ported&Tested
upoembed Takes the output of upo and create data files out of it None in GNU Octave FORTAN wrapped in C++/mfile/octfile code Ported&Tested
wiener1, wiener2 Wiener filter Wiener process exists, might be similar FORTRAN wrapped in C++/mfile/octfile code
xc2 Cross-correlation integral xcorr2 - Needs to be verified that works the same way FORTRAN wrapped in C++/mfile/octfile code
xcor Cross-correlations xcorr - Needs to be verified that works same way C wrapped in C++/mfile/octfile code
xrecur Cross-recurrence Plot xcorr, or xcorr2 or another function might cover this C wrapped in C++/mfile/octfile code
xzero Locally zeroth order cross-prediction None in GNU Octave C wrapped in C++/mfile/octfile code

External links[edit]