User talk:Genuinelucifer
GSoC 2016 Symbolic Project Improvements
References
Outline
Currently, Symbolic and PyTave each have their own mechanisms to convert objects into Octave types. As first step, we would put @sym creation login into PyTave and remove all the conversion login from Symbolic. Then we will target having the same via some m-file callback to have a more generalised conversion for objects.
We'll need PyTave to build successfully on all platforms (including windows) before we can ship it as a release. In ideal case, this is the target to be achieved as mid term goal.
Once we have full support for all platforms, we could have PyTave as the only mechanism to communicate with python. Also, we will improve the functionalities of PyTave by adding support for handling python objects.
After all this is acheived and we have all tests working fine on all platforms. We will have some @sym methods recoded to use the full power of PyTave. We can then head to "blue skies" and work on some octave core stuff like correcting display of unicode on windows and maybe trying to get PyTave merged into core Octave.
If time is still left, we can work on improvements to upstream Octave:
Midterm Goals
1a) Octave, Symbolic, and PyTave dev versions installed.
1b) Some basic communication working between PyTave and Symbolic. This might use some existing implementations in a non-optimal way (e.g., its ok if Symbolic returns objects as xml strings).
1c) Most of Symbolic's tests and doctests continue passing, although some failures ok at this point.
1d) The above works on at least one OS (probably GNU/Linux).
2a) PyTave converts various Python objects into appropriate Octave objects. PyTave needs to be extended to return proper @pyobj when it cannot convert the object. Also, symbolic must be able to convert such objects to @sym objects by calling proper python functions via PyTave (if they are indeed @sym). That is, bypass the current generating of xml strings.
2b) Improve the BIST test set coverage of both PyTave and Symbolic for any new features added.
2c) Improve doctest coverage of PyTave.
Stretch Goals
3) Improve on 2a) with some TBD mechanism: perhaps a pure m-file callback feature in the PyTave export code.
4) The above works on both GNU/Linux and MS Windows.
5) Objects passed from Symbolic back to Python should use PyTave mechanisms.
Time-line
| 30th May | Work on adding the sym conversion to PyTave and cleaning up the conversion mechanism in Symbolic. |
| 15th June | Improve tests and doctests. Work on building PyTave and testing on Windows. No more crufty XML! |
| 27th June | Try to get working PyTave with Symbolic on Windows (if needed, use cygwin) [Mid Term Evaluations] |
| 5th July | Get a successfully working Symbolic with PyTave (all the tests and features working). Continue work on Goal 3. |
| 20th July | Finalize implementation for Goal 3. |
| 31st July | Work on improvements to PyTave such as adding support for Python objects from within Octave (printing, calling methods, saving etc...). |
| 10th August | Recode @sym methods as required to take benefit of PyTave. Also, add some other methods into Symbolic from #215 |
| 15th August | "Blue-skies" stuff. Try to fix the unicode utf8 rendering on Windows. Explore possibility of incorporating PyTave into Octave. [gsoc final code submission begins] |
| 23rd August | Finish all the code, test on buildbots and Windows. Submit to Google. [Final deadline for code submission] |
| Afterwards | Complete any goals which are left. Continue contributing to Octave... |
Building Pytave on windows
Windows build for pytave was tried but we could not get it to work. Tatsuro was actually the one who did most of the work and I followed with his help. Here are the things that were tried with details:
OS used: Windows 10
Using Cygwin
At first I tried to build pytave using cygwin which is a linux environment for windows. I had used cygwin previously, so I tried the build using cygwin.
After installing apt-cyg in it, I downloaded many other packages like autotools, autoconf and others to make it suitable for the build. My octave was installed in e:/octave-4.0.0/. When I ran configure of pytave with ./configure OCTAVE_CONFIG=e:/octave-4.0.0/bin/octave-config.exe then it showed that "octave development packages" were not found. Also tried to use cygwin style with "cygdrive/e/octave-4.0.0/bin/octave-config.exe" but the same error occurred.
So I installed "octave-devel" package using "apt-cyg". It took more than 3 hours and installed a lot of packages into cygwin. It also installed octave in /bin/octave of cygwin. Now, I tried using the octave-config file in /bin of cygwin but I got the same error. I also tried using the octave-config in the octave installation directory on windows but the error persisted.
Using Msys2
Tatsuro told to use msys2 for the job. He also suggested to use octave-4.0.2 as octave-4.0.0 was buggy on windows. Later on he provided development version of octave that he built using octave itself (along with msys2). I started using that development version (4.1.0+) of octave. He also suggested the following steps after installing msys2 for the setup:
- Install Msys2 and update core and basics packages
- Install base-devel with pacman -S --needed base-devel msys/dos2unix
- Install mercurial with pacman -S mercurial
At this point I had many questions. There were mingw64 toolchains and also the non-mingw version of those toolchains on Msys2. Moreover, for any program such as Python, we can call either /c/Python27/python.exe from windows of simply /bin/python which will run the *nix executable. Also, I could not find libboost-python for Msys2 which is a dependency for pytave.
Tatsuro told to use the mingw64 toolchains that come with octave (for consistency reasons). As octave has msys toolchains along with it. And just use the extra MSYS2 toolchains on top of it.
- open octave (gui/cli)
- run system bash
- export path from bash export PATH=$PATH:/c/msys64/usr/bin
But at this point I just went ahead with using MSYS2 shell as when trying out msys shell of octave I found out that many things that I used in msys2, specially pacman, were missing.
Also, it's better to use the windows executables for the build. Moreover there were boost packages for mingw on msys2 which could be installed with pacman -S mingw64/mingw-w64-x86_64-boost for 64 bit version or pacman -S mingw32/mingw-w64-i686-boost for 32 bit version. After installing boost too, the build was failing with errors related to boost libraries. I saw that the *.a and *.dll files had '-mt' appended to their name and didn't work because of that... Once I renamed them to remove the '-mt' then the configure could find the libs and finally the configure steps was successful with just one warning that 'uselocale' was not supported!
Now when I ran make then libtool weirdly escaped all the '\' characters in the paths for libraries found during the configure step. Ex "-IE:\octave_dev\include\octave-4.1.0+ -IE:\octave_dev\include\octave-4.1.0+/octave" was passed as "-IE:octave_devincludeoctave-4.1.0+ -IE:octave_devincludeoctave-4.1.0+/octave". I have no knowledge of how exactly the configure and m4 scripts co-ordinate the tasks and hence it was dificult for me to find the cause for this. I tried to see how to change the default path separator but it didn't work. Tatsuro told to use 'CPPFLAGS' to pass these directories during configure only. This worked and the make step moved past that point.
Now a new error arised, the Makefile was calling 'grep/sed' at some point which showed that the library file have been moved (which was present in /e/octave_dev/lib/octave/4.1.0+/) And then the script tries to use it from /usr/lib/octave/4.1.0+. But the library files were actually present in the previous directory only and not in /usr/lib. So, we passed 'LDFLAGS' along with 'CPPFLAGS' during configure. But the error persisted. Editing the last parte of liboct*.la files to change the 'libdir' didn't work as well. As per Mike's suggestion deleting the liboct*.la files helped and the make step moved even furthur. Now the errors showed some missing libraries (lapack, GraphickMagik, ...) Since all these libraries were present along with octave so we added "/e/octave_dev/lib" to LDFLAGS too. Now the build went even furthur. This time there were some C++11 errors. As per Mike's and Tatsuro's suggestions Octave uses 4.9.3 gcc on windows and gcc changed the library around version 5. Msys2 had GCC 5.4 which caused the errors. So, either pass all the gcc tools in octave_dev/bin to configure on MSYS2 or use octave's shell with MSYS2 on top of it. At this point, I switched to using the bash shell present with octave.