User talk:Genuinelucifer: Difference between revisions

Jump to navigation Jump to search
m
 
(12 intermediate revisions by 2 users not shown)
Line 74: Line 74:
Windows build for pytave was tried but we could not get it to work. Tatsuro was actually the one who did most of the work and I followed with his help.   
Windows build for pytave was tried but we could not get it to work. Tatsuro was actually the one who did most of the work and I followed with his help.   
Here are the things that were tried with details:   
Here are the things that were tried with details:   
'''OS used''': Windows 10   
'''OS used''': Windows 10   


=== Using Cygwin ===
=== Using Cygwin ===


At first I tried to build pytave using [https://www.cygwin.com/ cygwin] which is a linux environment for windows. I had used cygwin previously, so I tried the build using cygwin.   
At first I tried to build pytave using [https://www.cygwin.com/ Cygwin] which provide functionality similar to a Linux distribution on Windows. I had used Cygwin previously, so I tried the build using Cygwin.   
    
    
After installing ''apt-cyg'' in it, I downloaded many other packages like autotools, autoconf and others to make it suitable for the build. My octave was installed in ''e:/octave-4.0.0/''. When I ran ''configure'' of pytave with ''./configure OCTAVE_CONFIG=e:/octave-4.0.0/bin/octave-config.exe'' then it showed that "octave development packages" were not found. Also tried to use cygwin style with "cygdrive/e/octave-4.0.0/bin/octave-config.exe" but the same error occurred.   
After installing ''apt-cyg'' in it, I downloaded many other packages like autotools, autoconf and others to make it suitable for the build. My octave was installed in ''e:/octave-4.0.0/''. When I ran ''configure'' of Pytave with ''./configure OCTAVE_CONFIG=e:/octave-4.0.0/bin/octave-config.exe'' then it showed that "octave development packages" were not found. Also tried to use Cygwin style with "/cygdrive/e/octave-4.0.0/bin/octave-config.exe" but the same error occurred.   


So I installed "octave-devel" package using "apt-cyg". It took more than 3 hours and installed a lot of packages into cygwin. It also installed octave in ''/bin/octave'' of cygwin. Now, I tried using the ''octave-config'' file in ''/bin'' of cygwin but I got the same error. I also tried using the octave-config in the octave installation directory on windows but the error persisted.
So I installed "octave-devel" package using "apt-cyg". It took more than 3 hours and installed a lot of packages into cygwin. It also installed octave in ''/bin/octave'' of cygwin. Now, I tried using the ''octave-config'' file in ''/bin'' of Cygwin but I got the same error. I also tried using the octave-config in the octave installation directory on windows but the error persisted.
 
'''Comments (Tatsuro)''' : Pytave requires octave 4.1.0+ (development version) so that build of 4.1.0+ on Cygwin is the first thing to do.


=== Using Msys2 ===
=== Using Msys2 ===


Tatsuro told to use [https://msys2.github.io/ msys2] for the job. He also suggested to use octave-4.0.2 as octave-4.0.0 was buggy on windows.   
Tatsuro told to use [https://msys2.github.io/ msys2], which aims to provide support to facilitate using the bash shell, autotools, revision control systems and the like for building native Windows applications using MinGW-w64 toolchains, for the job. He also suggested to use octave-4.0.2 as octave-4.0.0 was buggy on windows.   
Later on he provided development version of octave that he built using octave itself (along with msys2). I started using that development version (4.1.0+) of octave. He also suggested the following steps after installing msys2 for the setup:   
Later on he provided development version of octave that he built using octave itself (along with msys2). I started using that development version (4.1.0+) of octave. He also suggested the following steps after installing msys2 for the setup:   
# [https://sourceforge.net/p/msys2/wiki/MSYS2%20installation/ Install Msys2 and update core and basics packages]
# [https://sourceforge.net/p/msys2/wiki/MSYS2%20installation/ Install Msys2 and update core and basics packages]
Line 92: Line 95:
# Install mercurial with ''pacman -S mercurial''   
# Install mercurial with ''pacman -S mercurial''   


At this point I had many questions. There were mingw64 toolchains and also the non-mingw version of those toolchains on Msys2. Moreover, for any program such as Python, we can call either ''/c/Python27/python.exe'' from windows of simply ''/bin/python'' which will run the *nix executable. Also, I could not find ''libboost-python'' for Msys2 which is a dependency for pytave.
At this point I had many questions. There were mingw64 toolchains and also the non-mingw version of those toolchains on Msys2. Moreover, for any program such as Python, we can call either ''/c/Python27/python.exe'' from windows or simply ''/bin/python'' which will run the *nix executable. Also, I could not find ''libboost-python'' for Msys2 which is a dependency for pytave.


Tatsuro told to use the mingw64 toolchains that come with octave (for consistency reasons). As octave has msys toolchains along with it. And just use the extra MSYS2 toolchains on top of it.
Tatsuro told to use the mingw64 toolchains that come with octave (for consistency reasons). As octave has msys toolchains along with it. And just use the extra MSYS2 toolchains on top of it.
Line 104: Line 107:
After installing boost too, the build was failing with errors related to boost libraries. I saw that the *.a and *.dll files  had '-mt' appended to their name and didn't work because of that... Once I renamed them to remove the '-mt' then the configure could find the libs and finally the ''configure'' steps was successful with just one warning that 'uselocale' was not supported!
After installing boost too, the build was failing with errors related to boost libraries. I saw that the *.a and *.dll files  had '-mt' appended to their name and didn't work because of that... Once I renamed them to remove the '-mt' then the configure could find the libs and finally the ''configure'' steps was successful with just one warning that 'uselocale' was not supported!


Now when I ran ''make'' then ''libtool'' weirdly escaped all the '\' characters in the paths for libraries found during the configure step. Ex "-IE:\octave_dev\include\octave-4.1.0+ -IE:\octave_dev\include\octave-4.1.0+/octave" was passed as "-IE:octave_devincludeoctave-4.1.0+ -IE:octave_devincludeoctave-4.1.0+/octave". I have no knowledge of how exactly the configure and m4 scripts co-ordinate the tasks and hence it was dificult for me to find the cause for this. I tried to see how to change the default path separator but it didn't work.
Now when I ran ''make'' then ''libtool'' weirdly escaped all the '\' characters in the paths for libraries found during the configure step. Ex "-IE:\octave_dev\include\octave-4.1.0+ -IE:\octave_dev\include\octave-4.1.0+/octave" was passed as "-IE:octave_devincludeoctave-4.1.0+ -IE:octave_devincludeoctave-4.1.0+/octave". I have no knowledge of how exactly the configure and m4 scripts co-ordinate the tasks and hence it was difficult for me to find the cause for this. I tried to see how to change the default path separator but it didn't work.
Tatsuro told to use 'CPPFLAGS' to pass these directories during configure only. This worked and the ''make'' step moved past that point.
Tatsuro told to use 'CPPFLAGS' to pass these directories during configure only. This worked and the ''make'' step moved past that point.


Now a new error arised, the Makefile was calling 'grep/sed' at some point which showed that the library file have been moved (which was present in /e/octave_dev/lib/octave/4.1.0+/) And then the script tries to use it from /usr/lib/octave/4.1.0+. But the library files were actually present in the previous directory only and not in /usr/lib. So, we passed 'LDFLAGS' along with 'CPPFLAGS' during configure. But the error persisted. Editing the last parte of liboct*.la files to change the 'libdir' didn't work as well.
Now a new error arised, the Makefile was calling 'grep/sed' at some point which showed that the library file have been moved (which was present in /e/octave_dev/lib/octave/4.1.0+/) And then the script tries to use it from /usr/lib/octave/4.1.0+. But the library files were actually present in the previous directory only and not in /usr/lib. So, we passed 'LDFLAGS' along with 'CPPFLAGS' during configure. But the error persisted. Editing the last part of liboct*.la files to change the 'libdir' didn't work as well.
As per Mike's suggestion deleting the liboct*.la files helped and the ''make'' step moved even furthur. Now the errors showed some missing libraries (lapack, GraphickMagik, ...)
As per Mike's suggestion deleting the liboct*.la files helped and the ''make'' step moved even furthur. Now the errors showed some missing libraries (lapack, GraphickMagik, ...)
Since all these libraries were present along with octave so we added "/e/octave_dev/lib" to LDFLAGS too. Now the build went even furthur. This time there were some C++11 errors. As per Mike's and Tatsuro's suggestions Octave uses 4.9.3 gcc on windows and gcc changed the library around version 5. Msys2 had GCC 5.4 which caused the errors. So, either pass all the gcc tools in octave_dev/bin to configure on MSYS2 or use octave's shell with MSYS2 on top of it.
Since all these libraries were present along with octave so we added "/e/octave_dev/lib" to LDFLAGS too. Now the build went even further. This time there were some C++11 errors. As per Mike's and Tatsuro's suggestions Octave uses 4.9.3 gcc on windows and gcc changed the library around version 5. Msys2 had GCC 5.4 which caused the errors. So, either pass all the gcc tools in octave_dev/bin to configure on MSYS2 or use octave's shell with MSYS2 on top of it.
At this point, I switched to using the bash shell present with octave.
At this point, I switched to using the bash shell present with octave.


=== Use MSYS2 on top of Octave's MSYS environment ===
=== Use Octave's MSYS environment with help of msys2 toolchains ===
 
==== Build boost_python  ====
When using the bash prompt from 'octave-cli.exe' we could not use the libboost-python that we used octave Msys environment with help of MSYS2. First thing we had to do to build boost-python from source. The steps I used to build boost are: 
 
# Open octave-cli.exe
# cd e:/boost_1_61_0
# system bash
# export PATH=$PATH:/c/msys64/usr/bin
# ./bootstrap
# ./b2 install --with-python --prefix=/e/boost_build
 
 
At this moment building of boost-python with 64 bit tool chain fails at final link stage. So far octave-4.1.0+ 64 bit was used. From this point we shifted to using 32 bit octave. Using 32 bit octave 4.1.0+ for windows build of boost_python was successful.
 
==== Build Pytave  ====
 
# Open octave-cli.exe
# cd (Pytave build directoy)
# system bash
# export PATH=$PATH:/c/msys64/usr/bin
# ./configure (with CPPFLAGS LDFLAGS setting )
# make
 
There was an error in configure that PYTHON_EXTRA_LIBS was 'None' on windows and hence the configure failed. As per Mike's suggestion I set it to a valid gcc flag for the time being with ''export $PYTHON_EXTRA_LIBS=-g''
After this too the build failed with undefined references to few symbols. Tatsuro suggested to build boost with ''link=static,shared''. He had successfully built boost with it but I could not build boost with this option on my PC.
 
To sum up, Tatsuro has built 32 bit version of octave and used it to build libboost-python with the option ''link=static,shared'' and has successfully got the configure step done. But in the ''make'' step he is getting some undefined symbol errors for ''__imp__'' symbols.
(This information is obtained from trials of out-of-box of linking object files. )
 
=== Summary ===
 
Here, I will note down the steps (which worked) as per Tatsuro's mail from the very start. So, anyone can try to build pytave on windows. These are the details he sent on the mailing list (slightly edited):
 
* Build 32bit 4.1.0+ for windows by mxe-octave on linux. Refer to the [http://wiki.octave.org/Windows_Installer wiki page].
* Install 32 bit Python for windows from [https://www.python.org/ Python website].
* Install numpy on 32 bit Python for windows
:# Get get-pip.py (e.g. https://bootstrap.pypa.io/get-pip.py)
:# In command prompt cd to python2.7 32bit directory e.g. ''cd C:\Python27_32''
:# Install pip by executing ''python get-pip.py''
:# cd to Scripts dir with ''cd C:\Python27_32\Scripts''
:# Install numpy via pip by executing ''pip install numpy''
* Install msys2, update and install tools
:# Download and install [https://sourceforge.net/projects/msys2/files/Base/x86_64/ msys2].
:# Startup msys2 prompt from shortcut C:\msys64\
:# Execute ''update-core''
:# After update is finished, exit with "x" button and do not use exit command.
:# Start MSYS2 again and execute ''pacman -Suu''
:# Again exit with "x" button on title bar
:# Start MSYS2 and install build toolkit with ''pacman -S --needed base-devel msys/dos2unix'' and mercurial with ''pacman -S mercurial''
* Build liboost-python with shared libraries
:# Download [https://sourceforge.net/projects/boost/files/boost/1.61.0/ boost_1_61_0.tar.bz2] and un-archieve it.
:# Open octave-cli from bin directory of the 32 bit octave that you built
:# run ''system bash'' and then cd to the unarchieved directory of boost
:# export path to include msys2 tools and python with ''export PATH=$PATH:/c/msys64/usr/bin:/c/Python27_32:/c/Python27_32/scripts''
:# Run ''./bootstrap.sh'' then b2.exe and bjam.exe will be built
:# Compile source files and link object files to generate (shared) libraries.
::: ''./b2 install  link=static,shared --with-python cflags='IC:/Python27_32/include' \''
::: ''cxxflags='-IC:/Python27_32/include'  linkflags='-LC:/Python27_32/libs' \''
::: ''--prefix=/d/usr/Tatsu/program/Pytave/PytaveBuild/boost/mingw32/boost_1_16_inst \''
::: ''2>&1 | tee make.log''
::: cflags, cxxflags and linkflags are not required if youinstall python2.7 (32bit) to the default directory (C:\Python27).
::: ''link=static,shared'' gives static and shared libraries
::: ''--with-python'' builds only libboost_python. ''--prefix'' specifies install directory.
::: ''2>&1 | tee make.log'' is added to memorize build process.
:# copy (install dir )/lib/libboost_python-mgw49-mt-1_61.dll.a and rename it as libboost_python.dll.a
* Build PyTave
:# Open octave-cli and run ''system bash'' and export PATH to include msys2 tools and python ''export PATH=$PATH:/c/msys64/usr/bin:/c/Python27_32:/c/Python27_32/scripts''
:# Also export libboost_python dir to PATH with ''export PATH=$PATH:/d/usr/Tatsu/program/Pytave/PytaveBuild/boost/mingw32/boost_1_16_inst/lib''
:# cd to build directory for pytave ''cd "D:\usr\Tatsu\program\Pytave\PytaveBuild\Pytave\build32"''
:# Set CPPFLAGS as libtool is escaping th '\' characters...
::: ''export CPPFLAGS='-IC:/Python27_32/include -ID:/usr/Tatsu/program/Pytave/PytaveBuild/boost/mingw32/boost_1_16_inst/include -ID:/usr/Tatsu/program/Pytave/PytaveBuild/boost/mingw32/boost_1_16_inst/include/boost-1_61 -IC:/octave/octave-4.1.0plus_x86-2016-06-25-08-57/include/octave-4.1.0+ -IC:/octave/octave-4.1.0plus_x86-2016-06-25-08-57/include/octave-4.1.0+/octave -Ic:/Python27_32/include -Ic:/Python27_32/lib/site-packages/numpy/core/include'''
:# set LDFLAGS as pytave misdirects library directory of python.
::: ''export LDFLAGS='-Wl,--enable-auto-import -Wl,-export-all-symbols -LC:/Python27_32/libs -LD:/usr/Tatsu/program/Pytave/PytaveBuild/boost/mingw32/boost_1_16_inst/lib -LC:/octave/octave-4.1.0plus_x86-2016-06-25-08-57/lib -LC:/octave/octave-4.1.0plus_x86-2016-06-25-08-57/lib/octave/4.1.0+'''
:# Run configure of pytave
::: ''../pytave/configure LIBS='-LC:/Python27/libs' --prefix='D:/usr/Tatsu/program/Pytave/PytaveInst32' \''
::: ''--build=i686-w64-mingw32 --host=i686-w64-mingw32 \''
::: ''--with-boost='D:/usr/Tatsu/program/Pytave\/PytaveBuild/boost/mingw32/boost_1_16_inst'''
::: The build and host --build=i686-w64-mingw32 --host=i686-w64-mingw32 are set because many configure scripts mis-detect them when msys2 is 64bit.
:# Execute ''make''. Build fails with '''warning: undefined symbols not allowed in i686-w64-mingw32 shared libraries; building static only''' and '''error: object name conflicts in archive: .libs/_pytave.lax/libpytave.a//d/usr/Tatsu/program/Pytave/PytaveBuild/Pytave/build32/./.libs/libpytave.a'''
 
=== Conclusion ===
 
With some efforts, build Pytave on windows has not been completed yet.
Here list of unsolved issue including boost-python are listed
 
# libboost_python cannot be build be build on 64 bit environments
# path separator handling issue in libtool (windows standard path separator \ (back slash) is ignored.
# make fails at building shared library
 
For 3, to quote Tatsuro ''A lot of _imp__ symbols errors are detected because no _imp__(symbol)s exist in object files. While object files libboost_python for threading-multi have _imp__(symbol)s in their object files. Boost does not show full compiling states. Therefore I do not know what libboost is to do.''
 
I too have googled extensively but haven't gotten to any solution for this problem.

Navigation menu