Continuous Build: Difference between revisions

From Octave
Jump to navigation Jump to search
(→‎External links: Update URL)
 
(24 intermediate revisions by 4 users not shown)
Line 1: Line 1:
There is a build server continuously building the latest tip of the development branch of Octave. The build server is running [http://nixos.org/hydra/ Hydra], which is based on the [http://nixos.org/nix/ Nix] functional package management system. This is the official continuous build system of the GNU Project. The Hydra build of Octave is currently maintained by [[User:Mtmiller]].
GNU Octave uses [https://buildbot.net/ Buildbot] to build and test the current development version on multiple systems in a number of different configurations.


= Octave Job Overview =
{{Note|The current status of the builds may be found at https://buildbot.octave.org/#/waterfall.}}


The [http://hydra.nixos.org/jobset/gnu/octave-default octave-default jobset overview] page is the main entry point to the Hydra build of Octave. It shows the status of the most recent builds ("evaluations" in Hydra terminology).
= Systems and Configurations =


= Latest Build Logs =
The following systems and configurations are currently covered for Octave builds:


The full build logs for the latest builds are available at the following links, for quick reference:
{| class="wikitable"
|-
! Builder ID !! Hg Version !! System !! Compiler !! Build Options !! Frequency
|-
| clang-4.0-debian || default || Debian Testing || Clang 4.0 || || Any Change
|-
| stable-clang-4.0-debian || stable || Debian Testing || Clang 4.0 || || Any Change
|-
| clang-5.0-debian || default || Debian Testing || Clang 5.0 || || Any Change
|-
| stable-clang-5.0-debian || stable || Debian Testing || Clang 5.0 || || Any Change
|-
| clang-fedora || default || Fedora (current release) || Clang (system default) || || Any Change
|-
| stable-clang-fedora || stable || Fedora (current release) || Clang (system default) || || Any Change
|-
| clang-osx (currently inactive) || default || OS X || Clang || || Any Change
|-
| gcc-7-debian || default || Debian Testing || GCC 7 || || Any Change
|-
| gcc-7-lto-debian || default || Debian Testing || GCC 7 || Enable link time optimization || Any Change
|-
| gcc-fedora || default || Fedora (current release) || GCC (system default) || || Any Change
|-
| gcc-lto-fedora || default || Fedora (current release) || GCC (system default) || Enable link time optimization || Any Change
|-
| no-extras-debian || default || Debian Testing || GCC (system default) || Disable all optional dependencies || Any Change
|-
| stable-no-extras-debian || stable || Debian Testing || GCC (system default) || Disable all optional dependencies || Any Change
|-
|}


* [http://hydra.nixos.org/job/gnu/octave-default/tarball/latest/log/raw Tarball]
And for mxe-octave:
* [http://hydra.nixos.org/job/gnu/octave-default/build.x86_64-linux/latest/log/raw Linux x86_64]
* [http://hydra.nixos.org/job/gnu/octave-default/build.i686-linux/latest/log/raw Linux i686]
* [http://hydra.nixos.org/job/gnu/octave-default/coverage/latest/log/raw Coverage]


= Snapshot Source Tarball =
{| class="wikitable"
|-
! Builder ID !! Hg Version !! Build System !! Host System !! Compiler !! Build Options !! Frequency
|-
| mxe-native-all-on-debian || default || Debian Testing || Debian || GCC (mxe-octave default) || GNU Linux, build all dependencies || Daily
|-
| mxe-native-on-debian || default || Debian Testing || Debian || GCC (system default) || GNU Linux, use system compiler, fontconfig, and X11 libraries || Daily
|-
| w32-on-debian || default || Debian Testing || Windows || GCC (mxe-octave default) || Windows 32 || Daily
|-
| w32-stable-on-debian || stable || Debian Testing || Windows || GCC (mxe-octave default) || Windows 32 || Daily
|-
| w32-release-on-debian || release (tarball) || Debian Testing || Windows || GCC (mxe-octave default) || Windows 32 || Daily
|-
| w64-32-on-debian || default || Debian Testing || Windows || GCC (mxe-octave default) || Windows 64 || Daily
|-
| w64-32-stable-on-debian || stable || Debian Testing || Windows || GCC (mxe-octave default) || Windows 64 || Daily
|-
| w64-32-release-on-debian || release (tarball) || Debian Testing || Windows || GCC (mxe-octave default) || Windows 64 || Daily
|-
| w64-64-on-debian || default || Debian Testing || Windows || GCC (mxe-octave default) || Windows 64, 64-bit indexing || Daily
|-
| w64-64-stable-on-debian || stable || Debian Testing || Windows || GCC (mxe-octave default) || Windows 64, 64-bit indexing || Daily
|-
| w64-64-release-on-debian || release (tarball) || Debian Testing || Windows || GCC (mxe-octave default) || Windows 64, 64-bit indexing || Daily
|-
|}


A nice product of the Hydra continuous build of Octave is that there is always a [http://hydra.nixos.org/job/gnu/octave-default/tarball/latest/download source snapshot] available for download. This source snapshot is always built from the current tip of the development branch in whatever state it is in. Therefore, it may not compile or run properly and should definitely not be used for anything other than evaluation. This can be useful to evaluate the current state of Octave's development branch without needing to clone the Mercurial repository.
= Setup and run a Buildbot Worker =


= Coverage Report =
Your system may be behind a firewall.  It does not have to have a distinct public IP address.


Another product of the Hydra continuous build is the [http://hydra.nixos.org/job/gnu/octave-default/coverage/latest/download code coverage report]. The latest coverage report is always built from the current tip of the development branch. This report gives an analysis of the code coverage of the current Octave test suite. This can be used to find unexercised code paths and help improve the coverage of Octave's test suite (see [[Projects#Tests]]).
To support Octave development and run a Buildbot Worker, you must do the following:


[[Category:Development]]
* Contact the [https://octave.discourse.group/c/maintainers/7 Octave Maintainers on Discourse] to let us know that you wish to provide a system to use as a Buildbot Worker.  We will give you a <code>WORKERNAME</code> and a '''secret''' <code>PASSWORD</code> to configure your Buildbot Worker.
* Install buildbot.  Packages exist for most distributions.  See the buildbot docs for other options.  You should create a separate user account with no special privileges that will run buildbot.
* Decide for a <code>BASEDIR</code>.  For example, if the home directory for the buildbot user is {{Path|/var/lib/buildbot}} and your <code>WORKERNAME</code> is set to <code>'debian-x86_64'</code> , then <code>BASEDIR</code> might be {{Path|/var/lib/buildbot/worker/debian-x86_64}}.
* <code>MASTERHOST</code> is <code>buildbot.octave.org</code> and <code>PORT</code> is <code>9989</code>.
* Create the configuration<pre>buildbot-worker create-worker BASEDIR MASTERHOST:PORT WORKERNAME PASSWORD</pre>
* Run buildbot on the worker system, preferably by starting it automatically when your system boots.  It should be running with the buildbot user ID. <pre>buildbot-worker start BASEDIR</pre>
 
== ccache ==
 
You may also want to set up '''ccache''' to work with buildbot (strongly recommended to speed up builds).  If you create a directory {{Path|~/buildbot/bin}}, it will be added to the execution PATH when the Buildbot Master runs commands on the Buildbot Worker.  This directory can have symbolic links like the following:
 
cc      -> /usr/bin/ccache
c++      -> /usr/bin/ccache
gcc      -> /usr/bin/ccache
gfortran -> /usr/bin/ccache
 
They should point to the actual location of ccache if it is not in {{Path|/usr/bin}}.
 
== Space Requirements ==
 
Building Octave takes a significant amount of disk space.  With debugging symbols, you may need several GB for each build, plus room for ccache (possibly 50GB) if you use it.  If you use a cache size that is larger than the default, you'll need to specify that in the {{Path|.ccache/ccache.conf}} file using a line like
 
max_size = 50G
 
If the directory containing the build and ccache directories doesn't have sufficient space, then these directory names may point to a separate partition that does have enough space available.
 
= Continuous Deployment of Octave for Linux =
 
== Edge channel of Octave Snap App ==
 
The "edge" channel of Octave's Snap App is built from the current version of the stable branch. That means it contains changes that are likely to be included in the next minor release of Octave.
 
It can be download from the [https://snapcraft.io/octave Snap Store] selecting "latest/edge" from the dropdown menu.
 
= Continuous Deployment of Octave for Windows =
 
== Freshly brewed Octave ==
 
Unreleased versions of Octave for Windows are available from [https://nightly.octave.org/#/download nightly.octave.org]. These are installers built with MXE Octave very similarly how the "official" Octave for Windows is built. They can be installed just like the "official" versions of Octave for Windows.
 
Available variants include versions built from the release branch of MXE Octave built for Windows 64-bit (with 32-bit or 64-bit Fortran indexing size). Additionally, one variant is built from the default branch of MXE Octave (more up-to-date dependencies).
 
All of these versions are built from the stable branch of Octave. That means they contain changes that are likely to be included in the next minor release of Octave.
 
Unreleased versions might be more unstable than released versions. But they might also contain fixes for bugs that haven't been released yet.
 
== GitHub build artifacts ==
 
Build artifacts are available for versions of Octave for MINGW64 from the CI running on the mirror of Octave on GitHub. These artefacts can be downloaded from the bottom of the [https://github.com/gnu-octave/octave/actions workflow logs] for builds from the default branch of Octave. After downloading the build artifact, extract the <code>.zip</code> file to an empty folder (e.g., <code>C:\Octave\test</code>).
 
The default branch of Octave contains changes that are likely to be included in the next major release of Octave. Some functionality of Octave on the default branch might be broken. But it will likely contain new features that aren't yet included in the newest released version of Octave.
 
These artifacts are built with MSYS2. So, MSYS2 must be installed to be able to run the artifact. MSYS2 can be downloaded from their [https://www.msys2.org/ website].
After installing MSYS2, open a MINGW64 shell (the blue icon), update MSYS2 and install the necessary dependencies with (the second command must be executed in one single line):
  pacman -Syu
  pacman -S --needed mingw-w64-x86_64-gcc-libgfortran mingw-w64-x86_64-arpack mingw-w64-x86_64-curl mingw-w64-x86_64-fftw mingw-w64-x86_64-fltk mingw-w64-x86_64-ghostscript mingw-w64-x86_64-gl2ps mingw-w64-x86_64-glpk mingw-w64-x86_64-gnuplot mingw-w64-x86_64-graphicsmagick mingw-w64-x86_64-hdf5 mingw-w64-x86_64-libsndfile mingw-w64-x86_64-portaudio mingw-w64-x86_64-qhull mingw-w64-x86_64-qrupdate mingw-w64-x86_64-qscintilla mingw-w64-x86_64-qt5-tools mingw-w64-x86_64-sundials mingw-w64-x86_64-suitesparse
 
After that, <code>cd</code> to the directory with the extracted content of the <code>.zip</code> file. (It should be one single file named <code>octave.tar.gz</code>.) For the exemplary folder from above, that would be:
  cd /c/Octave/test
 
Extract the tarball and add the <code>bin</code> directory to the system search PATH:
  tar -xvzf octave.tar.gz
  export PATH=/c/Octave/test/mingw64/bin:$PATH
 
After that, it should be possible to start that "nightly" version with the command <code>octave --gui</code> from the same shell.
 
= External links =
 
* [https://hg.octave.org/octave-buildbot/ Buildbot configuration repository] for https://buildbot.octave.org/
* [https://github.com/gnu-octave/octave-buildbot Buildbot configuration] for https://nightly.octave.org/
 
[[Category:Building]]

Latest revision as of 18:19, 7 April 2024

GNU Octave uses Buildbot to build and test the current development version on multiple systems in a number of different configurations.

Info icon.svg
The current status of the builds may be found at https://buildbot.octave.org/#/waterfall.

Systems and Configurations[edit]

The following systems and configurations are currently covered for Octave builds:

Builder ID Hg Version System Compiler Build Options Frequency
clang-4.0-debian default Debian Testing Clang 4.0 Any Change
stable-clang-4.0-debian stable Debian Testing Clang 4.0 Any Change
clang-5.0-debian default Debian Testing Clang 5.0 Any Change
stable-clang-5.0-debian stable Debian Testing Clang 5.0 Any Change
clang-fedora default Fedora (current release) Clang (system default) Any Change
stable-clang-fedora stable Fedora (current release) Clang (system default) Any Change
clang-osx (currently inactive) default OS X Clang Any Change
gcc-7-debian default Debian Testing GCC 7 Any Change
gcc-7-lto-debian default Debian Testing GCC 7 Enable link time optimization Any Change
gcc-fedora default Fedora (current release) GCC (system default) Any Change
gcc-lto-fedora default Fedora (current release) GCC (system default) Enable link time optimization Any Change
no-extras-debian default Debian Testing GCC (system default) Disable all optional dependencies Any Change
stable-no-extras-debian stable Debian Testing GCC (system default) Disable all optional dependencies Any Change

And for mxe-octave:

Builder ID Hg Version Build System Host System Compiler Build Options Frequency
mxe-native-all-on-debian default Debian Testing Debian GCC (mxe-octave default) GNU Linux, build all dependencies Daily
mxe-native-on-debian default Debian Testing Debian GCC (system default) GNU Linux, use system compiler, fontconfig, and X11 libraries Daily
w32-on-debian default Debian Testing Windows GCC (mxe-octave default) Windows 32 Daily
w32-stable-on-debian stable Debian Testing Windows GCC (mxe-octave default) Windows 32 Daily
w32-release-on-debian release (tarball) Debian Testing Windows GCC (mxe-octave default) Windows 32 Daily
w64-32-on-debian default Debian Testing Windows GCC (mxe-octave default) Windows 64 Daily
w64-32-stable-on-debian stable Debian Testing Windows GCC (mxe-octave default) Windows 64 Daily
w64-32-release-on-debian release (tarball) Debian Testing Windows GCC (mxe-octave default) Windows 64 Daily
w64-64-on-debian default Debian Testing Windows GCC (mxe-octave default) Windows 64, 64-bit indexing Daily
w64-64-stable-on-debian stable Debian Testing Windows GCC (mxe-octave default) Windows 64, 64-bit indexing Daily
w64-64-release-on-debian release (tarball) Debian Testing Windows GCC (mxe-octave default) Windows 64, 64-bit indexing Daily

Setup and run a Buildbot Worker[edit]

Your system may be behind a firewall. It does not have to have a distinct public IP address.

To support Octave development and run a Buildbot Worker, you must do the following:

  • Contact the Octave Maintainers on Discourse to let us know that you wish to provide a system to use as a Buildbot Worker. We will give you a WORKERNAME and a secret PASSWORD to configure your Buildbot Worker.
  • Install buildbot. Packages exist for most distributions. See the buildbot docs for other options. You should create a separate user account with no special privileges that will run buildbot.
  • Decide for a BASEDIR. For example, if the home directory for the buildbot user is /var/lib/buildbot and your WORKERNAME is set to 'debian-x86_64' , then BASEDIR might be /var/lib/buildbot/worker/debian-x86_64.
  • MASTERHOST is buildbot.octave.org and PORT is 9989.
  • Create the configuration
    buildbot-worker create-worker BASEDIR MASTERHOST:PORT WORKERNAME PASSWORD
  • Run buildbot on the worker system, preferably by starting it automatically when your system boots. It should be running with the buildbot user ID. 
    buildbot-worker start BASEDIR

ccache[edit]

You may also want to set up ccache to work with buildbot (strongly recommended to speed up builds). If you create a directory ~/buildbot/bin, it will be added to the execution PATH when the Buildbot Master runs commands on the Buildbot Worker. This directory can have symbolic links like the following: 

cc       -> /usr/bin/ccache
c++      -> /usr/bin/ccache
gcc      -> /usr/bin/ccache
gfortran -> /usr/bin/ccache

They should point to the actual location of ccache if it is not in /usr/bin.

Space Requirements[edit]

Building Octave takes a significant amount of disk space. With debugging symbols, you may need several GB for each build, plus room for ccache (possibly 50GB) if you use it. If you use a cache size that is larger than the default, you'll need to specify that in the .ccache/ccache.conf file using a line like 

max_size = 50G

If the directory containing the build and ccache directories doesn't have sufficient space, then these directory names may point to a separate partition that does have enough space available.

Continuous Deployment of Octave for Linux[edit]

Edge channel of Octave Snap App[edit]

The "edge" channel of Octave's Snap App is built from the current version of the stable branch. That means it contains changes that are likely to be included in the next minor release of Octave.

It can be download from the Snap Store selecting "latest/edge" from the dropdown menu.

Continuous Deployment of Octave for Windows[edit]

Freshly brewed Octave[edit]

Unreleased versions of Octave for Windows are available from nightly.octave.org. These are installers built with MXE Octave very similarly how the "official" Octave for Windows is built. They can be installed just like the "official" versions of Octave for Windows.

Available variants include versions built from the release branch of MXE Octave built for Windows 64-bit (with 32-bit or 64-bit Fortran indexing size). Additionally, one variant is built from the default branch of MXE Octave (more up-to-date dependencies).

All of these versions are built from the stable branch of Octave. That means they contain changes that are likely to be included in the next minor release of Octave.

Unreleased versions might be more unstable than released versions. But they might also contain fixes for bugs that haven't been released yet.

GitHub build artifacts[edit]

Build artifacts are available for versions of Octave for MINGW64 from the CI running on the mirror of Octave on GitHub. These artefacts can be downloaded from the bottom of the workflow logs for builds from the default branch of Octave. After downloading the build artifact, extract the .zip file to an empty folder (e.g., C:\Octave\test).

The default branch of Octave contains changes that are likely to be included in the next major release of Octave. Some functionality of Octave on the default branch might be broken. But it will likely contain new features that aren't yet included in the newest released version of Octave.

These artifacts are built with MSYS2. So, MSYS2 must be installed to be able to run the artifact. MSYS2 can be downloaded from their website. After installing MSYS2, open a MINGW64 shell (the blue icon), update MSYS2 and install the necessary dependencies with (the second command must be executed in one single line): 

 pacman -Syu
 pacman -S --needed mingw-w64-x86_64-gcc-libgfortran mingw-w64-x86_64-arpack mingw-w64-x86_64-curl mingw-w64-x86_64-fftw mingw-w64-x86_64-fltk mingw-w64-x86_64-ghostscript mingw-w64-x86_64-gl2ps mingw-w64-x86_64-glpk mingw-w64-x86_64-gnuplot mingw-w64-x86_64-graphicsmagick mingw-w64-x86_64-hdf5 mingw-w64-x86_64-libsndfile mingw-w64-x86_64-portaudio mingw-w64-x86_64-qhull mingw-w64-x86_64-qrupdate mingw-w64-x86_64-qscintilla mingw-w64-x86_64-qt5-tools mingw-w64-x86_64-sundials mingw-w64-x86_64-suitesparse

After that, cd to the directory with the extracted content of the .zip file. (It should be one single file named octave.tar.gz.) For the exemplary folder from above, that would be: 

 cd /c/Octave/test

Extract the tarball and add the bin directory to the system search PATH: 

 tar -xvzf octave.tar.gz
 export PATH=/c/Octave/test/mingw64/bin:$PATH

After that, it should be possible to start that "nightly" version with the command octave --gui from the same shell.

External links[edit]

Retrieved from "https://wiki.octave.org/wiki/index.php?title=Continuous_Build&oldid=15122"