Continuous Build: Difference between revisions

Jump to navigation Jump to search
940 bytes removed ,  22 September 2020
(Improve markup.)
(6 intermediate revisions by 3 users not shown)
Line 1: Line 1:
We are using [https://buildbot.net/ Buildbot] to build and test the current development version of Octave on multiple systems in a number of different configurations.
GNU Octave uses [https://buildbot.net/ Buildbot] to build and test the current development version on multiple systems in a number of different configurations.


{{Note|The current status of the builds may be found at http://buildbot.octave.org:8010/#/waterfall.}}
{{Note|The current status of the builds may be found at http://buildbot.octave.org:8010/#/waterfall.}}
Line 11: Line 11:
! Builder ID !! Hg Version !! System !! Compiler !! Build Options !! Frequency
! Builder ID !! Hg Version !! System !! Compiler !! Build Options !! Frequency
|-
|-
| clang-3.8-debian || default || Debian Testing || Clang 3.8 || Disable GraphicsMagick || Any Change
| clang-4.0-debian || default || Debian Testing || Clang 4.0 || || Any Change
|-
|-
| clang-4.0-debian || default || Debian Testing || Clang 4.0 || || Any Change
| stable-clang-4.0-debian || stable || Debian Testing || Clang 4.0 || || Any Change
|-
|-
| clang-5.0-debian || default || Debian Testing || Clang 5.0 || || Any Change
| clang-5.0-debian || default || Debian Testing || Clang 5.0 || || Any Change
|-
|-
| clang-fedora || default || Fedora 25 || Clang (system default) || || Any Change
| stable-clang-5.0-debian || stable || Debian Testing || Clang 5.0 || || Any Change
|-
| clang-fedora || default || Fedora (current release) || Clang (system default) || || Any Change
|-
|-
| clang-osx || default || OS X || Clang || || Any Change
| stable-clang-fedora || stable || Fedora (current release) || Clang (system default) || || Any Change
|-
|-
| gcc-6-debian || default || Debian Testing || GCC 6 || || Any Change
| clang-osx (currently inactive) || default || OS X || Clang || || Any Change
|-
|-
| gcc-7-debian || default || Debian Testing || GCC 7 || || Any Change
| gcc-7-debian || default || Debian Testing || GCC 7 || || Any Change
|-
|-
| gcc-7-lto-debian || default || Debian Testing || GCC (system default) || || Any Change
| gcc-7-lto-debian || default || Debian Testing || GCC 7 || Enable link time optimization || Any Change
|-
|-
| gcc-fedora || default || Fedora 25 || GCC (system default) || || Any Change
| gcc-fedora || default || Fedora (current release) || GCC (system default) || || Any Change
|-
|-
| gcc-lto-fedora || default || Fedora 25 || GCC (system default) || Enable link time optimization || Any Change
| gcc-lto-fedora || default || Fedora (current release) || GCC (system default) || Enable link time optimization || Any Change
|-
|-
| no-extras-debian || default || Debian Testing || GCC (system default) || Disable all optional dependencies || Any Change
| no-extras-debian || default || Debian Testing || GCC (system default) || Disable all optional dependencies || Any Change
|-
| stable-no-extras-debian || stable || Debian Testing || GCC (system default) || Disable all optional dependencies || Any Change
|-
|-
|}
|}
Line 41: Line 45:
! Builder ID !! Hg Version !! Build System !! Host System !! Compiler !! Build Options !! Frequency
! Builder ID !! Hg Version !! Build System !! Host System !! Compiler !! Build Options !! Frequency
|-
|-
| mxe-native-all-on-debian || default || Debian Testing || Debian || GCC (system default) || GNU Linux, build all dependencies || Daily
| mxe-native-all-on-debian || default || Debian Testing || Debian || GCC (mxe-octave default) || GNU Linux, build all dependencies || Daily
|-
|-
| mxe-native-on-debian || default || Debian Testing || Debian || GCC (system default) || GNU Linux, use system compiler, fontconfig, and X11 libraries || Daily
| mxe-native-on-debian || default || Debian Testing || Debian || GCC (system default) || GNU Linux, use system compiler, fontconfig, and X11 libraries || Daily
Line 48: Line 52:
|-
|-
| w32-stable-on-debian || stable || Debian Testing || Windows || GCC (mxe-octave default) || Windows 32 || Daily
| w32-stable-on-debian || stable || Debian Testing || Windows || GCC (mxe-octave default) || Windows 32 || Daily
|-
| w32-release-on-debian || release (tarball) || Debian Testing || Windows || GCC (mxe-octave default) || Windows 32 || Daily
|-
|-
| w64-32-on-debian || default || Debian Testing || Windows || GCC (mxe-octave default) || Windows 64 || Daily
| w64-32-on-debian || default || Debian Testing || Windows || GCC (mxe-octave default) || Windows 64 || Daily
|-
|-
| w64-32-stable-on-debian || stable || Debian Testing || Windows || GCC (mxe-octave default) || Windows 64 || Daily
| w64-32-stable-on-debian || stable || Debian Testing || Windows || GCC (mxe-octave default) || Windows 64 || Daily
|-
| w64-32-release-on-debian || release (tarball) || Debian Testing || Windows || GCC (mxe-octave default) || Windows 64 || Daily
|-
|-
| w64-64-on-debian || default || Debian Testing || Windows || GCC (mxe-octave default) || Windows 64, 64-bit indexing || Daily
| w64-64-on-debian || default || Debian Testing || Windows || GCC (mxe-octave default) || Windows 64, 64-bit indexing || Daily
|-
| w64-64-stable-on-debian || stable || Debian Testing || Windows || GCC (mxe-octave default) || Windows 64, 64-bit indexing || Daily
|-
| w64-64-release-on-debian || release (tarball) || Debian Testing || Windows || GCC (mxe-octave default) || Windows 64, 64-bit indexing || Daily
|-
|-
|}
|}


= Build Slave Configuration =
= Setup and run a Buildbot Worker =
 
To run a build slave for Octave, you must do the following:
 
* Contact the [mailto:octave-maintainers@gnu.org Octave Maintainers] mailing list to let us know that you wish to provide a system to use as a build slave.
* Install buildbot.  Packages exist for most distributions.  See the buildbot docs for other options.  You should create a separate user account with no special privileges that will run buildbot.
* Create a configuration file (see below).
* Run buildbot on the slave system, preferably by starting it automatically when your system boots.  It should be running with the buildbot user ID.
 
You may also want to set up '''ccache''' to work with buildbot (strongly recommended to speed up builds).  If you create a directory {{Path|~/buildbot/bin}}, it will be added to the execution PATH when buildbot runs commands on the slave.  This directory can have symbolic links like the following:
 
lrwxrwxrwx 1 buildbot buildbot 15 Aug 26 11:39 gcc -> /usr/bin/ccache
lrwxrwxrwx 1 buildbot buildbot 15 Aug 26 11:40 cc -> /usr/bin/ccache
lrwxrwxrwx 1 buildbot buildbot 15 Aug 26 11:40 c++ -> /usr/bin/ccache
lrwxrwxrwx 1 buildbot buildbot 15 Aug 31 23:46 gfortran -> /usr/bin/ccache
 
They should point to the actual location of ccache if it is not in {{Path|/usr/bin}}.


Your system may be behind a firewall.  It does not have to have a distinct public IP address.
Your system may be behind a firewall.  It does not have to have a distinct public IP address.


== Sample Slave Configuration File ==
To support Octave development and run a Buildbot Worker, you must do the following:


You must edit the settings for <code>basedir</code>, <code>slavename</code>, and <code>password</code>.  The <code>basedir</code> should be the absolute and fully expanded name of the directory containing the configuration file.  For example, if the home directory for the buildbot user is {{Path|/var/lib/buildbot}} and your <code>slavename</code> is set to 'debian-x86_64' , then <code>basedir</code> might be {{Path|/var/lib/buildbot/slaves/debian-x86_64}}. The password should not be anything valuable. You'll be asked to provide the <code>slavename</code> and <code>password</code> so that they may be added to the master configuration on buildbot.octave.orgDo '''not''' post the password to the octave-maintainers mailing list.
* Contact the [https://octave.discourse.group/c/maintainers/7 Octave Maintainers on Discourse] to let us know that you wish to provide a system to use as a Buildbot Worker.  We will give you a <code>WORKERNAME</code> and a '''secret''' <code>PASSWORD</code> to configure your Buildbot Worker.
* Install buildbotPackages exist for most distributions.  See the buildbot docs for other options.  You should create a separate user account with no special privileges that will run buildbot.
* Decide for a <code>BASEDIR</code>.  For example, if the home directory for the buildbot user is {{Path|/var/lib/buildbot}} and your <code>WORKERNAME</code> is set to <code>'debian-x86_64'</code> , then <code>BASEDIR</code> might be {{Path|/var/lib/buildbot/worker/debian-x86_64}}.
* <code>MASTERHOST</code> is <code>buildbot.octave.org</code> and <code>PORT</code> is <code>9989</code>.
* Create the configuration<pre>buildbot-worker create-worker BASEDIR MASTERHOST:PORT WORKERNAME PASSWORD</pre>
* Run buildbot on the worker system, preferably by starting it automatically when your system bootsIt should be running with the buildbot user ID. <pre>buildbot-worker start BASEDIR</pre>


<syntaxhighlight lang="python">
== ccache ==
import os


from buildslave.bot import BuildSlave
You may also want to set up '''ccache''' to work with buildbot (strongly recommended to speed up builds). If you create a directory {{Path|~/buildbot/bin}}, it will be added to the execution PATH when the Buildbot Master runs commands on the Buildbot Worker. This directory can have symbolic links like the following:
from twisted.application import service


basedir = '/PATH/TO/DIRECTORY/CONTAINING/THIS/CONFIGURATION/FILE'
cc      -> /usr/bin/ccache
rotateLength = 10000000
c++      -> /usr/bin/ccache
maxRotatedFiles = 10
gcc      -> /usr/bin/ccache
gfortran -> /usr/bin/ccache


# if this is a relocatable tac file, get the directory containing the TAC
They should point to the actual location of ccache if it is not in {{Path|/usr/bin}}.
if basedir == '.':
    import os.path
    basedir = os.path.abspath(os.path.dirname(__file__))
 
# note: this line is matched against to check that this is a buildslave
# directory; do not edit it.
application = service.Application('buildslave')
 
try:
  from twisted.python.logfile import LogFile
  from twisted.python.log import ILogObserver, FileLogObserver
  logfile = LogFile.fromFullPath(os.path.join(basedir, "twistd.log"), rotateLength=rotateLength,
                                maxRotatedFiles=maxRotatedFiles)
  application.setComponent(ILogObserver, FileLogObserver(logfile).emit)
except ImportError:
  # probably not yet twisted 8.2.0 and beyond, can't set log yet
  pass
 
buildmaster_host = 'buildbot.octave.org'
port = 9989
slavename = 'SLAVE-NAME'
passwd = 'PASSWORD'
keepalive = 600
usepty = 0
umask = None
maxdelay = 300
allow_shutdown = None
 
s = BuildSlave(buildmaster_host, port, slavename, passwd, basedir,
              keepalive, usepty, umask=umask, maxdelay=maxdelay,
              allow_shutdown=allow_shutdown)
s.setServiceParent(application)
</syntaxhighlight>


== Space Requirements ==
== Space Requirements ==
Line 134: Line 101:
If the directory containing the build and ccache directories doesn't have sufficient space, then these directory names may point to a separate partition that does have enough space available.
If the directory containing the build and ccache directories doesn't have sufficient space, then these directory names may point to a separate partition that does have enough space available.


== Starting the Slave ==
= External links =
 
With everything in place, you can run the slave server with the following commands (assuming that your <code>basedir</code> is {{Path|/var/lib/buildbot/slaves/debian-x86_64}}).


cd ~buildbot/slaves
* [https://hg.octave.org/octave-buildbot/ Buildbot configuration repository] for http://buildbot.octave.org:8010
sudo -u buildbot buildslave start debian-x86_64


[[Category:Development]]
[[Category:Building]]
Anonymous user

Navigation menu