Continuous Build

We are using Buildbot to build and test the current development version of Octave on multiple systems in a number of different configurations.

The current status of the builds may be found at http://buildbot.octave.org:8010/#/waterfall.

Systems and Configurations

The following systems and configurations are currently covered for Octave builds:

Builder ID	Hg Version	System	Compiler	Build Options	Frequency
clang-3.8-debian	default	Debian Testing	Clang 3.8	Disable GraphicsMagick	Any Change
clang-4.0-debian	default	Debian Testing	Clang 4.0		Any Change
clang-5.0-debian	default	Debian Testing	Clang 5.0		Any Change
clang-fedora	default	Fedora 25	Clang (system default)		Any Change
clang-osx	default	OS X	Clang		Any Change
gcc-6-debian	default	Debian Testing	GCC 6		Any Change
gcc-7-debian	default	Debian Testing	GCC 7		Any Change
gcc-7-lto-debian	default	Debian Testing	GCC (system default)		Any Change
gcc-fedora	default	Fedora 25	GCC (system default)		Any Change
gcc-lto-fedora	default	Fedora 25	GCC (system default)	Enable link time optimization	Any Change
no-extras-debian	default	Debian Testing	GCC (system default)	Disable all optional dependencies	Any Change

And for mxe-octave:

Builder ID	Hg Version	Build System	Host System	Compiler	Build Options	Frequency
mxe-native-all-on-debian	default	Debian Testing	Debian	GCC (system default)	GNU Linux, build all dependencies	Daily
mxe-native-on-debian	default	Debian Testing	Debian	GCC (system default)	GNU Linux, use system compiler, fontconfig, and X11 libraries	Daily
w32-on-debian	default	Debian Testing	Windows	GCC (mxe-octave default)	Windows 32	Daily
w32-stable-on-debian	stable	Debian Testing	Windows	GCC (mxe-octave default)	Windows 32	Daily
w64-32-on-debian	default	Debian Testing	Windows	GCC (mxe-octave default)	Windows 64	Daily
w64-32-stable-on-debian	stable	Debian Testing	Windows	GCC (mxe-octave default)	Windows 64	Daily
w64-64-on-debian	default	Debian Testing	Windows	GCC (mxe-octave default)	Windows 64, 64-bit indexing	Daily

Build Slave Configuration

To run a build slave for Octave, you must do the following:

• Contact the Octave Maintainers mailing list to let us know that you wish to provide a system to use as a build slave.
• Install buildbot. Packages exist for most distributions. See the buildbot docs for other options. You should create a separate user account with no special privileges that will run buildbot.
• Create a configuration file (see below).
• Run buildbot on the slave system, preferably by starting it automatically when your system boots. It should be running with the buildbot user ID.

You may also want to set up ccache to work with buildbot (strongly recommended to speed up builds). If you create a directory ~/buildbot/bin, it will be added to the execution PATH when buildbot runs commands on the slave. This directory can have symbolic links like the following:

lrwxrwxrwx 1 buildbot buildbot 15 Aug 26 11:39 gcc -> /usr/bin/ccache
lrwxrwxrwx 1 buildbot buildbot 15 Aug 26 11:40 cc -> /usr/bin/ccache
lrwxrwxrwx 1 buildbot buildbot 15 Aug 26 11:40 c++ -> /usr/bin/ccache
lrwxrwxrwx 1 buildbot buildbot 15 Aug 31 23:46 gfortran -> /usr/bin/ccache

They should point to the actual location of ccache if it is not in /usr/bin.

Your system may be behind a firewall. It does not have to have a distinct public IP address.

Sample Slave Configuration File

You must edit the settings for basedir, slavename, and password. The basedir should be the absolute and fully expanded name of the directory containing the configuration file. For example, if the home directory for the buildbot user is /var/lib/buildbot and your slavename is set to 'debian-x86_64' , then basedir might be '/var/lib/buildbot/slaves/debian-x86_64'. The password should not be anything valuable. You'll be asked to provide the slavename and password so that they may be added to the master configuration on buildbot.octave.org. Do not post the password to the octave-maintainers mailing list.

import os

from buildslave.bot import BuildSlave
from twisted.application import service

basedir = '/PATH/TO/DIRECTORY/CONTAINING/THIS/CONFIGURATION/FILE'
rotateLength = 10000000
maxRotatedFiles = 10

# if this is a relocatable tac file, get the directory containing the TAC
if basedir == '.':
    import os.path
    basedir = os.path.abspath(os.path.dirname(__file__))

# note: this line is matched against to check that this is a buildslave
# directory; do not edit it.
application = service.Application('buildslave')

try:
  from twisted.python.logfile import LogFile
  from twisted.python.log import ILogObserver, FileLogObserver
  logfile = LogFile.fromFullPath(os.path.join(basedir, "twistd.log"), rotateLength=rotateLength,
                                 maxRotatedFiles=maxRotatedFiles)
  application.setComponent(ILogObserver, FileLogObserver(logfile).emit)
except ImportError:
  # probably not yet twisted 8.2.0 and beyond, can't set log yet
  pass

buildmaster_host = 'buildbot.octave.org'
port = 9989
slavename = 'SLAVE-NAME'
passwd = 'PASSWORD'
keepalive = 600
usepty = 0
umask = None
maxdelay = 300
allow_shutdown = None

s = BuildSlave(buildmaster_host, port, slavename, passwd, basedir,
               keepalive, usepty, umask=umask, maxdelay=maxdelay,
               allow_shutdown=allow_shutdown)
s.setServiceParent(application)

Space Requirements

Building Octave takes a significant amount of disk space. With debugging symbols, you may need several GB for each build, plus room for ccache (possibly 50GB) if you use it. If you use a cache size that is larger than the default, you'll need to specify that in the .ccache/ccache.conf file using a line like

max_size = 50G

If the directory containing the build and ccache directories doesn't have sufficient space, then these directory names may point to a separate partition that does have enough space available.

Starting the Slave

With everything in place, you can run the slave server with the following commands (assuming that your basedir is '/var/lib/buildbot/slaves/debian-x86_64').

cd ~buildbot/slaves
sudo -u buildbot buildslave start debian-x86_64

If successful, buildbot will respond with

... FIXME ...